Generic Ray-based members¶
Objects which are storage format agnostic but require specific Ray implementation
are placed in modin.core.execution.ray.generic
.
Their purpose is to implement certain parallel I/O operations and to serve as a foundation for building storage format specific objects:
GenericRayDataframePartitionManager
– implements parallelto_numpy()
.
- class modin.core.execution.ray.generic.io.io.RayIO¶
Base class for doing I/O operations over Ray.
- classmethod to_csv(qc, **kwargs)¶
Write records stored in the qc to a CSV file.
- Parameters
qc (BaseQueryCompiler) – The query compiler of the Modin dataframe that we want to run
to_csv
on.**kwargs (dict) – Parameters for
pandas.to_csv(**kwargs)
.
- classmethod to_parquet(qc, **kwargs)¶
Write a
DataFrame
to the binary parquet format.- Parameters
qc (BaseQueryCompiler) – The query compiler of the Modin dataframe that we want to run to_parquet on.
**kwargs (dict) – Parameters for pandas.to_parquet(**kwargs).
- classmethod to_sql(qc, **kwargs)¶
Write records stored in the qc to a SQL database.
- Parameters
qc (BaseQueryCompiler) – The query compiler of the Modin dataframe that we want to run
to_sql
on.**kwargs (dict) – Parameters for
pandas.to_sql(**kwargs)
.
- class modin.core.execution.ray.generic.partitioning.partition_manager.GenericRayDataframePartitionManager¶
The class implements the interface in PandasDataframePartitionManager.
- classmethod to_numpy(partitions, **kwargs)¶
Convert partitions into a NumPy array.
- Parameters
partitions (NumPy array) – A 2-D array of partitions to convert to local NumPy array.
**kwargs (dict) – Keyword arguments to pass to each partition
.to_numpy()
call.
- Returns
- Return type
NumPy array