OmnisciOnNativeDataframePartitionManager#

Public API#

class modin.experimental.core.execution.native.implementations.omnisci_on_native.partitioning.partition_manager.OmnisciOnNativeDataframePartitionManager#

Frame manager for OmnisciOnNativeDataframe.

This class handles several features of OmnisciOnNativeDataframe:

  • frame always has a single partition

  • frame cannot process some data types

  • frame has to use mangling for index labels

  • frame uses OmniSci storage format for execution

classmethod from_arrow(at, return_dims=False, unsupported_cols=None)#

Build frame from Arrow table.

Parameters

  • at (pyarrow.Table) – Input table.

  • return_dims (bool, default: False) – True to include dimensions into returned tuple.

  • unsupported_cols (list of str, optional) – List of columns holding unsupported data. If None then check all columns to compute the list.

Returns

Tuple holding array of partitions, list of columns with unsupported data and optionally partitions’ dimensions.

Return type

tuple

classmethod from_pandas(df, return_dims=False)#

Create OmnisciOnNativeDataframe from pandas.DataFrame.

Parameters

  • df (pandas.DataFrame) – Source frame.

  • return_dims (bool, default: False) – Include reulsintg dimensions into the returned value.

Returns

Tuple holding array of partitions, list of columns with unsupported data and optionally partitions’ dimensions.

Return type

tuple

classmethod run_exec_plan(plan, index_cols, dtypes, columns)#

Run execution plan in OmniSci storage format to materialize frame.

Parameters

  • plan (DFAlgNode) – A root of an execution plan tree.

  • index_cols (list of str) – A list of index columns.

  • dtypes (pandas.Index) – Column data types.

  • columns (list of str) – A frame column names.

Returns

Created frame’s partitions.

Return type

np.array