OmnisciOnNativeFramePartitionManager

Public API

class modin.experimental.engines.omnisci_on_native.frame.partition_manager.OmnisciOnNativeFramePartitionManager

Frame manager for OmnisciOnNativeFrame.

This class handles several features of OmnisciOnNativeFrame:

  • frame always has a single partition

  • frame cannot process some data types

  • frame has to use mangling for index labels

  • frame uses OmniSci backend for execution

classmethod from_arrow(at, return_dims=False, unsupported_cols=None)

Build frame from Arrow table.

Parameters

  • at (pyarrow.Table) – Input table.

  • return_dims (bool, default: False) – True to include dimensions into returned tuple.

  • unsupported_cols (list of str, optional) – List of columns holding unsupported data. If None then check all columns to compute the list.

Returns

Tuple holding array of partitions, list of columns with unsupported data and optionally partitions’ dimensions.

Return type

tuple

classmethod from_pandas(df, return_dims=False)

Create OmnisciOnNativeFrame from pandas.DataFrame.

Parameters

  • df (pandas.DataFrame) – Source frame.

  • return_dims (bool, default: False) – Include reulsintg dimensions into the returned value.

Returns

Tuple holding array of partitions, list of columns with unsupported data and optionally partitions’ dimensions.

Return type

tuple

classmethod run_exec_plan(plan, index_cols, dtypes, columns)

Run execution plan in OmniSci backend to materialize frame.

Parameters

  • plan (DFAlgNode) – A root of an execution plan tree.

  • index_cols (list of str) – A list of index columns.

  • dtypes (pandas.Index) – Column data types.

  • columns (list of str) – A frame column names.

Returns

Created frame’s partitions.

Return type

np.array