HdkOnNativeDataframePartition#

Public API#

class modin.experimental.core.execution.native.implementations.hdk_on_native.partitioning.partition.HdkOnNativeDataframePartition(frame_id=None, pandas_df=None, arrow_table=None, length=None, width=None)#

A partition of HdkOnNativeDataframe frame.

Class holds either a pandas.DataFrame or pyarrow.Table.

Parameters

  • frame_id (str, optional) – A corresponding HDK table name or None.

  • pandas_df (pandas.DataFrame, optional) – Partition data in pandas format.

  • arrow_table (pyarrow.Table, optional) – Partition data in Arrow format.

  • length (int, optional) – Length of the partition.

  • width (int, optional) – Width of the partition.

frame_id#

A corresponding HDK table name if partition was imported into HDK. Otherwise None.

Type

str

pandas_df#

Partition data in pandas format.

Type

pandas.DataFrame, optional

arrow_table#

Partition data in Arrow format. None for partitions holding pandas.DataFrame.

Type

pyarrow.Table

_length_cache#

Length of the partition.

Type

int

_width_cache#

Width of the partition.

Type

int

get()#

Get partition data.

Return type

pandas.DataFrame or pyarrow.Table

classmethod put(obj)#

Create partition from pandas.DataFrame or pandas.Series.

Parameters

obj (pandas.Series or pandas.DataFrame) – Source frame.

Returns

The new partition.

Return type

HdkOnNativeDataframePartition

classmethod put_arrow(obj)#

Create partition from pyarrow.Table.

Parameters

obj (pyarrow.Table) – Source table.

Returns

The new partition.

Return type

HdkOnNativeDataframePartition

to_numpy(**kwargs)#

Transform to NumPy format.

Parameters

**kwargs (dict) – Additional keyword arguments to be passed in to_numpy.

Return type

np.ndarray

to_pandas()#

Transform to pandas format.

Return type

pandas.DataFrame

wait()#

Wait until the partition data is ready for use.

Returns

The partition that is ready to be used.

Return type

pandas.DataFrame