HdkOnNativeDataframePartition

Public API

class modin.experimental.core.execution.native.implementations.hdk_on_native.partitioning.partition.HdkOnNativeDataframePartition(data: Union[DbTable, Table, DataFrame])

A partition of HdkOnNativeDataframe frame.

Class holds either a DbTable or pandas.DataFrame or pyarrow.Table.

Parameters:

data (DbTable or pandas.DataFrame or pyarrow.Table) – Partition data in either pandas or PyArrow format.

_data

Partition data in either pandas or PyArrow format.

Type:

DbTable or pandas.DataFrame or pyarrow.Table

_length_cache

Length of the partition.

Type:

int

_width_cache

Width of the partition.

Type:

int

get(to_arrow: bool = False) → Union[DbTable, DataFrame, Table]

Get partition data.

Parameters:

to_arrow (bool, default: False) – Convert the data to pyarrow.Table.

Return type:

DbTable or pandas.DataFrame or pyarrow.Table

insert(idx: int, name: str, value: Union[ExtensionArray, ndarray, Index, Series])

Insert column into this raw partition.

Parameters:

• idx (int) –

• name (str) –

• value (AnyArrayLike) –

Return type:

tuple of HdkOnNativeDataframePartition, dtype

classmethod put(obj)

Create partition from DbTable or pandas.DataFrame or pyarrow.Table.

Parameters:

obj (DbTable or pandas.DataFrame or pyarrow.Table) – Source frame.

Returns:

The new partition.

Return type:

HdkOnNativeDataframePartition

property raw

True if the partition contains a raw data.

The raw data is either pandas.DataFrame or pyarrow.Table.

Return type:

bool

to_numpy(**kwargs)

Transform to NumPy format.

Parameters:

**kwargs (dict) – Additional keyword arguments to be passed in to_numpy.

Return type:

np.ndarray

to_pandas()

Transform to pandas format.

Return type:

pandas.DataFrame