PandasFramePartition¶
The class is base for any partition class of pandas backend and serves as the last level
on which operations that were conveyed from the partition manager are being performed on an
individual block partition.
The class provides an API that has to be overridden by child classes in order to manipulate on data and metadata they store.
The public API exposed by the children of this class is used in PandasFramePartitionManager.
The objects wrapped by the child classes are treated as immutable by PandasFramePartitionManager subclasses
and no logic for updating inplace.
Public API¶
- class modin.engines.base.frame.partition.PandasFramePartition¶
An abstract class that is base for any partition class of
pandasbackend.The class providing an API that has to be overridden by child classes.
- add_to_apply_calls(func, *args, **kwargs)¶
Add a function to the call queue.
- Parameters
func (callable) – Function to be added to the call queue.
*args (iterable) – Additional positional arguments to be passed in func.
**kwargs (dict) – Additional keyword arguments to be passed in func.
- Returns
New PandasFramePartition object with the function added to the call queue.
- Return type
Notes
This function will be executed when apply is called. It will be executed in the order inserted; apply’s func operates the last and return.
- apply(func, *args, **kwargs)¶
Apply a function to the object wrapped by this partition.
- Parameters
func (callable) – Function to apply.
*args (iterable) – Additional positional arguments to be passed in func.
**kwargs (dict) – Additional keyword arguments to be passed in func.
- Returns
New PandasFramePartition object.
- Return type
Notes
It is up to the implementation how kwargs are handled. They are an important part of many implementations. As of right now, they are not serialized.
- drain_call_queue()¶
Execute all operations stored in the call queue on the object wrapped by this partition.
- classmethod empty()¶
Create a new partition that wraps an empty pandas DataFrame.
- Returns
New PandasFramePartition object.
- Return type
- get()¶
Get the object wrapped by this partition.
- Returns
The object that was wrapped by this partition.
- Return type
object
Notes
This is the opposite of the classmethod put. E.g. if you assign x = PandasFramePartition.put(1), x.get() should always return 1.
- length()¶
Get the length of the object wrapped by this partition.
- Returns
The length of the object.
- Return type
int
- mask(row_indices, col_indices)¶
Lazily create a mask that extracts the indices provided.
- Parameters
row_indices (list-like, slice or label) – The indices for the rows to extract.
col_indices (list-like, slice or label) – The indices for the columns to extract.
- Returns
New PandasFramePartition object.
- Return type
- classmethod preprocess_func(func)¶
Preprocess a function before an apply call.
- Parameters
func (callable) – Function to preprocess.
- Returns
An object that can be accepted by apply.
- Return type
callable
Notes
This is a classmethod because the definition of how to preprocess should be class-wide. Also, we may want to use this before we deploy a preprocessed function to multiple PandasFramePartition objects.
- classmethod put(obj)¶
Put an object into a store and wrap it with partition object.
- Parameters
obj (object) – An object to be put.
- Returns
New PandasFramePartition object.
- Return type
- to_numpy(**kwargs)¶
Convert the object wrapped by this partition to a NumPy array.
- Parameters
**kwargs (dict) – Additional keyword arguments to be passed in to_numpy.
- Returns
- Return type
np.ndarray
Notes
If the underlying object is a pandas DataFrame, this will return a 2D NumPy array.
- to_pandas()¶
Convert the object wrapped by this partition to a pandas DataFrame.
- Returns
- Return type
Notes
If the underlying object is a pandas DataFrame, this will likely only need to call get.
- wait()¶
Wait for completion of computations on the object wrapped by the partition.
- width()¶
Get the width of the object wrapped by the partition.
- Returns
The width of the object.
- Return type
int