PyArrow storage format """""""""""""""""""""" .. toctree:: :hidden: query_compiler parsers In general, PyArrow storage formats follow the flow of the pandas ones: query compiler contains an instance of Modin Dataframe, which is internally split into partitions. The main difference is that partitions contain PyArrow tables, instead of ``pandas.DataFrame``-s like with :doc:`pandas storage format `. To learn more about this approach please visit :doc:`PyArrowOnRay execution ` section. High-Level Module Overview '''''''''''''''''''''''''' This module houses submodules which are responsible for communication between the query compiler level and execution implementation level for PyArrow storage format: - :doc:`Query compiler ` is responsible for compiling efficient queries for :doc:`PyarrowOnRayDataframe `. - :doc:`Parsers ` are responsible for parsing data on workers during IO operations. .. note:: Currently the only one available PyArrow storage format factory is ``PyarrowOnRay`` which works in :doc:`experimental mode ` only.