CalciteBaseNode#

Public API#

class modin.experimental.core.execution.native.implementations.hdk_on_native.calcite_algebra.CalciteBaseNode(relOp)#

A base class for a Calcite computation sequence node.

Calcite nodes are not combined into a tree but usually stored in a sequence which works similar to a stack machine: the result of the previous operation is an implicit operand of the current one. Input nodes also can be referenced directly via its unique ID number.

Calcite nodes structure is based on a JSON representation used by HDK for parsed queries serialization/deserialization for interactions with a Calcite server. Currently, this format is internal and is not a part of public API. It’s not documented and can be modified in an incompatible way in the future.

Parameters

relOp (str) – An operation name.

id#

Id of the node. Should be unique within a single query.

Type

int

relOp#

Operation name.

Type

str

classmethod reset_id()#

Reset ID to be used for the next new node to 0.

Can be used to have a zero-based numbering for each generated query.

CalciteScanNode#

Public API#

class modin.experimental.core.execution.native.implementations.hdk_on_native.calcite_algebra.CalciteScanNode(modin_frame)#

A node to represent a scan operation.

Scan operation can only be applied to physical tables.

Parameters

modin_frame (HdkOnNativeDataframe) – A frame to scan. The frame should have a materialized table in HDK.

table#

A list holding a database name and a table name.

Type

list of str

fieldNames#

A list of columns to include into the scan.

Type

list of str

inputs#

An empty list existing for the sake of serialization simplicity. Has no meaning but is expected by HDK deserializer.

Type

list

CalciteProjectionNode#

Public API#

class modin.experimental.core.execution.native.implementations.hdk_on_native.calcite_algebra.CalciteProjectionNode(fields, exprs)#

A node to represent a projection operation.

Parameters

  • fields (list of str) – Output column names.

  • exprs (list of BaseExpr) – Output column expressions.

fields#

A list of output columns.

Type

list of str

exprs#

A list of expressions describing how output columns are computed. Order of expression follows fields order.

Type

list of BaseExpr

CalciteFilterNode#

Public API#

class modin.experimental.core.execution.native.implementations.hdk_on_native.calcite_algebra.CalciteFilterNode(condition)#

A node to represent a filter operation.

Parameters

condition (BaseExpr) – A filtering condition.

condition#

A filter to apply.

Type

BaseExpr

CalciteAggregateNode#

Public API#

class modin.experimental.core.execution.native.implementations.hdk_on_native.calcite_algebra.CalciteAggregateNode(fields, group, aggs)#

A node to represent an aggregate operation.

Parameters

  • fields (list of str) – Output field names.

  • group (list of CalciteInputIdxExpr) – Group key columns.

  • aggs (list of BaseExpr) – Aggregates to compute.

fields#

Output field names.

Type

list of str

group#

Group key columns.

Type

list of CalciteInputIdxExpr

aggs#

Aggregates to compute.

Type

list of BaseExpr

CalciteCollation#

Public API#

class modin.experimental.core.execution.native.implementations.hdk_on_native.calcite_algebra.CalciteCollation(field, dir='ASCENDING', nulls='LAST')#

A structure to describe sorting order.

Parameters

  • field (CalciteInputIdxExpr) – A column to sort by.

  • dir ({"ASCENDING", "DESCENDING"}, default: "ASCENDING") – A sort order.

  • nulls ({"LAST", "FIRST"}, default: "LAST") – NULLs position after the sort.

field#

A column to sort by.

Type

CalciteInputIdxExpr

dir#

A sort order.

Type

{“ASCENDING”, “DESCENDING”}

nulls#

NULLs position after the sort.

Type

{“LAST”, “FIRST”}

CalciteSortNode#

Public API#

class modin.experimental.core.execution.native.implementations.hdk_on_native.calcite_algebra.CalciteSortNode(collation)#

A node to represent a sort operation.

Parameters

collation (list of CalciteCollation) – Sort keys.

collation#

Sort keys.

Type

list of CalciteCollation

CalciteJoinNode#

Public API#

class modin.experimental.core.execution.native.implementations.hdk_on_native.calcite_algebra.CalciteJoinNode(left_id, right_id, how, condition)#

A node to represent a join operation.

Parameters

  • left_id (int) – ID of the left join operand.

  • right_id (int) – ID of the right join operand.

  • how (str) – Type of the join.

  • condition (BaseExpr) – Join condition.

inputs#

IDs of the left and the right operands of the join.

Type

list of int

joinType#

Type of the join.

Type

str

condition#

Join condition.

Type

BaseExpr

CalciteUnionNode#

Public API#

class modin.experimental.core.execution.native.implementations.hdk_on_native.calcite_algebra.CalciteUnionNode(inputs, all)#

A node to represent a union operation.

Parameters

  • inputs (list of int) – Input frame IDs.

  • all (bool) – True for UNION ALL operation.

inputs#

Input frame IDs.

Type

list of int

all#

True for UNION ALL operation.

Type

bool

CalciteInputRefExpr#

Public API#

class modin.experimental.core.execution.native.implementations.hdk_on_native.calcite_algebra.CalciteInputRefExpr(idx)#

Calcite version of input column reference.

Calcite translation should replace all InputRefExpr.

Calcite references columns by their indexes (positions in input table). If there are multiple input tables for Calcite node, then a position in a concatenated list of all columns is used.

Parameters

idx (int) – Input column index.

input#

Input column index.

Type

int

copy()#

Make a shallow copy of the expression.

Return type

CalciteInputRefExpr

CalciteInputIdxExpr#

Public API#

class modin.experimental.core.execution.native.implementations.hdk_on_native.calcite_algebra.CalciteInputIdxExpr(idx)#

Basically the same as CalciteInputRefExpr but with a different serialization.

Parameters

idx (int) – Input column index.

input#

Input column index.

Type

int

copy()#

Make a shallow copy of the expression.

Return type

CalciteInputIdxExpr