Spark.Core.Internal.Joins

Exposes some of Spark's joining algorithms.

Documentation

join :: Column ref1 key -> Column ref1 value1 -> Column ref2 key -> Column ref2 value2 -> Dataset (key, value1, value2)

Standard (inner) join on two sets of data.

join' :: DynColumn -> DynColumn -> DynColumn -> DynColumn -> DataFrame

Untyped version of the standard join.

joinInner :: Column ref1 key -> Column ref1 value1 -> Column ref2 key -> Column ref2 value2 -> Dataset (key, value1, value2)

Explicit inner join.

joinInner' :: DynColumn -> DynColumn -> DynColumn -> DynColumn -> DataFrame

Untyped version of the inner join.

joinObs :: HasCallStack => Column ref val -> LocalData val' -> Dataset (val, val')

Broadcasts an observable alongside a dataset to make it available as an extra column.

joinObs' :: DynColumn -> LocalFrame -> DataFrame

Broadcasts an observable along side a dataset to make it available as an extra column.

The resulting dataframe has 2 columns: - one column called values - one column called broadcast

Note: this is a low-level operation. Users may want to use broadcastObs instead.