Spark.Core.Internal.ContextStructures

Documentation

data SparkSessionConf

The configuration of a remote spark session in krapsh.

Constructors

SparkSessionConf

Fields

• confEndPoint :: !Text

  The URL of the end point.

• confPort :: !Int

  The port used to configure the end point.

• confPollingIntervalMillis :: !Int

  (internal) the polling interval

• confRequestedSessionName :: !Text

  (optional) the requested name of the session. This name must obey a number of rules: - it must consist in alphanumerical and -,_: [a-zA-Z0-9-_] - if it already exists on the server, it will be reconnected to

  The default value is "" (a new random context name will be chosen).

Instances

Show SparkSessionConf

Methods

showsPrec :: Int -> SparkSessionConf -> ShowS

show :: SparkSessionConf -> String

showList :: [SparkSessionConf] -> ShowS

data SparkSession

A session in Spark. Encapsualates all the state needed to communicate with Spark and to perfor some simple optimizations on the code.

Constructors

SparkSession

Fields

• ssConf :: SparkSessionConf
• ssId :: LocalSessionId
• ssCommandCounter :: Integer

Instances

Show SparkSession

Methods

showsPrec :: Int -> SparkSession -> ShowS

show :: SparkSession -> String

showList :: [SparkSession] -> ShowS

type SparkState a = LoggingT (StateT SparkSession IO) a

Represents the state of a session and accounts for the communication with the server.

type SparkStatePure x = State SparkSession x

type ComputeGraph = ComputeDag UntypedNode StructureEdge

internal

A graph of computations. This graph is a direct acyclic graph. Each node is associated to a global path.