Safe Haskell | None |
---|---|
Language | Haskell2010 |
This module defines session objects that act as entry points to spark.
There are two ways to interact with Spark: using an explicit state object, or using the default state object (interactive session).
While the interactive session is the most convenient, it should not be used for more than quick experimentations. Any complex code should use the SparkSession and SparkState objects.
- data SparkSessionConf = SparkSessionConf {}
- data SparkSession
- type SparkState a = SparkStateT IO a
- data SparkInteractiveException
- class FromSQL a
- defaultConf :: SparkSessionConf
- executeCommand1 :: forall a. FromSQL a => LocalData a -> SparkState (Try a)
- executeCommand1' :: UntypedLocalData -> SparkState (Try Cell)
- computationStats :: ComputationID -> SparkState BatchComputationResult
- createSparkSessionDef :: SparkSessionConf -> IO ()
- closeSparkSessionDef :: IO ()
- currentSessionDef :: IO (Maybe SparkSession)
- computationStatsDef :: ComputationID -> IO BatchComputationResult
- exec1Def :: FromSQL a => LocalData a -> IO a
- exec1Def' :: LocalFrame -> IO Cell
- execStateDef :: SparkState a -> IO a
Documentation
data SparkSessionConf Source #
The configuration of a remote spark session in Karps.
SparkSessionConf | |
|
data SparkSession Source #
A session in Spark. Encapsualates all the state needed to communicate with Spark and to perfor some simple optimizations on the code.
type SparkState a = SparkStateT IO a Source #
Represents the state of a session and accounts for the communication with the server.
data SparkInteractiveException Source #
The exception thrown when a request cannot be completed in an interactive session.
FromSQL Bool Source # | |
FromSQL Double Source # | |
FromSQL Int Source # | |
FromSQL Text Source # | |
FromSQL DataTypeRepr Source # | |
FromSQL DataTypeElementRepr Source # | |
FromSQL Cell Source # | |
FromSQL a => FromSQL [a] Source # | |
FromSQL a => FromSQL (Maybe a) Source # | |
(FromSQL a1, FromSQL a2) => FromSQL (a1, a2) Source # | |
defaultConf :: SparkSessionConf Source #
The default configuration if the Karps server is being run locally.
executeCommand1 :: forall a. FromSQL a => LocalData a -> SparkState (Try a) Source #
Executes a command: - performs the transforms and the optimizations in the pure state - sends the computation to the backend - waits for the terminal nodes to reach a final state - commits the final results to the state
If any failure is detected that is internal to Karps, it returns an error. If the error comes from an underlying library (http stack, programming failure), an exception may be thrown instead.
executeCommand1' :: UntypedLocalData -> SparkState (Try Cell) Source #
computationStats :: ComputationID -> SparkState BatchComputationResult Source #
Exposed for debugging
createSparkSessionDef :: SparkSessionConf -> IO () Source #
Creates a spark session that will be used as the default session.
If a session already exists, an exception will be thrown.
closeSparkSessionDef :: IO () Source #
Closes the default session. The default session is empty after this call completes.
NOTE: This does not currently clear up the resources! It is a stub implementation used in testing.
exec1Def :: FromSQL a => LocalData a -> IO a Source #
Executes a command using the default spark session.
This is the most unsafe way of running a command: it executes a command using the default spark session, and throws an exception if any error happens.
execStateDef :: SparkState a -> IO a Source #
Runs the computation described in the state transform, using the default Spark session.
Will throw an exception if no session currently exists.