Scientific.Workflow.Internal.Builder

Documentation

node

Arguments

:: ToExpQ fun

=> PID

Node id

-> fun

Template Haskell expression representing functions with type a -> IO b.

-> State Attribute ()

Attribues

-> Builder ()

Declare an IO computational step.

node'

Arguments

:: ToExpQ fun

=> PID

-> fun

Template Haskell expression representing functions with type a -> b.

-> State Attribute ()

-> Builder ()

Declare a pure computational step.

nodeS

Arguments

:: ToExpQ fun

=> PID

-> fun

Template Haskell expression representing functions with type "a -> WorkflowConfig st b".

-> State Attribute ()

-> Builder ()

Declare a stateful computational step.

nodeP

Arguments

:: ToExpQ fun

=> Int

Batch size for parallel execution.

-> PID

-> fun

-> State Attribute ()

-> Builder ()

Declare an IO and parallel computational step. This will turn functions with type "a -> IO b" into functions with type "[a] -> IO [b]". And [a] will be processed in parallel with provided batch size. Note: Currently, parallelism is available only when "--remote" flag is on.

nodeP' :: ToExpQ fun => Int -> PID -> fun -> State Attribute () -> Builder ()

Same as nodeP but work with pure functions.

nodePS :: ToExpQ fun => Int -> PID -> fun -> State Attribute () -> Builder ()

Same as nodeP but work with stateful functions.

nodeSharedP

Arguments

:: ToExpQ fun

=> Int

-> PID

-> fun

Template Haskell expression representing functions with type ContextData context a -> IO b.

-> State Attribute ()

-> Builder ()

Similar to nodeP but work with inputs that are associated with a shared context. Turn ContextData context a -> IO b into ContextData context [a] -> IO [b].

nodeSharedP' :: ToExpQ fun => Int -> PID -> fun -> State Attribute () -> Builder ()

nodeSharedPS :: ToExpQ fun => Int -> PID -> fun -> State Attribute () -> Builder ()

link :: [PID] -> PID -> Builder ()

Declare the dependency between nodes. Example:

node' "step1" [| \() -> 1 :: Int |] $ return ()
node' "step2" [| \() -> 2 :: Int |] $ return ()
node' "step3" [| \(x, y) -> x * y |] $ return ()
link ["step1", "step2"] "step3"

(~>) :: [PID] -> PID -> Builder ()

(~>) = link.

path :: [PID] -> Builder ()

"path [a, b, c]" is equivalent to "link a b >> link b c"

namespace :: Text -> Builder () -> Builder ()

Add a prefix to IDs of nodes for a given builder, i.e., id becomes prefix_id.

buildWorkflow

Arguments

:: String

Name of the workflow

-> Builder ()

Builder

-> Q [Dec]

Build the workflow. This function will first create functions defined in the builder. These pieces will then be assembled to form a function that will execute each individual function in a correct order, named $name$.

buildWorkflowPart

Arguments

:: FilePath

Path to the db

-> String

Name of the workflow

-> Builder ()

Builder

-> Q [Dec]

Build only a part of the workflow that has not been executed. This is used during development for fast compliation.

mkDAG :: Builder () -> DAG

Contruct a DAG representing the workflow

mkProc :: (DBData a, DBData b, ToJSON config) => PID -> (a -> ProcState config b) -> Processor config a b