SciFlow: Scientific workflow management system

[ control, library, mit ] [ Propose Tags ]

SciFlow is to help programmers design complex workflows with ease.


[Skip to Readme]
Versions [faq] 0.1.0, 0.2.0, 0.3.0, 0.4.0, 0.4.1, 0.5.0, 0.5.1, 0.5.3.1, 0.6.0, 0.6.1, 0.6.2, 0.7.0
Dependencies base (>=4.0 && <5.0), bytestring, containers, data-default-class, fgl, graphviz, lens (>=4.0), mtl, optparse-applicative, shelly, split, template-haskell, text, th-lift, transformers, yaml [details]
License MIT
Copyright (c) 2016 Kai Zhang
Author Kai Zhang
Maintainer kai@kzhang.org
Category Control
Source repo head: git clone https://github.com/kaizhang/SciFlow.git
Uploaded by kaizhang at 2016-06-03T22:03:57Z
Distributions NixOS:0.7.0
Downloads 7045 total (10 in the last 30 days)
Rating (no votes yet) [estimated by Bayesian average]
Your Rating
  • λ
  • λ
  • λ
Status Hackage Matrix CI
Docs available [build log]
Last success reported on 2016-06-03 [all 1 reports]

Modules

[Index]

Flags

NameDescriptionDefaultType
debug

Enable debug support

DisabledAutomatic

Use -f <flag> to enable a flag, or -f -<flag> to disable that flag. More info

Downloads

Maintainer's Corner

For package maintainers and hackage trustees


Readme for SciFlow-0.4.1

[back to package description]

Scientific workflow management system

A scientific workflow is a series of computational steps which usually can be presented as a Directed Acyclic Graph (DAG).

SciFlow is to help programmers design complex workflows with ease. Here is a trivial example. (Since we use template haskell, we need to divide this small program into two parts.)

---------------------------------------------------
-- File 1: MyModule.hs
---------------------------------------------------
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE TemplateHaskell #-}
module Functions
    (builder) where

import Control.Lens ((^.), (.=))
import qualified Data.Text as T
import Shelly hiding (FilePath)
import Text.Printf (printf)

import Scientific.Workflow

create :: () -> IO FilePath
create _ = do
    writeFile "hello.txt" "hello world"
    return "hello.txt"

countWords :: FilePath -> IO Int
countWords fl = do
    content <- readFile fl
    return $ length $ words content

countChars :: FilePath -> IO Int
countChars fl = do
    content <- readFile fl
    return $ sum $ map length $ words content

output :: (Int, Int) -> IO Bool
output (ws, cs) = do
    putStrLn $ printf "Number of words: %d" ws
    putStrLn $ printf "Number of characters: %d" cs
    return True

cleanUp :: (Bool, FilePath) -> IO ()
cleanUp (toBeRemoved, fl) = if toBeRemoved
    then shelly $ rm $ fromText $ T.pack fl
    else return ()

-- builder monad
builder :: Builder ()
builder = do
    node "step0" 'create $ label .= "write something to a file"
    node "step1" 'countWords $ label .= "word count"
    node "step2" 'countChars $ label .= "character count"
    node "step3" 'output $ label .= "print"
    node "step4" 'cleanUp $ label .= "remove the file"

    ["step0"] ~> "step1"
    ["step0"] ~> "step2"
    ["step1", "step2"] ~> "step3"
    ["step3", "step0"] ~> "step4"

---------------------------------------------------
-- File 2: main.hs
---------------------------------------------------
{-# LANGUAGE TemplateHaskell #-}

import System.Environment

import qualified Functions as F
import qualified Data.Text.Lazy.IO as T

import Scientific.Workflow
import Scientific.Workflow.Visualize

buildWorkflow "wf" F.builder

main :: IO ()
main = do
    (cmd:args) <- getArgs
    case cmd of
        "run" -> runWorkflow wf def
        "view" -> T.putStrLn $ renderBuilder F.builder

The workflow can be visualized by running runghc main.hs view | dot -Tpng > example.png.

example

To run the workflow, simply type runghc main.hs run. The program will create a directory to store results of each step. If being terminated prematurely, the program will use the saved data to continue from the last step.