FeedGipeda.TaskScheduler

A task scheduler implementation based on http://haskell-distributed.github.io/tutorials/4ch.html#building-a-task-queue. It is however flexible in the number of active slaves, which are discovered via the SimpleLocalnet backend of cloud haskell. Also, failures are automatically retried.

Documentation

type Task a = (Static (SerializableDict a), Closure (Process a), Maybe a -> IO ())

A Task contains all information to execute a process on a remote node and finalize the result with a continuation. It consists of:

• A static pointer for the slave to find the entry point of task to execute
• The state in which to execute the task
• A continuation to call with the result of the (possibly timed out) task

start :: forall a r. Serializable a => IO (Task a) -> Process ()

Spawn the task queue on the local node and start to discover slave nodes. Tasks which don't finish within timeout return Nothing.

The consumer tries to keep the task list short and awaits new task items only when there are idle slaves. This way, upstream can react more timely to changes in task items priority (e.g. when a newer commit comes in or to interleave commits of a new repository).

work

Arguments

:: String

Host name of the local node, e.g. its IP address.

-> String

Port number of the local node.

-> NodeId

NodeId of the master node.

-> RemoteTable

-> IO ()

Register as a slave node and request tasks from the master node. Blocks. Slave discovery is done in a P2P fashion. We have a star topology with the master node at the center.