System.Periodic

This module contains a very basic periodic job processor backed by Redis, suitable to be run by clusters of machines (jobs will be run at least once per scheduled time, and will not run more than once unless the (configurable) timeout is reached, at which point retry occurs. There is also no guarantee that distribution of jobs across machines will be fair). It depends upon Redis not losing data once it has acknowledged it, and guaranteeing the atomicity that is specified for commands like EVAL (ie, that if you do several things within an EVAL, they will all happen or none will happen). Nothing has been tested with Redis clusters (and it likely will not work).

An example use is the following (provided in repository), noting that we create two threads to process jobs, and could run this entire executable many times and the jobs would only run at the scheduled time.

import           Control.Monad      (forever)
import           System.Periodic
import           Control.Concurrent      (threadDelay, forkIO)
import qualified Data.Text.IO            as T
import qualified Database.Redis          as R
main = do rconn <- R.connect R.defaultConnectInfo
          scheduler <- create "default" rconn (CheckInterval (Seconds 1)) (LockTimeout (Seconds 1000)) (T.putStrLn)
          addTask scheduler "print-hello-job" (Every (Seconds 100)) (T.putStrLn "hello")
          addTask scheduler "print-bye-job" (Every (Seconds 10)) (T.putStrLn "bye")
          forkIO (run scheduler)
          forkIO (run scheduler)
          forever (threadDelay 1000000)

Types

newtype Time

Time from midnight UTC

Constructors

Time DiffTime

newtype Seconds

Constructors

Seconds Int

data Period

When the job should run - either at a particular offset from midnight UTC, or every N seconds.

Constructors

Daily Time

Every Seconds

newtype Name

The name of the scheduler. This, when combined with the task name, should be unique on a given Redis server.

Constructors

Name Text

newtype CheckInterval

How frequently we should check if jobs need to get run. If all your jobs are infrequent (daily, or occurring hours apart), setting this to a high number (every minute, or more) decreases the number of queries to Redis.

Constructors

CheckInterval Seconds

newtype LockTimeout

How long a job that has been started and not marked as finished should take before we run it again. Note that if you put this number above the period, we will never retry the jobs (but they will still keep running every period).

Constructors

LockTimeout Seconds

type Logger = Text -> IO ()

A function where log messages are sent.

data Scheduler

Internal type representing the current tasks that are scheduled.

Managing Schedulers

create :: Name -> Connection -> CheckInterval -> LockTimeout -> (Text -> IO ()) -> IO Scheduler

This function creates a new scheduler, which can then have tasks scheduled in it. The tasks themselves are _not_ stored in Redis; it is only used to ensure we run the tasks at the right time (and not more than once across multiple machines). Note that this does not actually run any of the tasks - you must fork threads that call run.

run :: Scheduler -> IO ()

Start a scheduler thread. You must start at least one, but can start multiple threads if you have many tasks that take a long time to run.

destroy :: Scheduler -> IO ()

This clears out redis of any mention of tasks related to the scheduler. It isn't necessary to do in normal scenarios.

Scheduling Tasks

addTask :: Scheduler -> Text -> Period -> IO () -> IO ()

Add a new task to a given scheduler, to be run at the specified period. Note that the name, when combined with the scheduler name, should be unique for the given redis database, or else only one of the jobs with the same name will be run.