Safe Haskell | None |
---|---|
Language | Haskell2010 |
This module contains a very basic periodic job processor backed by Redis, suitable to be run by clusters of machines (jobs will be run at least once per scheduled time, and will not run more than once unless the (configurable) timeout is reached, at which point retry occurs. There is also no guarantee that distribution of jobs across machines will be fair). It depends upon Redis not losing data once it has acknowledged it, and guaranteeing the atomicity that is specified for commands like EVAL (ie, that if you do several things within an EVAL, they will all happen or none will happen). Nothing has been tested with Redis clusters (and it likely will not work).
An example use is the following (provided in repository), noting that we create two threads to process jobs, and could run this entire executable many times and the jobs would only run at the scheduled time.
import Control.Monad (forever) import System.Periodic import Control.Concurrent (threadDelay, forkIO) import qualified Data.Text.IO as T import qualified Database.Redis as R main = do rconn <- R.connect R.defaultConnectInfo scheduler <- create "default" rconn (CheckInterval (Seconds 1)) (LockTimeout (Seconds 1000)) (T.putStrLn) addTask scheduler "print-hello-job" (Every (Seconds 100)) (T.putStrLn "hello") addTask scheduler "print-bye-job" (Every (Seconds 10)) (T.putStrLn "bye") forkIO (run scheduler) forkIO (run scheduler) forever (threadDelay 1000000)
- newtype Time = Time DiffTime
- newtype Seconds = Seconds Int
- data Period
- newtype Name = Name Text
- newtype CheckInterval = CheckInterval Seconds
- newtype LockTimeout = LockTimeout Seconds
- type Logger = Text -> IO ()
- data Scheduler
- create :: Name -> Connection -> CheckInterval -> LockTimeout -> (Text -> IO ()) -> IO Scheduler
- run :: Scheduler -> IO ()
- destroy :: Scheduler -> IO ()
- addTask :: Scheduler -> Text -> Period -> IO () -> IO ()
Types
When the job should run - either at a particular offset from midnight UTC, or every N seconds.
The name of the scheduler. This, when combined with the task name, should be unique on a given Redis server.
newtype CheckInterval Source #
How frequently we should check if jobs need to get run. If all your jobs are infrequent (daily, or occurring hours apart), setting this to a high number (every minute, or more) decreases the number of queries to Redis.
newtype LockTimeout Source #
How long a job that has been started and not marked as finished should take before we run it again. Note that if you put this number above the period, we will never retry the jobs (but they will still keep running every period).
Managing Schedulers
create :: Name -> Connection -> CheckInterval -> LockTimeout -> (Text -> IO ()) -> IO Scheduler Source #
This function creates a new scheduler, which can then have tasks
scheduled in it. The tasks themselves are _not_ stored in Redis; it
is only used to ensure we run the tasks at the right time (and not
more than once across multiple machines). Note that this does not
actually run any of the tasks - you must fork threads that call
run
.
run :: Scheduler -> IO () Source #
Start a scheduler thread. You must start at least one, but can start multiple threads if you have many tasks that take a long time to run.
destroy :: Scheduler -> IO () Source #
This clears out redis of any mention of tasks related to the scheduler. It isn't necessary to do in normal scenarios.
Scheduling Tasks
addTask :: Scheduler -> Text -> Period -> IO () -> IO () Source #
Add a new task to a given scheduler, to be run at the specified period. Note that the name, when combined with the scheduler name, should be unique for the given redis database, or else only one of the jobs with the same name will be run.