hasql-queue: A PostgreSQL backed queue

[ bsd3, library, program, web ] [ Propose Tags ]

This module utilize PostgreSQL to implement a durable queue for efficently processing arbitrary payloads which can be represented as JSON.

Typically a producer would enqueue a new payload as part of larger database transaction

 createAccount userRecord = do
    'runDBTSerializable' $ do
       createUserDB userRecord
       'enqueueDB' "queue_schema" $ makeVerificationEmail userRecord

In another thread or process, the consumer would drain the queue.

  forever $ do
    -- Attempt get a payload or block until one is available
    payload <- lock "queue" conn

    -- Perform application specifc parsing of the payload value
    case fromJSON $ pValue payload of
      Success x -> sendEmail x -- Perform application specific processing
      Error err -> logErr err

    -- Remove the payload from future processing
    dequeue "queue" conn $ pId payload

To support multiple queues in the same database, the API expects a table name string
to determine which queue tables to use.

[Skip to Readme]

Modules

[Last Documentation]

  • Hasql
    • Queue
      • Hasql.Queue.IO
      • Hasql.Queue.Internal
      • Hasql.Queue.Migrate
      • Hasql.Queue.Session

Downloads

Maintainer's Corner

Package maintainers

For package maintainers and hackage trustees

Candidates

  • No Candidates
Versions [RSS] 1.0.1, 1.0.1.1, 1.2.0.0, 1.2.0.1, 1.2.0.2
Change log CHANGELOG.md
Dependencies aeson, async, base (>=4.7 && <5), base64-bytestring, bytestring, cryptohash-sha1, exceptions, hasql, hasql-notifications, hasql-queue, here, monad-control, random, resource-pool, stm, text, time, tmp-postgres, transformers [details]
License BSD-3-Clause
Copyright 2020 Jonathan Fischoff
Author Jonathan Fischoff
Maintainer jonathangfischoff@gmail.com
Category Web
Home page https://github.com/jfischoff/postgresql-queue#readme
Bug tracker https://github.com/jfischoff/postgresql-queue/issues
Source repo head: git clone https://github.com/jfischoff/postgresql-queue
Uploaded by JonathanFischoff at 2020-06-14T06:10:21Z
Distributions
Executables benchmark
Downloads 1073 total (24 in the last 30 days)
Rating (no votes yet) [estimated by Bayesian average]
Your Rating
  • λ
  • λ
  • λ
Status Docs not available [build log]
All reported builds failed as of 2020-06-14 [all 3 reports]

Readme for hasql-queue-1.0.1

[back to package description]

Travis CI Status

hasql-queue

This module utilizes PostgreSQL to implement a durable queue for efficently processing payloads.

Typically a producer would enqueue a new payload as part of larger database transaction

createAccount userRecord = transaction Serializable Write $ do
  createUser userRecord
  enqueueNotification "queue_channel" emailEncoder [makeVerificationEmail userRecord]

In another thread or process the consumer would drain the queue.

  -- Wait for a single new record and try to send the email 5 times for giving
  -- up and marking the payload as failed.
  forever $ withDequeue "queue_channel" conn emailDecoder 5 1 $
    mapM_ sendEmail

In the example above we used the Session API for enqueuing and the IO for dequeuing.

The Session API is useful for composing larger transactions and the IO utilizes PostgreSQL notifications to avoid polling.

Installation

stack install hasql-queue