faktory: Faktory Worker for Haskell

[ library, mit, network, program ] [ Propose Tags ]

Haskell client and worker process for the Faktory background job server.

Architecture overview

|                        +--------------------+
|                        |                    |
|                        |     Faktory        |
|                        |     Server         |
|         +---------->>>>|                    +>>>>--------+
|         |              |                    |            |
|         |              |                    |            |
|         |              +--------------------+            |
| +-----------------+                            +-------------------+
| |                 |                            |                   |
| |    Client       |                            |     Worker        |
| |    pushes       |                            |     pulls         |
| |     jobs        |                            |      jobs         |
| |                 |                            |                   |
| |                 |                            |                   |
| +-----------------+                            +-------------------+

This package contains only the Client and Worker.


[Skip to Readme]
Versions 1.0.0.0
Change log CHANGELOG.md
Dependencies aeson (>=1.3 && <2), aeson-casing (>=0.1 && <1), base (>=4.11 && <5), bytestring (>=0.1 && <1), connection (>=0.2 && <1), cryptonite (>=0.2 && <1), faktory, megaparsec (>=6.5 && <7), memory (>=0.1 && <1), network (>=2.6 && <3), random (>=1.1 && <2), safe-exceptions (>=0.1 && <1), scanner (>=0.2 && <1), text (>=1.2 && <2), time (>=1.8 && <2), unix (>=2.7 && <3) [details]
License MIT
Copyright 2018 Freckle Education
Author Freckle Engineering
Maintainer engineering@freckle.com
Category Network
Home page https://github.com/frontrowed/faktory_worker_haskell#readme
Bug tracker https://github.com/frontrowed/faktory_worker_haskell/issues
Source repo head: git clone https://github.com/frontrowed/faktory_worker_haskell
Uploaded by PatrickBrisbin at Wed Dec 5 20:23:23 UTC 2018
Distributions NixOS:1.0.0.0
Executables faktory-example-producer, faktory-example-consumer
Downloads 29 total (29 in the last 30 days)
Rating (no votes yet) [estimated by rule of succession]
Your Rating
  • λ
  • λ
  • λ
Status Docs available [build log]
Last success reported on 2018-12-05 [all 1 reports]
Hackage Matrix CI

Modules

[Index] [Quick Jump]

Downloads

Maintainer's Corner

For package maintainers and hackage trustees


Readme for faktory-1.0.0.0

[back to package description]

faktory_worker_haskell

CircleCI

Haskell client and worker process for the Faktory background job server.

Architecture overview from Ruby client README:

                       +--------------------+
                       |                    |
                       |     Faktory        |
                       |     Server         |
        +---------->>>>|                    +>>>>--------+
        |              |                    |            |
        |              |                    |            |
        |              +--------------------+            |
+-----------------+                            +-------------------+
|                 |                            |                   |
|    Client       |                            |     Worker        |
|    pushes       |                            |     pulls         |
|     jobs        |                            |      jobs         |
|                 |                            |                   |
|                 |                            |                   |
+-----------------+                            +-------------------+
  • Client - an API any process can use to push jobs to the Faktory server.
  • Worker - a process that pulls jobs from Faktory and executes them.
  • Server - the Faktory daemon which stores background jobs in queues to be processed by Workers.

This package contains only the client and worker parts. The server part is here

Installation

TODO.

Documentation

See the wiki for more details.

Usage

<!-- ```haskell import Data.Aeson import Prelude import Faktory.Client import Faktory.Job import Faktory.Settings import Faktory.Worker import GHC.Generics

{- Don't actually run anything -} main :: IO () main = if True then pure () else (workerMain >> clientMain) workerMain :: IO () clientMain :: IO ()

-->

### Job

Any value can be a "Job" that is pushed and pulled to and from Faktory via its
`ToJSON` and `FromJSON` instances:

newtype MyJob = MyJob { myJobMessage :: String } deriving (Generic)

instance ToJSON MyJob instance FromJSON MyJob


### Worker

workerMain = do settings <- envSettings

runWorker settings $ \job -> -- Process your Job here putStrLn $ myJobMessage job

-- If any exception is thrown, the job will be marked as Failed in Faktory
-- and retried. Note: you will not otherwise hear about any such exceptions,
-- unless you catch-and-rethrow them yourself.

### Client

clientMain = do settings <- envSettings client <- newClient settings Nothing -- N.B. A WorkerId is not necessary if -- only pushing Jobs.

jobId <- perform mempty client $ MyJob "Hello world"

print jobId

closeClient client


### Configuration

When using `envSettings`, the following variables will be used:

- `FAKTORY_QUEUE`: the name of the queue to consume from. This is Worker-only,
  for `perform`, a non-default Queue should be given by the `queue` option
- `FAKTORY_PROVIDER`: the name of another environment variable where the
  connection string can be found. Defaults to `FAKTORY_URL`.
- `FAKTORY_URL` (or whatever you named in `FAKTORY_PROVIDER`): connection string
  to the Faktory server. Format is `tcp(+tls)://(:password@)host:port`. Defaults
  to `tcp://localhost:4719`.

## Examples

See the [examples](./examples). To run them:

1. Run a local Faktory server

   ```console
   docker run --rm \
     --publish 7419:7419 \
     --publish 7420:7420 \
     contribsys/faktory
   ```

1. Run the consumer example

   ```console
   % stack exec faktory-example-consumer
   Starting consumer loop
   ```

   (Assumes you've built the project.)

1. Submit a Job through the producer example

   ```console
   % stack exec faktory-example-producer hello world
   Pushed job: "ljcjlbexbgun"
   ```

   *NOTE*: if you submit "BOOM" as a Job, the processing loop will raise an
   exception, so you can see how a Failed Job looks in Faktory.

1. See that your Job was processed back in the consumer

   ```console
   % stack exec faktory-example-consumer
   Starting consumer loop
   hello world
   ```

## Development & Tests

stack build --dependencies-only --test --no-run-tests stack build --pedantic --test --no-run-tests stack build --pedantic --test


*NOTE*: `FactorySpec` requires a local Faktory server is running, and it will
flush all Jobs from this server as part of running the tests.

---

[CHANGELOG](./CHANGELOG.md) | [LICENSE](./LICENSE)