Safe Haskell | None |
---|---|
Language | Haskell2010 |
Kafka is a module for _writing_ to Kafka
See Kafka.Worker for the basic building blocks of a CLI app that will poll & process kafka messages
Synopsis
- data Handler
- data Settings
- decoder :: Decoder Settings
- handler :: Settings -> Acquire Handler
- data Msg
- emptyMsg :: Text -> Msg
- addPayload :: (FromJSON a, ToJSON a) => a -> Msg -> Msg
- addKey :: Text -> Msg -> Msg
- sendAsync :: Handler -> Task Never () -> Msg -> Task Text ()
- sendSync :: Handler -> Msg -> Task Text ()
- topic :: Msg -> Text
- payload :: FromJSON a => Msg -> Maybe a
- key :: Msg -> Maybe Text
Setup
decoder :: Decoder Settings Source #
decodes Settings from environmental variables KAFKA_BROKER_ADDRESSES=localhost:9092 # comma delimeted list KAFKA_LOG_LEVEL=Debug KAFKA_DELIVERY_TIMEOUT=120000 KAFKA_BATCH_SIZE=10000
handler :: Settings -> Acquire Handler Source #
Function for creating a Kafka handler.
See Settings
for potential customizations.
Creating messages
A message that can be written to Kafka
emptyMsg :: Text -> Msg Source #
Creates a Kafka-writable message for a topic.
msg = emptyMsg "groceries" |> addPayload "broccoli" |> addKey "vegetables"
Sending messags
sendAsync :: Handler -> Task Never () -> Msg -> Task Text () Source #
sends messages asynchronously with to Kafka
This is the recommended approach for high throughput. The C++ library behind hte scenes, librdkafka, will batch messages together.
sendSync :: Handler -> Msg -> Task Text () Source #
sends messages synchronously with to Kafka
This can have a large negative impact on throughput. Use sparingly!
Reading messages
The topic of a message. This function might sometimes be useful in tests.