Logstash

A simple Logstash client.

Documentation

module Logstash.Connection

Running Logstash actions

class LogstashContext ctx where

A type class of types that provide Logstash connections.

Methods

runLogstash :: (MonadMask m, MonadUnliftIO m) => ctx -> RetryPolicyM m -> Integer -> (RetryStatus -> ReaderT LogstashConnection m a) -> m a

Instances

LogstashContext LogstashPool

Defined in Logstash

Methods

runLogstash :: (MonadMask m, MonadUnliftIO m) => LogstashPool -> RetryPolicyM m -> Integer -> (RetryStatus -> ReaderT LogstashConnection m a) -> m a

LogstashContext (Acquire LogstashConnection)

Defined in Logstash

Methods

runLogstash :: (MonadMask m, MonadUnliftIO m) => Acquire LogstashConnection -> RetryPolicyM m -> Integer -> (RetryStatus -> ReaderT LogstashConnection m a) -> m a

runLogstashConn :: (MonadMask m, MonadUnliftIO m) => Acquire LogstashConnection -> RetryPolicyM m -> Integer -> (RetryStatus -> ReaderT LogstashConnection m a) -> m a

runLogstashConn connectionAcquire computation runs computation using a Logstash connection produced by connectionAcquire.

runLogstashPool :: (MonadMask m, MonadUnliftIO m) => LogstashPool -> RetryPolicyM m -> Integer -> (RetryStatus -> ReaderT LogstashConnection m a) -> m a

runLogstashPool pool computation takes a LogstashConnection from pool and runs computation with it.

data LogstashQueueCfg ctx

Configurations for withLogstashQueue which control the general Logstash configuration as well as the size of the bounded queue and the number of worker threads.

Constructors

MkLogstashQueueCfg

Fields

• logstashQueueContext :: ctx

  The connection context for the worker threads.

• logstashQueueSize :: Int

  The size of the queue.

• logstashQueueWorkers :: Int

  The number of workers. This must be at least 1.

• logstashQueueRetryPolicy :: RetryPolicyM IO

  The retry policy.

• logstashQueueTimeout :: Integer

  The timeout for each attempt at sending data to the Logstash server.

defaultLogstashQueueCfg :: LogstashContext ctx => ctx -> LogstashQueueCfg ctx

defaultLogstashQueueCfg ctx constructs a LogstashQueueCfg with some default values for the Logstash context given by ctx:

• A queue size limited to 1000 entries
• Two worker threads
• 25ms exponential backoff with a maximum of five retries as retry policy
• 1s timeout for each logging attempt

You may wish to customise these settings to suit your needs.

withLogstashQueue :: (LogstashContext ctx, MonadUnliftIO m) => LogstashQueueCfg ctx -> (RetryStatus -> item -> ReaderT LogstashConnection IO ()) -> [item -> Handler ()] -> (TBMQueue item -> m a) -> m a

withLogstashQueue cfg codec exceptionHandlers action initialises a queue with space for a finite number of log messages given by cfg to allow for log messages to be sent to Logstash asynchronously. This addresses the following issues with synchronous logging:

• Since writing log messages to Logstash involves network I/O, this may be slower than queueing up messages in memory and therefore synchronously sending messages may delay the computation that originated the log message.
• With a finite number of Logstash connections, synchronous logging may also get blocked until a connection is available.

The queue is read from by a configurable number of worker threads which use Logstash connections from a LogstashContext. The queue is given to action as an argument. The retryPolicy and timeout parameters serve the same purpose as for runLogstash. We recommend that, if the LogstashContext is a LogstashPool, it should contain at least as many connections as the number of works to avoid contention between worker threads.

codec is the handler for how messages should be sent to the Logstash server, this is typically a codec like stashJsonLine. The RetryStatus is provided as an additional argument to codec in case a handler wishes to inspect this.

Codecs

stash :: MonadIO m => ByteString -> ReaderT LogstashConnection m ()

stash timeout message is a computation which sends message to the Logstash server.

stashJsonLine :: (MonadIO m, ToJSON a) => a -> ReaderT LogstashConnection m ()

stashJsonLine timeout document is a computation which serialises document and sends it to the Logstash server. This function is intended to be used with the json_lines codec.