katip-0.5.0.0: A structured logging framework.

Safe HaskellNone
LanguageHaskell2010

Katip

Contents

Description

Includes all of the APIs youll need to use Katip. Be sure to check out the included examples directory for an example of usage.

Here's a basic example:

import Control.Exception
import Katip

main :: IO ()
main = do
  handleScribe <- mkHandleScribe ColorIfTerminal stdout InfoS V2
  let makeLogEnv = registerScribe "stdout" handleScribe defaultScribeSettings =<< initLogEnv MyApp "production"
  -- closeScribes will stop accepting new logs, flush existing ones and clean up resources
  bracket makeLogEnv closeScribes $ le -> do
    let initialContext = () -- this context will be attached to every log in your app and merged w/ subsequent contexts
    let initialNamespace = "main"
    runKatipContextT le initialContext initialNamespace $ do
      $(logTM) InfoS "Hello Katip"
      -- This adds a namespace to the current namespace and merges a piece of contextual data into your context
      katipAddNamespace "additional_namespace" $ katipAddContext (sl "some_context" True) $ do
        $(logTM) WarningS "Now we're getting fancy"
      katipNoLogging $ do
        $(logTM) DebugS "You will never see this!"

To get up and running, the workflow is generally:

  • Set up a LogEnv using initLogEnv.
  • Add Scribes using registerScribe.
  • Either use KatipT or KatipContextT for a pre-built transformer stack or add Katip and KatipContext instances to your own transformer stack. If you do the latter, you may want to look in the examples dir for some tips on composing contexts and namespaces.
  • Define some structured log data throughout your application and implement ToObject and LogItem for them.
  • Begin logging with logT, logTM, etc.
  • Define your own Scribe if you need to output to some as-yet unsupported format or service. If you think it would be useful to others, consider releasing your own package.

Synopsis

Framework Types

newtype Namespace Source

Represents a heirarchy of namespaces going from general to specific. For instance: ["processname", "subsystem"]. Note that single-segment namespaces can be created using IsString/OverloadedStrings, so "foo" will result in Namespace ["foo"].

Constructors

Namespace 

Fields

unNamespace :: [Text]
 

newtype Environment Source

Application environment, like prod, devel, testing.

Constructors

Environment 

Fields

getEnvironment :: Text
 

data Severity Source

Constructors

DebugS

Debug messages

InfoS

Information

NoticeS

Normal runtime Conditions

WarningS

General Warnings

ErrorS

General Errors

CriticalS

Severe situations

AlertS

Take immediate action

EmergencyS

System is unusable

data Verbosity Source

Verbosity controls the amount of information (columns) a Scribe emits during logging.

The convention is: - V0 implies no additional payload information is included in message. - V3 implies the maximum amount of payload information. - Anything in between is left to the discretion of the developer.

Constructors

V0 
V1 
V2 
V3 

class ToObject a where Source

Katip requires JSON objects to be logged as context. This typeclass provides a default instance which uses ToJSON and produces an empty object if toJSON results in any type other than object. If you have a type you want to log that produces an Array or Number for example, you'll want to write an explicit instance here. You can trivially add a ToObject instance for something with a ToJSON instance like:

instance ToObject Foo

Minimal complete definition

Nothing

Methods

toObject :: a -> Object Source

class ToObject a => LogItem a where Source

Payload objects need instances of this class. LogItem makes it so that you can have very verbose items getting logged with lots of extra fields but under normal circumstances, if your scribe is configured for a lower verbosity level, it will only log a selection of those keys. Furthermore, each Scribe can be configured with a different Verbosity level. You could even use registerScribe, unregisterScribe, and clearScribes to at runtime swap out your existing scribes for more verbose debugging scribes if you wanted to.

When defining payloadKeys, don't redundantly declare the same keys for higher levels of verbosity. Each level of verbosity automatically and recursively contains all keys from the level before it.

Methods

payloadKeys :: Verbosity -> a -> PayloadSelection Source

List of keys in the JSON object that should be included in message.

data Item a Source

This has everything each log message will contain.

Instances

Functor Item Source 
Show a => Show (Item a) Source 
Generic (Item a) Source 
ToJSON a => ToJSON (Item a) Source 
FromJSON a => FromJSON (Item a) Source 
type Rep (Item a) Source 

data PayloadSelection Source

Field selector by verbosity within JSON payload.

Constructors

AllKeys 
SomeKeys [Text] 

data Scribe Source

Scribes are handlers of incoming items. Each registered scribe knows how to push a log item somewhere.

Guidelines for writing your own Scribe

Scribes should always take a Severity and Verbosity.

Severity is used to *exclude log messages* that are < the provided Severity. For instance, if the user passes InfoS, DebugS items should be ignored. Katip provides the permitItem utility for this.

Verbosity is used to select keys from the log item's payload. Each LogItem instance describes what keys should be retained for each Verbosity level. Use the payloadObject utility for extracting the keys that should be permitted.

There is no built-in mechanism in katip for telling a scribe that its time to shut down. unregisterScribe merely drops it from the LogEnv. This means there are 2 ways to handle resources as a scribe:

  1. Pass in the resource when the scribe is created. Handle allocation and release of the resource elsewhere. This is what the Handle scribe does.
  2. Return a finalizing function that tells the scribe to shut down. katip-elasticsearch's mkEsScribe returns an IO (Scribe, IO ()). The finalizer will flush any queued log messages and shut down gracefully before returning. This can be hooked into your application's shutdown routine to ensure you never miss any log messages on shutdown.

Constructors

Scribe 

Fields

liPush :: forall a. LogItem a => Item a -> IO ()
 
scribeFinalizer :: IO ()

Provide a *blocking* finalizer to call when your scribe is removed. If this is not relevant to your scribe, return () is fine.

Instances

data LogEnv Source

Constructors

LogEnv 

Fields

_logEnvHost :: HostName
 
_logEnvPid :: ProcessID
 
_logEnvApp :: Namespace

Name of application. This will typically never change. This field gets prepended to the namespace of your individual log messages. For example, if your app is MyApp and you write a log using "logItem" and the namespace WebServer, the final namespace will be MyApp.WebServer

_logEnvEnv :: Environment
 
_logEnvTimer :: IO UTCTime

Action to fetch the timestamp. You can use something like AutoUpdate for high volume logs but note that this may cause some output forms to display logs out of order. Alternatively, you could just use getCurrentTime.

_logEnvScribes :: Map Text ScribeHandle
 

data SimpleLogPayload Source

Instances

Monoid SimpleLogPayload Source 
Semigroup SimpleLogPayload Source 
ToJSON SimpleLogPayload Source

A built-in convenience log payload that won't log anything on V0, but will log everything in any other level of verbosity. Intended for easy in-line usage without having to define new log types.

Construct using sl and combine multiple tuples using <> from Monoid.

LogItem SimpleLogPayload Source 
ToObject SimpleLogPayload Source 

sl :: ToJSON a => Text -> a -> SimpleLogPayload Source

Construct a simple log from any JSON item.

defaultScribeSettings :: ScribeSettings Source

Reasonable defaults for a scribe. Buffer size of 4096.

lens-compatible Lenses

itemApp :: forall a. Lens' (Item a) Namespace Source

itemEnv :: forall a. Lens' (Item a) Environment Source

itemSeverity :: forall a. Lens' (Item a) Severity Source

itemThread :: forall a. Lens' (Item a) ThreadIdText Source

itemHost :: forall a. Lens' (Item a) HostName Source

itemProcess :: forall a. Lens' (Item a) ProcessID Source

itemPayload :: forall a a. Lens (Item a) (Item a) a a Source

itemMessage :: forall a. Lens' (Item a) LogStr Source

itemTime :: forall a. Lens' (Item a) UTCTime Source

itemNamespace :: forall a. Lens' (Item a) Namespace Source

itemLoc :: forall a. Lens' (Item a) (Maybe Loc) Source

logEnvHost :: Lens' LogEnv HostName Source

A Built-in Monad For Simple Logging

newtype KatipT m a Source

A concrete monad you can use to run logging actions. Use this if you prefer an explicit monad transformer stack and adding layers as opposed to implementing Katip for your monad.

Constructors

KatipT 

Fields

unKatipT :: ReaderT LogEnv m a
 

Instances

MonadTrans KatipT Source 
MonadTransControl KatipT Source 
MonadBase b m => MonadBase b (KatipT m) Source 
MonadBaseControl b m => MonadBaseControl b (KatipT m) Source 
Monad m => Monad (KatipT m) Source 
Functor m => Functor (KatipT m) Source 
Applicative m => Applicative (KatipT m) Source 
MonadIO m => MonadIO (KatipT m) Source 
MonadThrow m => MonadThrow (KatipT m) Source 
MonadMask m => MonadMask (KatipT m) Source 
MonadCatch m => MonadCatch (KatipT m) Source 
MonadIO m => Katip (KatipT m) Source 
(Monad m, KatipContext m) => KatipContext (KatipT m) Source 
type StT KatipT a = a Source 
type StM (KatipT m) a = ComposeSt KatipT m a Source 

runKatipT :: LogEnv -> KatipT m a -> m a Source

Execute KatipT on a log env.

Initializing Loggers

initLogEnv Source

Arguments

:: Namespace

A base namespace for this application

-> Environment

Current run environment (e.g. prod vs. devel)

-> IO LogEnv 

Create a reasonable default InitLogEnv. Uses an AutoUdate with the default settings as the timer. If you are concerned about timestamp precision or event ordering in log outputs like ElasticSearch, you should replace the timer with getCurrentTime

registerScribe Source

Arguments

:: Text

Name the scribe

-> Scribe 
-> ScribeSettings 
-> LogEnv 
-> IO LogEnv 

Add a scribe to the list. All future log calls will go to this scribe in addition to the others.

Dropping scribes temporarily

unregisterScribe Source

Arguments

:: Text

Name of the scribe

-> LogEnv 
-> LogEnv 

Remove a scribe from the environment. This does *not* finalize the scribe. This mainly only makes sense to use with something like MonadReader's local function to temporarily disavow a single logger for a block of code.

clearScribes :: LogEnv -> LogEnv Source

Unregister *all* scribes. Note that this is *not* for closing or finalizing scribes, use closeScribes for that. This mainly only makes sense to use with something like MonadReader's local function to temporarily disavow any loggers for a block of code.

Finalizing scribes at shutdown

closeScribes :: LogEnv -> IO LogEnv Source

Call this at the end of your program. This is a blocking call that stop writing to a scribe's queue, waits for the queue to empty, finalizes each scribe in the log environment and then removes it. Finalizers are all run even if one of them throws, but the exception will be re-thrown at the end.

closeScribe Source

Arguments

:: Text

Name of the scribe

-> LogEnv 
-> IO LogEnv 

Finalize a scribe. The scribe is removed from the environment, its finalizer is called and it can never be written to again. Note that this will throw any exceptions yoru finalizer will throw, and that LogEnv is immutable, so it will not be removed in that case.

Logging Functions

newtype LogStr Source

Log message with Builder underneath; use <> to concat in O(1).

Constructors

LogStr 

Fields

unLogStr :: Builder
 

logStr :: StringConv a Text => a -> LogStr Source

Pack any string-like thing into a LogStr. This will automatically work on String, ByteString, Text and any of the lazy variants.

ls :: StringConv a Text => a -> LogStr Source

Shorthand for logStr

showLS :: Show a => a -> LogStr Source

Convert any showable type into a LogStr.

Katip LoggingFunctions

These logging functions use the basic Katip constraint and thus will require varying degrees of explicit detail such as Namespace and individual log items to be passed in. These can be described as the primitives of Katip logging. If you find yourself making multiple log statements within a logical logging context for your app, you may want to look into the KatipContext family of logging functions like logFM and logTM. KatipContext in most applications should be considered the default. Here's an example of the pain point:

doDatabaseThings = do
  connId <- getConnectionId
  logF (ConnectionIDContext connId) "database" InfoS "Doing database stuff"
  -- ...
  logF (ConnectionIDContext connId) "database" InfoS "Wow, passing in the same context is getting tedious"

Another pain point to look out for is nesting actions that log in each other. Let's say you were writing a web app. You want to capture some detail such as the user's ID in the logs, but you also want that info to show up in doDatabaseThings' logs so you can associate those two pieces of information:

webRequestHandler = do
  uid <- getUserId
  logF (UserIDContext uid) "web" InfoS "Starting web request"
  doDatabaseThings

In the above example, doDatabaseThings would overwrite that UserIDContext with its own context and namespace. Sometimes this is what you want and that's why logF and other functions which only require Katip exist. If you are interested in combining log contexts and namespaces, see KatipContext.

class MonadIO m => Katip m where Source

Monads where katip logging actions can be performed. Katip is the most basic logging monad. You will typically use this directly if you either don't want to use namespaces/contexts heavily or if you want to pass in specific contexts and/or namespaces at each log site.

For something more powerful, look at the docs for KatipContext, which keeps a namespace and merged context. You can write simple functions that add additional namespacing and merges additional context on the fly.

localLogEnv was added to allow for lexically-scoped modifications of the log env that are reverted when the supplied monad completes. katipNoLogging, for example, uses this to temporarily pause log outputs.

Methods

getLogEnv :: m LogEnv Source

localLogEnv :: (LogEnv -> LogEnv) -> m a -> m a Source

Instances

Katip m => Katip (MaybeT m) Source 
Katip m => Katip (ResourceT m) Source 
MonadIO m => Katip (KatipT m) Source 
MonadIO m => Katip (KatipContextT m) Source 
Katip m => Katip (ReaderT s m) Source 
Katip m => Katip (StateT s m) Source 
Katip m => Katip (StateT s m) Source 
Katip m => Katip (ExceptT s m) Source 
(Katip m, Monoid s) => Katip (WriterT s m) Source 
(Katip m, Monoid s) => Katip (WriterT s m) Source 
Katip m => Katip (EitherT s m) Source 
(Katip m, Monoid w) => Katip (RWST r w s m) Source 
(Katip m, Monoid w) => Katip (RWST r w s m) Source 

logF Source

Arguments

:: (Applicative m, LogItem a, Katip m) 
=> a

Contextual payload for the log

-> Namespace

Specific namespace of the message.

-> Severity

Severity of the message

-> LogStr

The log message

-> m () 

Log with full context, but without any code location.

logMsg :: (Applicative m, Katip m) => Namespace -> Severity -> LogStr -> m () Source

Log a message without any payload/context or code location.

logT :: ExpQ Source

Loc-tagged logging when using template-haskell.

$(logT) obj mempty InfoS "Hello world"

logItem :: (Applicative m, LogItem a, Katip m) => a -> Namespace -> Maybe Loc -> Severity -> LogStr -> m () Source

Log with everything, including a source code location. This is very low level and you typically can use logT in its place.

logException Source

Arguments

:: (Katip m, LogItem a, MonadCatch m, Applicative m) 
=> a

Log context

-> Namespace

Namespace

-> Severity

Severity

-> m b

Main action being run

-> m b 

Perform an action while logging any exceptions that may occur. Inspired by onException.

>>> > logException () mempty ErrorS (error "foo")

KatipContext: Logging With Context

These logging functions use the KatipContext constraint which is a superclass of Katip that also has a mechanism for keeping track of the current context and namespace. This means a few things:

  1. Functions that use KatipContext like logFM and logTM do not require you to pass in LogItems or Namespaces, they pull them from the monadic environment.
  2. It becomes easy to add functions which add namespaces and/or contexts to the current stack of them. You can (and should) make that action scoped to a monadic action so that when it finishes, the previous context and namespace will be automatically restored.

KatipContextT provides a simple, ReaderT-based implementation of the KatipContext typeclass, and provides katipAddContext and katipAddNamespace functions to append to the context for the duration of a block:

main = do
  le <- initLogEnv MyApp "production"
  -- set up scribes here
  runKatipContext le () "main" $ do
    katipAddNamespace "nextlevel" $ do
      $(logTM) InfoS "Logs here will have namespace MyApp.main.nextlevel"

    katipAddContext TrivialContext $ do
      $(logTM) InfoS "Logs here will have context from TrivialContext"

      katipAddContext AnotherContext $ do
        $(logTM) InfoS "Logs here will have context from TrivialContext *merged with* context from AnotherContext!"

    $(logTM) InfoS "Log context restored to () and namespace to MyApp.main"

katipAddNamespace and katipAddContext are one-liners, implemented in terms of local from MonadReader. If you have a custom monad transformer stack and want to add your own version of these, check out <https://github.com/Soostone/katip/tree/master/katip/examples these examples>.

class Katip m => KatipContext m where Source

A monadic context that has an inherant way to get logging context and namespace. Examples include a web application monad or database monad. The local variants are just like local from Reader and indeed you can easily implement them with local if you happen to be using a Reader in your monad. These give us katipAddNamespace and katipAddContext that works with *any* KatipContext, as opposed to making users have to implement these functions on their own in each app.

Methods

getKatipContext :: m LogContexts Source

localKatipContext :: (LogContexts -> LogContexts) -> m a -> m a Source

Temporarily modify the current context for the duration of the supplied monad. Used in katipAddContext

getKatipNamespace :: m Namespace Source

localKatipNamespace :: (Namespace -> Namespace) -> m a -> m a Source

Temporarily modify the current namespace for the duration of the supplied monad. Used in katipAddContext

logFM Source

Arguments

:: (Applicative m, KatipContext m) 
=> Severity

Severity of the message

-> LogStr

The log message

-> m () 

Log with full context, but without any code location. Automatically supplies payload and namespace.

logTM :: ExpQ Source

Loc-tagged logging when using template-haskell. Automatically supplies payload and namespace.

$(logTM) InfoS "Hello world"

logItemM :: (Applicative m, KatipContext m) => Maybe Loc -> Severity -> LogStr -> m () Source

Log with everything, including a source code location. This is very low level and you typically can use logTM in its place. Automatically supplies payload and namespace.

logExceptionM Source

Arguments

:: (KatipContext m, MonadCatch m, Applicative m) 
=> m a

Main action to run

-> Severity

Severity

-> m a 

Perform an action while logging any exceptions that may occur. Inspired by onException.

>>> > error "foo" `logExceptionM` ErrorS

data AnyLogContext Source

A wrapper around a log context that erases type information so that contexts from multiple layers can be combined intelligently.

data LogContexts Source

Heterogeneous list of log contexts that provides a smart LogContext instance for combining multiple payload policies. This is critical for log contexts deep down in a stack to be able to inject their own context without worrying about other context that has already been set. Also note that contexts are treated as a sequence and <> will be appended to the right hand side of the sequence. If there are conflicting keys in the contexts, the /right side will take precedence/, which is counter to how monoid works for Map and HashMap, so bear that in mind. The reasoning is that if the user is sequentially adding contexts to the right side of the sequence, on conflict the intent is to overwrite with the newer value (i.e. the rightmost value).

Additional note: you should not mappend LogContexts in any sort of infinite loop, as it retains all data, so that would be a memory leak.

liftPayload :: LogItem a => a -> LogContexts Source

Lift a log context into the generic wrapper so that it can combine with the existing log context.

Temporarily Changing Logging Behavior

katipAddNamespace :: KatipContext m => Namespace -> m a -> m a Source

Append a namespace segment to the current namespace for the given monadic action, then restore the previous state afterwards. Works with anything implementing KatipContext.

katipAddContext :: (LogItem i, KatipContext m) => i -> m a -> m a Source

Append some context to the current context for the given monadic action, then restore the previous state afterwards. Important note: be careful using this in a loop. If you're using something like forever or replicateM_ that does explicit sharing to avoid a memory leak, youll be fine as it will *sequence* calls to katipAddNamespace, so each loop will get the same context added. If you instead roll your own recursion and you're recursing in the action you provide, you'll instead accumulate tons of redundant contexts and even if they all merge on log, they are stored in a sequence and will leak memory. Works with anything implementing KatipContext.

katipNoLogging :: Katip m => m a -> m a Source

Disable all scribes for the given monadic action, then restore them afterwards. Works in any Katip monad.

Included Scribes

mkHandleScribe :: ColorStrategy -> Handle -> Severity -> Verbosity -> IO Scribe Source

Logs to a file handle such as stdout, stderr, or a file. Contexts and other information will be flattened out into bracketed fields. For example:

[2016-05-11 21:01:15][MyApp][Info][myhost.example.com][1724][ThreadId 1154][main:Helpers.Logging Helpers/Logging.hs:32:7] Started
[2016-05-11 21:01:15][MyApp.confrabulation][Debug][myhost.example.com][1724][ThreadId 1154][confrab_factor:42.0][main:Helpers.Logging Helpers/Logging.hs:41:9] Confrabulating widgets, with extra namespace and context
[2016-05-11 21:01:15][MyApp][Info][myhost.example.com][1724][ThreadId 1154][main:Helpers.Logging Helpers/Logging.hs:43:7] Namespace and context are back to normal

Returns the newly-created Scribe. The finalizer flushes the handle.

data ColorStrategy Source

Constructors

ColorLog Bool

Whether to use color control chars in log output

ColorIfTerminal

Color if output is a terminal

Tools for implementing Scribes

permitItem :: Severity -> Item a -> Bool Source

Should this item be logged given the user's maximum severity?

payloadObject :: LogItem a => Verbosity -> a -> Object Source

Constrain payload based on verbosity. Backends should use this to automatically bubble higher verbosity levels to lower ones.

itemJson :: LogItem a => Verbosity -> Item a -> Value Source

Convert log item to its JSON representation while trimming its payload based on the desired verbosity. Backends that push JSON messages should use this to obtain their payload.

KatipContextT - Utility transformer that provides Katip and KatipContext instances

data KatipContextT m a Source

Provides a simple transformer that defines a KatipContext instance for a fixed namespace and context. Just like KatipT, you should use this if you prefer an explicit transformer stack and don't want to (or cannot) define KatipContext for your monad . This is the slightly more powerful version of KatipT in that it provides KatipContext instead of just Katip. For instance:

  threadWithLogging = do
    le <- getLogEnv
    ctx <- getKatipContext
    ns <- getKatipNamespace
    forkIO $ runKatipContextT le ctx ns $ do
      $(logTM) InfoS "Look, I can log in IO and retain context!"
      doOtherStuff

Instances

MonadTrans KatipContextT Source 
MonadTransControl KatipContextT Source 
MonadError e m => MonadError e (KatipContextT m) Source 
MonadReader r m => MonadReader r (KatipContextT m) Source 
MonadState s m => MonadState s (KatipContextT m) Source 
MonadWriter w m => MonadWriter w (KatipContextT m) Source 
MonadBase b m => MonadBase b (KatipContextT m) Source 
MonadBaseControl b m => MonadBaseControl b (KatipContextT m) Source 
Monad m => Monad (KatipContextT m) Source 
Functor m => Functor (KatipContextT m) Source 
MonadFix m => MonadFix (KatipContextT m) Source 
Applicative m => Applicative (KatipContextT m) Source 
Alternative m => Alternative (KatipContextT m) Source 
MonadPlus m => MonadPlus (KatipContextT m) Source 
MonadIO m => MonadIO (KatipContextT m) Source 
MonadThrow m => MonadThrow (KatipContextT m) Source 
MonadMask m => MonadMask (KatipContextT m) Source 
MonadCatch m => MonadCatch (KatipContextT m) Source 
MonadIO m => Katip (KatipContextT m) Source 
MonadIO m => KatipContext (KatipContextT m) Source 
type StT KatipContextT a = StT (ReaderT KatipContextTState) a Source 
type StM (KatipContextT m) a = ComposeSt KatipContextT m a Source