Data.Metrics.Histogram

Histogram metrics allow you to measure not just easy things like the min, mean, max, and standard deviation of values, but also quantiles like the median or 95th percentile.

Traditionally, the way the median (or any other quantile) is calculated is to take the entire data set, sort it, and take the value in the middle (or 1% from the end, for the 99th percentile). This works for small data sets, or batch processing systems, but not for high-throughput, low-latency services.

The solution for this is to sample the data as it goes through. By maintaining a small, manageable reservoir which is statistically representative of the data stream as a whole, we can quickly and easily calculate quantiles which are valid approximations of the actual quantiles. This technique is called reservoir sampling.

Documentation

data Histogram m

A measure of the distribution of values in a stream of data.

Instances

PrimMonad m => TakeSnapshot m (Histogram m)

PrimMonad m => Statistics m (Histogram m)

PrimMonad m => Clear m (Histogram m)

PrimMonad m => Count m (Histogram m)

PrimMonad m => Update m (Histogram m) Double

Register (Histogram IO)

histogram :: PrimMonad m => m NominalDiffTime -> Reservoir -> m (Histogram m)

Create a histogram using a custom time data supplier function and a custom reservoir.

exponentiallyDecayingHistogram :: IO (Histogram IO)

The recommended histogram type. It provides a fast histogram that probabilistically evicts older entries using a weighting system. This ensures that snapshots remain relatively fresh.

uniformHistogram :: Seed -> IO (Histogram IO)

A histogram that gives all entries an equal likelihood of being evicted.

Probably not what you want for most time-series data.

module Data.Metrics.Types