BuildBox.Benchmark.BenchResult

Benchmark results

data BenchResult c

We include the name of the original benchmark to it's easy to lookup the results. If the BenchResult is carrying data derived directly by running a benchmark, there will be an element of the benchResultRuns for each iteration. On the other hand, If the BenchResult is carrying statistics or comparison data there should be a single element with an index of 0. This is suggested usage, and adhered to by the functions in this module, but not required.

Constructors

BenchResult

Fields

benchResultName :: String

benchResultRuns :: [BenchRunResult c]

Instances

(HasUnits (c Bytes) Bytes, Read (c Bytes), HasUnits (c Seconds) Seconds, Read (c Seconds)) => Read (BenchResult c)

(Show (c Seconds), Show (c Bytes)) => Show (BenchResult c)

(Pretty (c Seconds), Pretty (c Bytes)) => Pretty (BenchResult c)

data BenchRunResult c

Holds the result of running a benchmark once.

Constructors

BenchRunResult

Fields

benchRunResultIndex :: Integer

What iteration this run was. Use 1 for the first ''real'' iteration derived by running a program. Use 0 for ''fake'' iterations computed by statistics or comparisons.

benchRunResultQuirks :: [Quirk]

Information about the run that doesn't carry units, eg whether it timed out or segfaulted.

benchRunResultAspects :: [WithUnits (Aspect c)]

Aspects of the benchmark run that carry units and can have statistics extracted from them.

Instances

(HasUnits (c Bytes) Bytes, Read (c Bytes), HasUnits (c Seconds) Seconds, Read (c Seconds)) => Read (BenchRunResult c)

(Show (c Seconds), Show (c Bytes)) => Show (BenchRunResult c)

(Pretty (c Seconds), Pretty (c Bytes)) => Pretty (BenchRunResult c)

Concatenation

concatBenchResult :: BenchResult c1 -> BenchResult c1

Concatenate the results of all runs. The the resulting BenchResult has a single BenchRunResult with an index of 0, containing all aspects.

Collation

collateBenchResult :: BenchResult Single -> BenchResult []

Collate the aspects of each run. See collateWithUnits for an explanation and example.

Statistics

statCollatedBenchResult :: BenchResult [] -> BenchResult Stats

Compute statistics from collated aspects of a run.

statBenchResult :: BenchResult Single -> BenchResult Stats

Collate the aspects, then compute statistics of a run.

Comparison

compareBenchResults :: BenchResult Stats -> BenchResult Stats -> BenchResult StatsComparison

Compute comparisons of benchmark results. Both results must have the same benchResultName else error.

compareBenchResultWith :: [BenchResult Stats] -> BenchResult Stats -> BenchResult StatsComparison

Compute comparisons of benchmark result, looking up the baseline results from a given list. If there are no matching baseline results then this creates a ComparisonNew in the output.

compareManyBenchResults :: [BenchResult Stats] -> [BenchResult Stats] -> [BenchResult StatsComparison]

Compare some baseline results against new results. If there are no matching baseline results then this creates a ComparisonNew in the output.

predBenchResult :: (forall units. Real units => c units -> Bool) -> BenchResult c -> Bool

Return true if any of the aspect data in a result matches a given predicate.

swungBenchResult :: Double -> BenchResult StatsComparison -> Bool

Return true if any of the aspects have swung by more than a given fraction since last time. For example, use 0.1 for 10 percent.

Merging

mergeBenchResults :: [[BenchResult c]] -> [BenchResult c]

Merge lists of BenchResults, preferring results from earlier lists. In the output list there is one result for every named benchmark in the input.

Advancement

splitBenchResults :: Double -> [BenchResult StatsComparison] -> ([BenchResult StatsComparison], [BenchResult StatsComparison], [BenchResult StatsComparison])

Given a fraction (like 0.1 for 10 percent), split some results into three groups: ''winners'', ''losers'' and ''others''. The losers are benchmarks had any aspect increase by more than the fraction. Winners are non-losers, where any aspect decreased by the fraction. Others are not winners or losers.

advanceBenchResults

Arguments

:: Double

-> [BenchResult StatsComparison]

Comparisons to guide the advancement.

-> [BenchResult Single]

Baseline results.

-> [BenchResult Single]

Recent results.

-> [BenchResult Single]

New baseline.

Create a new baseline from original baseline, and recent results. If any of the recent results are winners then use them, otherwise use results from the old baseline.

Application functions

appBenchRunResult :: ([BenchRunResult c1] -> b) -> BenchResult c1 -> b

Apply a function to the aspects of a BenchRunResult

appRunResultAspects :: ([WithUnits (Aspect c1)] -> b) -> BenchRunResult c1 -> b

Apply a function to the aspects of a BenchRunResult

Lifting functions

liftBenchRunResult :: ([BenchRunResult c1] -> [BenchRunResult c2]) -> BenchResult c1 -> BenchResult c2

Lift a function to the BenchRunResult in a BenchResult

liftBenchRunResult2 :: ([BenchRunResult c1] -> [BenchRunResult c2] -> [BenchRunResult c3]) -> BenchResult c1 -> BenchResult c2 -> BenchResult c3

Lift a binary function to the BenchResults in a BenchResult

liftToAspectsOfBenchResult :: ([WithUnits (Aspect c1)] -> [WithUnits (Aspect c2)]) -> BenchResult c1 -> BenchResult c2

Lift a function to the aspects of each BenchRunResult.

liftToAspectsOfBenchResult2 :: ([WithUnits (Aspect c1)] -> [WithUnits (Aspect c2)] -> [WithUnits (Aspect c3)]) -> BenchResult c1 -> BenchResult c2 -> BenchResult c3

Lift a binary function to the aspects of each BenchRunResult.

liftRunResultAspects :: ([WithUnits (Aspect c1)] -> [WithUnits (Aspect c2)]) -> BenchRunResult c1 -> BenchRunResult c2

Lift a function to the aspects of a BenchRunResult

liftRunResultAspects2 :: ([WithUnits (Aspect c1)] -> [WithUnits (Aspect c2)] -> [WithUnits (Aspect c3)]) -> BenchRunResult c1 -> BenchRunResult c2 -> BenchRunResult c3

Lift a binary function to the aspects of two BenchRunResults. The resulting BenchRunResult gets all the quirks from both.