buildbox-1.5.1.1: Rehackable components for writing buildbots and test harnesses.

BuildBox.Benchmark.BenchResult

Contents

Synopsis

Benchmark results

data BenchResult c Source

We include the name of the original benchmark to it's easy to lookup the results. If the BenchResult is carrying data derived directly by running a benchmark, there will be an element of the benchResultRuns for each iteration. On the other hand, If the BenchResult is carrying statistics or comparison data there should be a single element with an index of 0. This is suggested usage, and adhered to by the functions in this module, but not required.

data BenchRunResult c Source

Holds the result of running a benchmark once.

Constructors

BenchRunResult 

Fields

benchRunResultIndex :: Integer

What iteration this run was. Use 1 for the first ''real'' iteration derived by running a program. Use 0 for ''fake'' iterations computed by statistics or comparisons.

benchRunResultQuirks :: [Quirk]

Information about the run that doesn't carry units, eg whether it timed out or segfaulted.

benchRunResultAspects :: [WithUnits (Aspect c)]

Aspects of the benchmark run that carry units and can have statistics extracted from them.

Concatenation

concatBenchResult :: BenchResult c1 -> BenchResult c1Source

Concatenate the results of all runs. The the resulting BenchResult has a single BenchRunResult with an index of 0, containing all aspects.

Collation

collateBenchResult :: BenchResult Single -> BenchResult []Source

Collate the aspects of each run. See collateWithUnits for an explanation and example.

Statistics

statCollatedBenchResult :: BenchResult [] -> BenchResult StatsSource

Compute statistics from collated aspects of a run.

statBenchResult :: BenchResult Single -> BenchResult StatsSource

Collate the aspects, then compute statistics of a run.

Comparison

compareBenchResults :: BenchResult Stats -> BenchResult Stats -> BenchResult StatsComparisonSource

Compute comparisons of benchmark results. Both results must have the same benchResultName else error.

compareBenchResultWith :: [BenchResult Stats] -> BenchResult Stats -> BenchResult StatsComparisonSource

Compute comparisons of benchmark result, looking up the baseline results from a given list. If there are no matching baseline results then this creates a ComparisonNew in the output.

compareManyBenchResults :: [BenchResult Stats] -> [BenchResult Stats] -> [BenchResult StatsComparison]Source

Compare some baseline results against new results. If there are no matching baseline results then this creates a ComparisonNew in the output.

predBenchResult :: (forall units. Real units => c units -> Bool) -> BenchResult c -> BoolSource

Return true if any of the aspect data in a result matches a given predicate.

swungBenchResult :: Double -> BenchResult StatsComparison -> BoolSource

Return true if any of the aspects have swung by more than a given fraction since last time. For example, use 0.1 for 10 percent.

Merging

mergeBenchResults :: [[BenchResult c]] -> [BenchResult c]Source

Merge lists of BenchResults, preferring results from earlier lists. In the output list there is one result for every named benchmark in the input.

Advancement

splitBenchResults :: Double -> [BenchResult StatsComparison] -> ([BenchResult StatsComparison], [BenchResult StatsComparison], [BenchResult StatsComparison])Source

Given a fraction (like 0.1 for 10 percent), split some results into three groups: ''winners'', ''losers'' and ''others''. The losers are benchmarks had any aspect increase by more than the fraction. Winners are non-losers, where any aspect decreased by the fraction. Others are not winners or losers.

advanceBenchResultsSource

Arguments

:: Double 
-> [BenchResult StatsComparison]

Comparisons to guide the advancement.

-> [BenchResult Single]

Baseline results.

-> [BenchResult Single]

Recent results.

-> [BenchResult Single]

New baseline.

Create a new baseline from original baseline, and recent results. If any of the recent results are winners then use them, otherwise use results from the old baseline.

Application functions

appBenchRunResult :: ([BenchRunResult c1] -> b) -> BenchResult c1 -> bSource

Apply a function to the aspects of a BenchRunResult

appRunResultAspects :: ([WithUnits (Aspect c1)] -> b) -> BenchRunResult c1 -> bSource

Apply a function to the aspects of a BenchRunResult

Lifting functions

liftBenchRunResult2 :: ([BenchRunResult c1] -> [BenchRunResult c2] -> [BenchRunResult c3]) -> BenchResult c1 -> BenchResult c2 -> BenchResult c3Source

Lift a binary function to the BenchResults in a BenchResult

liftToAspectsOfBenchResult :: ([WithUnits (Aspect c1)] -> [WithUnits (Aspect c2)]) -> BenchResult c1 -> BenchResult c2Source

Lift a function to the aspects of each BenchRunResult.

liftToAspectsOfBenchResult2 :: ([WithUnits (Aspect c1)] -> [WithUnits (Aspect c2)] -> [WithUnits (Aspect c3)]) -> BenchResult c1 -> BenchResult c2 -> BenchResult c3Source

Lift a binary function to the aspects of each BenchRunResult.

liftRunResultAspects :: ([WithUnits (Aspect c1)] -> [WithUnits (Aspect c2)]) -> BenchRunResult c1 -> BenchRunResult c2Source

Lift a function to the aspects of a BenchRunResult

liftRunResultAspects2 :: ([WithUnits (Aspect c1)] -> [WithUnits (Aspect c2)] -> [WithUnits (Aspect c3)]) -> BenchRunResult c1 -> BenchRunResult c2 -> BenchRunResult c3Source

Lift a binary function to the aspects of two BenchRunResults. The resulting BenchRunResult gets all the quirks from both.