bench-graph-0.1.0: Plot and compare benchmarks

BenchGraph

Description

BenchGraph generates a graph from benchmarking results (CSV) generated by gauge or criterion, specifically, it generates comparative graphs for several groups of benchmarks that can be compared. The groups could correspond to different packages providing similar and comparable operations or even different versions of the same package. This is a convenient tool to compare performance results of alternative libraries or versions of the same package after you make a change that may impact the performance benchmarks of the package.

The input is the CSV file generated by gauge --csv=results.csv or a similar output generated by criterion. You need to invoke the bgraph function with an appropriate Config to control various parameters of graph generation. Benchmark results found in the CSV file can be classified into several groups using a classifier function and each group is displayed side by side in the graph on the same scale for comparison. The constituent benchmarks in each benchmark group are placed together as a group and a legend is displayed to mark who is who.

See the test directory for an example of how to use it. A sample output can be found in the sample-charts directory.

Synopsis

# Documentation

How to show the comparisons among benchmark groups.

Since: 0.1.0

Constructors

 CompareFull Show full results for all groups CompareDelta Show the first group with full results, show delta from the first group for the subsequent groups.

Instances

 Source # Methods

data Config Source #

Configuration governing generation of chart.

Since: 0.1.0

Constructors

 Config FieldsoutputDir :: FilePathThe directory where the output graph should be placed.chartTitle :: Maybe StringThe title to be embedded in the generated graph.classifyBenchmark :: String -> Maybe (String, String)User supplied function that translates a benchmark name into a tuple (groupname, benchname), where groupname is the name of the group the benchmark should be placed in and benchname is the translated benchmark name to be displayed on the graph. If it returns Nothing the benchmark is omitted from the results.sortBenchmarks :: [String] -> [String]User supplied function to sort or reorder the list of benchmark names generated by classifyBenchmark. These are the benchmarks to be plotted for each benchmark group.sortBenchGroups :: [String] -> [String]User supplied function to sort or reorder the benchmark group names generated by classifyBenchmark.setYScale :: Maybe (Double, Double, Int)(RangeMin, RangeMax, NumIntervals) of the plot on the y (time) axis in microseconds.comparisonStyle :: ComparisonStyleHow to show the comparisons.

Default configuration. Use this as the base configuration and modify the required fields. The defaults are:

 outputDir         = "."
chartTitle        = Nothing
classifyBenchmark = b -> Just ("default", b)
sortBenchmarks    = id
sortBenchGroups   = id
setYScale         = Nothing
comparisonStyle   = CompareFull


Since: 0.1.0

bgraph :: FilePath -> FilePath -> String -> Config -> IO () Source #

The first parameter is an input file containing CSV data as generated by gauge --csv=results.csv or a similar output generated by criterion. The second parameter is the name of the output file containing the graph SVG image. The third parameter is the name of the field that should be plotted. The field is matched with the fields in the header line of the CSV input using a case insensitive match. The last parameter is the configuration to customize the graph, you can start with defaultConfig as the base and set any of the fields that you may want to change.

For example:

bgraph "results.csv" "Plot mean time" "mean" defaultConfig


Since: 0.1.0