Changelog for golds-gym-0.3.0.0
Changelog
[0.3.0]
Added
-
Lens-based expectation combinators for custom performance assertions
- New
Test.Hspec.BenchGolden.Lensesmodule with van Laarhoven lenses forGoldenStatsfields - Lenses:
_statsMean,_statsMedian,_statsTrimmedMean,_statsStddev,_statsMAD,_statsIQR,_statsMin,_statsMax - Smart metric selectors:
metricForandvarianceForautomatically choose appropriate lens based onBenchConfig Expectationtype for composable performance assertionsTolerancevariants:Percent,Absolute,Hybrid,MustImprove,MustRegress- Boolean composition operators:
(&&~)for AND,(||~)for OR - Infix operators:
(@~)for percentage,(@<)for absolute,(@<<)for must-improve,(@>>)for must-regress benchGoldenWithExpectationcombinator for custom lens-based expectations- Enables assertions like "must be 10% faster", "median within 10%", "IQR < 0.1ms"
- New
-
Tolerance helper functions for manual comparison
withinPercent,withinAbsolute,withinHybridfor tolerance checkingmustImprove,mustRegressfor directional performance expectationspercentDiff,absDiffutilities
Changed
- Refactored comparison logic to use lenses internally (non-breaking)
compareStatsnow usesmetricForinstead of if/else branchingcheckVariancenow usesvarianceForfor cleaner metric selection
Dependencies
- Added
microlens >= 0.4 && < 0.6
[0.2.0] - 2026-01-30
Added
-
Hybrid tolerance mechanism to prevent false failures from measurement noise
- New
absoluteToleranceMsfield inBenchConfig(default:Just 0.01ms) - Benchmarks now pass if EITHER percentage tolerance OR absolute tolerance is satisfied
- Eliminates random failures for sub-millisecond operations where measurement noise causes large percentage variations despite negligible absolute differences
- New
-
Enhanced failure messages showing both tolerance thresholds
- Regression/improvement messages now display:
"tolerance: 15.0% or 0.010 ms"when absolute tolerance is configured
- Regression/improvement messages now display:
-
Example benchmarks demonstrating tolerance configurations
- Hybrid tolerance (default)
- Percentage-only tolerance
- Strict absolute tolerance
- Relaxed tolerance for CI environments
Changed
- Default
BenchConfignow includesabsoluteToleranceMs = Just 0.01(10 microseconds) BenchResultconstructors (Regression,Improvement) now includeMaybe Doublefor absolute tolerance- Updated "sort already sorted" example to use robust statistics to handle outliers
- Increased tolerance for percentage-only example to 30% to reduce false failures
Fixed
- Random benchmark failures for fast operations (< 1ms) due to measurement noise
- False regressions when absolute time differences are negligible but percentage variations are large
- Inconsistent test results across runs for sub-millisecond operations
[0.1.0] - 2026-01-30
Added
- Initial release of golds-gym
- Golden testing framework for performance benchmarks
- Architecture-specific golden files
- Integration with hspec and benchpress
- Configurable tolerance for mean time comparison
- Robust statistics mode (trimmed mean, MAD, outlier detection)
- Variance warnings
- Environment variables for accepting/skipping benchmarks
- Support for both standard and robust statistical methods