Test.Tasty.Hedgehog.Coverage

Provide some functionality for tracking the distribution of test inputs when using Hedgehog property-based testing.

Data types

data Cover

Equivalent to the Property type from Hedgehog, but slightly modified for the purpose of enabling the classification functionality.

Constructors

Cover

Fields

• _coverageConf :: !PropertyConfig
• _coverageProp :: PropertyT (StateT Tally IO) ()

newtype Tally

This is the type used to store the information about the inputs.

Constructors

Tally

Fields

• unTally :: Map Text Int

Instances

Eq Tally

Defined in Test.Tasty.Hedgehog.Coverage

Methods

(==) :: Tally -> Tally -> Bool

(/=) :: Tally -> Tally -> Bool

Show Tally

Defined in Test.Tasty.Hedgehog.Coverage

Methods

showsPrec :: Int -> Tally -> ShowS

show :: Tally -> String

showList :: [Tally] -> ShowS

Test helpers

testPropertyCoverage :: TestName -> Cover -> TestTree

Create a TestTree using a Cover property test.

withCoverage :: HasCallStack => PropertyT (StateT Tally IO) () -> Cover

Simiar to the property function in Hedgehog, this creates a Cover that lets us track the required information.

Coverage functions

classify

Arguments

:: MonadState Tally m

=> Bool

True if this case should be included.

-> Text

The label for this input.

-> m ()

Records how many test cases satisfy a given condition.

prop_reverse_involutive :: Cover
prop_reverse_involutive = withCoverage $ do
  xs <- forAll $ Gen.list (Range.linear 0 100) Gen.alpha
  classify (length xs > 50) "non-trivial"
  test_involutive reverse xs

Which gives output similar to:

 reverse involutive:          OK
   18.00% non-trivial

label

Arguments

:: MonadState Tally m

=> Text

The label for the input.

-> m ()

Attach a simple label to a property.

prop_reverse_reverse :: Cover
prop_reverse_reverse = withCoverage $ do
  xs <- forAll $ Gen.list (Range.linear 0 100) Gen.alpha
  label ("length of input is " ++ show (length xs))
  reverse (reverse xs) === xs

Which gives output similar to:

reverse involutive:          OK
    4.00% with a length of 0
    7.00% with a length of 1
    3.00% with a length of 11
    2.00% with a length of 12
    2.00% with a length of 13
    ...

collect

Arguments

:: (MonadState Tally m, Show a)

=> a

The input to collect.

-> m ()

Uses the input itself as the label, useful for recording test case distribution.

prop_reverse_reverse :: Cover
prop_reverse_reverse = withCoverage $ do
  xs <- forAll $ Gen.list (Range.linear 0 100) Gen.alpha
  collect (length xs)
  reverse (reverse xs) === xs

Which gives output similar to:

reverse involutive:          OK
    8.00% ""
    1.00% "AFkNJBLiWYEBFRyZhulpMkkqIvsDpLAmaYoFTnNNFfkrbPUqDIRUuZOFGohTfB"
    1.00% "AWWfLCfmZPoydVYXwnFHyCEWztXanEzdoc"
    1.00% "CJJVBGOeaIkLfcOUGV"
    1.00% "CNrTsblqfEz"
    1.00% "CxDqm"

Property Config Helpers

These functions work exactly as their original Hedgehog counterparts, only modified to work with the Cover type.

withTests :: TestLimit -> Cover -> Cover

Set the number of times a property should be executed before it is considered successful.

If you have a test that does not involve any generators and thus does not need to run repeatedly, you can use withTests 1 to define a property that will only be checked once.

withRetries :: ShrinkRetries -> Cover -> Cover

Set the number of times a property will be executed for each shrink before the test runner gives up and tries a different shrink. See ShrinkRetries for more information.

withDiscards :: DiscardLimit -> Cover -> Cover

Set the number of times a property is allowed to discard before the test runner gives up.

withShrinks :: ShrinkLimit -> Cover -> Cover

Set the number of times a property is allowed to shrink before the test runner gives up and prints the counterexample.