Safe Haskell | Safe-Inferred |
---|---|
Language | Haskell2010 |
This module defines functions for declaring different test settings
Synopsis
- modifyMaxSuccess :: (Int -> Int) -> TestDefM a b c -> TestDefM a b c
- modifyMaxDiscardRatio :: (Int -> Int) -> TestDefM a b c -> TestDefM a b c
- modifyMaxSize :: (Int -> Int) -> TestDefM a b c -> TestDefM a b c
- modifyMaxShrinks :: (Int -> Int) -> TestDefM a b c -> TestDefM a b c
- modifyRunSettings :: (TestRunSettings -> TestRunSettings) -> TestDefM a b c -> TestDefM a b c
- data TestRunSettings = TestRunSettings {}
- sequential :: TestDefM a b c -> TestDefM a b c
- parallel :: TestDefM a b c -> TestDefM a b c
- withParallelism :: Parallelism -> TestDefM a b c -> TestDefM a b c
- data Parallelism
- randomiseExecutionOrder :: TestDefM a b c -> TestDefM a b c
- doNotRandomiseExecutionOrder :: TestDefM a b c -> TestDefM a b c
- withExecutionOrderRandomisation :: ExecutionOrderRandomisation -> TestDefM a b c -> TestDefM a b c
- data ExecutionOrderRandomisation
- flaky :: Int -> TestDefM a b c -> TestDefM a b c
- flakyWith :: Int -> String -> TestDefM a b c -> TestDefM a b c
- notFlaky :: TestDefM a b c -> TestDefM a b c
- withFlakiness :: FlakinessMode -> TestDefM a b c -> TestDefM a b c
- data FlakinessMode
- = MayNotBeFlaky
- | MayBeFlakyUpTo !Int !(Maybe String)
Declaring different test settings
modifyRunSettings :: (TestRunSettings -> TestRunSettings) -> TestDefM a b c -> TestDefM a b c Source #
data TestRunSettings Source #
Instances
Declaring parallelism
sequential :: TestDefM a b c -> TestDefM a b c Source #
Declare that all tests below must be run sequentially
parallel :: TestDefM a b c -> TestDefM a b c Source #
Declare that all tests below may be run in parallel. (This is the default.)
withParallelism :: Parallelism -> TestDefM a b c -> TestDefM a b c Source #
Annotate a test group with Parallelism
.
Declaring randomisation order
randomiseExecutionOrder :: TestDefM a b c -> TestDefM a b c Source #
Declare that the order of execution of all tests below may be randomised.
doNotRandomiseExecutionOrder :: TestDefM a b c -> TestDefM a b c Source #
Declare that the order of execution of all tests below must not be randomised.
withExecutionOrderRandomisation :: ExecutionOrderRandomisation -> TestDefM a b c -> TestDefM a b c Source #
Annotate a test group with ExecutionOrderRandomisation
.
Declaring flakiness
flaky :: Int -> TestDefM a b c -> TestDefM a b c Source #
Mark a test suite as "potentially flaky".
This will retry any test in the given test group up to the given number of tries, and pass a test if it passes once. The test output will show which tests were flaky.
WARNING: This is only a valid approach to dealing with test flakiness if it is true that tests never pass accidentally. In other words: tests using flaky must be guaranteed to fail every time if an error is introduced in the code, it should only be added to deal with accidental failures, never accidental passes.
flakyWith :: Int -> String -> TestDefM a b c -> TestDefM a b c Source #
Like flaky
, but also shows the given message to the user whenever the test is flaky.
You could use it like this:
>>>
flakyWith 3 "Something sometimes goes wrong with the database, see issue 6346" ourTestSuite
notFlaky :: TestDefM a b c -> TestDefM a b c Source #
Mark a test suite as "must not be flaky".
This is useful to have a subgroup of a group marked as flaky
that must not be flaky afteral.
withFlakiness :: FlakinessMode -> TestDefM a b c -> TestDefM a b c Source #
Annotate a test group with FlakinessMode
.