Test.Tasty.Runners

API for test runners

Working with the test tree

data TestTree

The main data structure defining a test suite.

It consists of individual test cases and properties, organized in named groups which form a tree-like hierarchy.

There is no generic way to create a test case. Instead, every test provider (tasty-hunit, tasty-smallcheck etc.) provides a function to turn a test case into a TestTree.

Groups can be created using testGroup.

Constructors

forall t . IsTest t => SingleTest TestName t

A single test of some particular type

TestGroup TestName [TestTree]

Assemble a number of tests into a cohesive group

PlusTestOptions (OptionSet -> OptionSet) TestTree

Add some options to child tests

foldTestTree

Arguments

:: Monoid b

=> (forall t. IsTest t => OptionSet -> TestName -> t -> b)

interpret a single test

-> (TestName -> b -> b)

interpret a test group

-> OptionSet

initial options

-> TestTree

-> b

Fold a test tree into a single value.

Apart from pure convenience, this function also does the following useful things:

1. Keeping track of the current options (which may change due to PlusTestOptions nodes)
2. Filtering out the tests which do not match the patterns

Thus, it is preferred to an explicit recursive traversal of the tree.

Note: right now, the patterns are looked up only once, and won't be affected by the subsequent option changes. This shouldn't be a problem in practice; OTOH, this behaviour may be changed later.

Command line handling

treeOptionParser :: TestTree -> Parser OptionSet

Generate a command line parser for all the options relevant for this test tree. Also includes coreOptions.

optionParser :: [OptionDescription] -> Parser OptionSet

Generate a command line parser from a list of option descriptions

defaultMainWithRunner :: Runner -> TestTree -> IO ()

Parse the command line arguments and run the tests using the provided runner

Running tests

data Status

Current status of a test

Constructors

NotStarted

test has not started running yet

Executing Progress

test is being run

Exception SomeException

test threw an exception and was aborted

Done Result

test finished with a given result

type StatusMap = IntMap (TVar Status)

Mapping from test numbers (starting from 0) to their status variables.

This is what a runner uses to analyse and display progress, and to detect when tests finish.

type Runner = OptionSet -> TestTree -> StatusMap -> IO Bool

A Runner is responsible for user interaction during the test run.

It is provided with the StatusMap, so the tests are already launched and all it needs to do is notifying the user about the progress and then displaying the overall results in the end.

The function's result should indicate whether all the tests passed.

execRunner :: Runner -> OptionSet -> TestTree -> IO Bool

Execute a Runner.

This is a shortcut which runs launchTestTree behind the scenes.

launchTestTree :: OptionSet -> TestTree -> IO StatusMap

Start running all the tests in a test tree in parallel. The number of threads is determined by the NumThreads option.

Return a map from the test number (starting from 0) to its status variable.

runUI :: Runner

A simple console UI

Core options

newtype NumThreads

Number of parallel threads to use for running tests

Constructors

NumThreads

Fields

getNumThreads :: Int

Instances

Eq NumThreads

Num NumThreads

Ord NumThreads

Typeable NumThreads

IsOption NumThreads

coreOptions :: [OptionDescription]

The list of all core options

Patterns

data TestPattern

A pattern to filter tests. For the syntax description, see http://documentup.com/feuerbach/tasty#using-patterns

Instances

Read TestPattern

Typeable TestPattern

IsOption TestPattern

parseTestPattern :: String -> TestPattern

Parse a pattern

noPattern :: TestPattern

A pattern that matches anything.

testPatternMatches :: TestPattern -> [String] -> Bool

Test a path (which is the sequence of group titles, possibly followed by the test title) against a pattern