tasty- Modern and extensible testing framework

Safe HaskellNone




API for test runners


Working with the test tree

data TestTree Source

The main data structure defining a test suite.

It consists of individual test cases and properties, organized in named groups which form a tree-like hierarchy.

There is no generic way to create a test case. Instead, every test provider (tasty-hunit, tasty-smallcheck etc.) provides a function to turn a test case into a TestTree.

Groups can be created using testGroup.


forall t . IsTest t => SingleTest TestName t

A single test of some particular type

TestGroup TestName [TestTree]

Assemble a number of tests into a cohesive group

PlusTestOptions (OptionSet -> OptionSet) TestTree

Add some options to child tests

forall a . WithResource (ResourceSpec a) (IO a -> TestTree)

Acquire the resource before the tests in the inner tree start and release it after they finish. The tree gets an IO action which yields the resource, although the resource is shared across all the tests.

AskOptions (OptionSet -> TestTree)

Ask for the options and customize the tests based on them



:: Monoid b 
=> TreeFold b

the algebra (i.e. how to fold a tree)

-> OptionSet

initial options

-> TestTree

the tree to fold

-> b 

Fold a test tree into a single value.

The fold result type should be a monoid. This is used to fold multiple results in a test group. In particular, empty groups get folded into mempty.

Apart from pure convenience, this function also does the following useful things:

  1. Keeping track of the current options (which may change due to PlusTestOptions nodes)
  2. Filtering out the tests which do not match the patterns

Thus, it is preferred to an explicit recursive traversal of the tree.

Note: right now, the patterns are looked up only once, and won't be affected by the subsequent option changes. This shouldn't be a problem in practice; OTOH, this behaviour may be changed later.

data TreeFold b Source

An algebra for folding a TestTree.

Instead of constructing fresh records, build upon trivialFold instead. This way your code won't break when new nodes/fields are indroduced.




foldSingle :: forall t. IsTest t => OptionSet -> TestName -> t -> b
foldGroup :: TestName -> b -> b
foldResource :: forall a. ResourceSpec a -> (IO a -> b) -> b

trivialFold :: Monoid b => TreeFold bSource

trivialFold can serve as the basis for custom folds. Just override the fields you need.

Here's what it does:

  • single tests are mapped to mempty (you probably do want to override that)
  • test groups are returned unmodified
  • for a resource, an IO action that throws an exception is passed (you want to override this for runners/ingredients that execute tests)

data ResourceSpec a Source

ResourceSpec describes how to acquire a resource (the first field) and how to release it (the second field).


ResourceSpec (IO a) (a -> IO ()) 

newtype Traversal f Source

Monoid generated by *>




getTraversal :: f ()


newtype Ap f a Source

Monoid generated by liftA2 (<>)




getApp :: f a


Monad f => Monad (Ap f) 
Functor f => Functor (Ap f) 
Applicative f => Applicative (Ap f) 
(Applicative f, Monoid a) => Monoid (Ap f a) 


data Ingredient Source

Ingredients make your test suite tasty.

Ingredients represent different actions that you can perform on your test suite. One obvious ingredient that you want to include is one that runs tests and reports the progress and results.

Another standard ingredient is one that simply prints the names of all tests.

Similar to test providers (see IsTest), every ingredient may specify which options it cares about, so that those options are presented to the user if the ingredient is included in the test suite.

An ingredient can choose, typically based on the OptionSet, whether to run. That's what the Maybe is for. The first ingredient that agreed to run does its work, and the remaining ingredients are ignored. Thus, the order in which you arrange the ingredients may matter.

Usually, the ingredient which runs the tests is unconditional and thus should be placed last in the list. Other ingredients usually run only if explicitly requested via an option. Their relative order thus doesn't matter.

That's all you need to know from an (advanced) user perspective. Read on if you want to create a new ingredient.

There are two kinds of ingredients. TestReporter, if it agrees to run, automatically launches tests execution, and gets the StatusMap which it uses to report the progress and results to the user.

TestManager is the second kind of ingredient. It is typically used for test management purposes (such as listing the test names), although it can also be used for running tests (but, unlike TestReporter, it has to launch the tests manually). It is therefore more general than TestReporter. TestReporter is provided just for convenience.

The function's result should indicate whether all the tests passed.

In the TestManager case, it's up to the ingredient author to decide what the result should be. When no tests are run, the result should probably be True. Sometimes, even if some tests run and fail, it still makes sense to return True.

tryIngredients :: [Ingredient] -> OptionSet -> TestTree -> Maybe (IO Bool)Source

Run the first Ingredient that agrees to be run.

If no one accepts the task, return Nothing. This is usually a sign of misconfiguration.

ingredientOptions :: Ingredient -> [OptionDescription]Source

Return the options which are relevant for the given ingredient.

Note that this isn't the same as simply pattern-matching on Ingredient. E.g. options for a TestReporter automatically include NumThreads.

ingredientsOptions :: [Ingredient] -> [OptionDescription]Source

Like ingredientOption, but folds over multiple ingredients.

Standard console ingredients

NOTE: the exports in this section are deprecated and will be removed in the future. Please import Test.Tasty.Ingredients.Basic if you need them.

Console test reporter

consoleTestReporter :: IngredientSource

A simple console UI

Tests list

listingTests :: IngredientSource

The ingredient that provides the test listing functionality

newtype ListTests Source

This option, when set to True, specifies that we should run in the «list tests» mod


ListTests Bool 

testsNames :: OptionSet -> TestTree -> [TestName]Source

Obtain the list of all tests in the suite

Command line handling

optionParser :: [OptionDescription] -> Parser OptionSetSource

Generate a command line parser from a list of option descriptions

suiteOptionParser :: [Ingredient] -> TestTree -> Parser OptionSetSource

The command line parser for the test suite

defaultMainWithIngredients :: [Ingredient] -> TestTree -> IO ()Source

Parse the command line arguments and run the tests using the provided ingredient list

Running tests

data Status Source

Current status of a test



test has not started running yet

Executing Progress

test is being run

Done Result

test finished with a given result

data Result Source

A test result




resultOutcome :: Outcome

Did the test fail? If so, why?

resultDescription :: String

resultDescription may contain some details about the test. For a passed test it's ok to leave it empty. Providers like SmallCheck and QuickCheck use it to provide information about how many tests were generated.

For a failed test, resultDescription should typically provide more information about the failure.

data Outcome Source

Outcome of a test run

Note: this is isomorphic to Maybe FailureReason. You can use the generic-maybe package to exploit that.



test succeeded

Failure FailureReason

test failed because of the FailureReason

data FailureReason Source

If a test failed, FailureReason describes why



test provider indicated failure

TestThrewException SomeException

test resulted in an exception. Note that some test providers may catch exceptions in order to provide more meaningful errors. In that case, the FailureReason will be TestFailed, not TestThrewException.

TestTimedOut Integer

test didn't complete in allotted time


resultSuccessful :: Result -> BoolSource

True for a passed test, False for a failed one.

data Progress Source

Test progress information.

This may be used by a runner to provide some feedback to the user while a long-running test is executing.




progressText :: String

textual information about the test's progress

progressPercent :: Float

progressPercent should be a value between 0 and 1. If it's impossible to compute the estimate, use 0.

type StatusMap = IntMap (TVar Status)Source

Mapping from test numbers (starting from 0) to their status variables.

This is what an ingredient uses to analyse and display progress, and to detect when tests finish.

launchTestTree :: OptionSet -> TestTree -> (StatusMap -> IO a) -> IO aSource

Start running all the tests in a test tree in parallel. The number of threads is determined by the NumThreads option.

Return a map from the test number (starting from 0) to its status variable.

newtype NumThreads Source

Number of parallel threads to use for running tests.

Note that this is not included in coreOptions. Instead, it's automatically included in the options for any TestReporter ingredient by ingredientOptions, because the way test reporters are handled already involves parallelism. Other ingredients may also choose to include this option.




getNumThreads :: Int


suiteOptions :: [Ingredient] -> TestTree -> [OptionDescription]Source

All the options relevant for this test suite. This includes the options for the test tree and ingredients, and the core options.

coreOptions :: [OptionDescription]Source

The list of all core options, i.e. the options not specific to any provider or ingredient, but to tasty itself. Currently contains TestPattern and Timeout.


noPattern :: TestPatternSource

A pattern that matches anything.

testPatternMatches :: TestPattern -> [String] -> BoolSource

Test a path (which is the sequence of group titles, possibly followed by the test title) against a pattern