Safe Haskell | None |
---|
API for test runners
- data TestTree
- = forall t . IsTest t => SingleTest TestName t
- | TestGroup TestName [TestTree]
- | PlusTestOptions (OptionSet -> OptionSet) TestTree
- | WithResource ResourceSpec TestTree
- | AskOptions (OptionSet -> TestTree)
- foldTestTree :: Monoid b => (forall t. IsTest t => OptionSet -> TestName -> t -> b) -> (TestName -> b -> b) -> (ResourceSpec -> b -> b) -> OptionSet -> TestTree -> b
- newtype AppMonoid f = AppMonoid {
- getApp :: f ()
- data ResourceSpec = forall a . ResourceSpec (IO a) (a -> IO ())
- data Ingredient
- = TestReporter [OptionDescription] (OptionSet -> TestTree -> Maybe (StatusMap -> IO Bool))
- | TestManager [OptionDescription] (OptionSet -> TestTree -> Maybe (IO Bool))
- tryIngredients :: [Ingredient] -> OptionSet -> TestTree -> Maybe (IO Bool)
- ingredientOptions :: Ingredient -> [OptionDescription]
- ingredientsOptions :: [Ingredient] -> [OptionDescription]
- consoleTestReporter :: Ingredient
- listingTests :: Ingredient
- newtype ListTests = ListTests Bool
- testsNames :: OptionSet -> TestTree -> [TestName]
- optionParser :: [OptionDescription] -> Parser OptionSet
- suiteOptionParser :: [Ingredient] -> TestTree -> Parser OptionSet
- defaultMainWithIngredients :: [Ingredient] -> TestTree -> IO ()
- data Status
- data Result = Result {}
- data Progress = Progress {}
- type StatusMap = IntMap (TVar Status)
- launchTestTree :: OptionSet -> TestTree -> IO StatusMap
- newtype NumThreads = NumThreads {
- getNumThreads :: Int
- suiteOptions :: [Ingredient] -> TestTree -> [OptionDescription]
- coreOptions :: [OptionDescription]
- data TestPattern
- parseTestPattern :: String -> TestPattern
- noPattern :: TestPattern
- testPatternMatches :: TestPattern -> [String] -> Bool
Working with the test tree
The main data structure defining a test suite.
It consists of individual test cases and properties, organized in named groups which form a tree-like hierarchy.
There is no generic way to create a test case. Instead, every test
provider (tasty-hunit, tasty-smallcheck etc.) provides a function to
turn a test case into a TestTree
.
Groups can be created using testGroup
.
forall t . IsTest t => SingleTest TestName t | A single test of some particular type |
TestGroup TestName [TestTree] | Assemble a number of tests into a cohesive group |
PlusTestOptions (OptionSet -> OptionSet) TestTree | Add some options to child tests |
WithResource ResourceSpec TestTree | |
AskOptions (OptionSet -> TestTree) |
:: Monoid b | |
=> (forall t. IsTest t => OptionSet -> TestName -> t -> b) | interpret a single test |
-> (TestName -> b -> b) | interpret a test group |
-> (ResourceSpec -> b -> b) | deal with the resource |
-> OptionSet | initial options |
-> TestTree | |
-> b |
Fold a test tree into a single value.
Apart from pure convenience, this function also does the following useful things:
- Keeping track of the current options (which may change due to
PlusTestOptions
nodes) - Filtering out the tests which do not match the patterns
Thus, it is preferred to an explicit recursive traversal of the tree.
Note: right now, the patterns are looked up only once, and won't be affected by the subsequent option changes. This shouldn't be a problem in practice; OTOH, this behaviour may be changed later.
Useful wrapper for use with foldTestTree
Applicative f => Monoid (AppMonoid f) |
data ResourceSpec Source
ResourceSpec
describes how to acquire a resource (the first field)
and how to release it (the second field).
forall a . ResourceSpec (IO a) (a -> IO ()) |
Ingredients
data Ingredient Source
Ingredient
s make your test suite tasty.
Ingredients represent different actions that you can perform on your test suite. One obvious ingredient that you want to include is one that runs tests and reports the progress and results.
Another standard ingredient is one that simply prints the names of all tests.
Similar to test providers (see IsTest
), every ingredient may specify
which options it cares about, so that those options are presented to
the user if the ingredient is included in the test suite.
An ingredient can choose, typically based on the OptionSet
, whether to
run. That's what the Maybe
is for. The first ingredient that agreed to
run does its work, and the remaining ingredients are ignored. Thus, the
order in which you arrange the ingredients may matter.
Usually, the ingredient which runs the tests is unconditional and thus should be placed last in the list. Other ingredients usually run only if explicitly requested via an option. Their relative order thus doesn't matter.
That's all you need to know from an (advanced) user perspective. Read on if you want to create a new ingredient.
There are two kinds of ingredients. TestReporter
, if it agrees to run,
automatically launches tests execution, and gets the StatusMap
which
it uses to report the progress and results to the user.
TestManager
is the second kind of ingredient. It is typically used for
test management purposes (such as listing the test names), although it
can also be used for running tests (but, unlike TestReporter
, it has
to launch the tests manually). It is therefore more general than
TestReporter
. TestReporter
is provided just for convenience.
The function's result should indicate whether all the tests passed.
In the TestManager
case, it's up to the ingredient author to decide
what the result should be. When no tests are run, the result should
probably be True
. Sometimes, even if some tests run and fail, it still
makes sense to return True
.
TestReporter [OptionDescription] (OptionSet -> TestTree -> Maybe (StatusMap -> IO Bool)) | |
TestManager [OptionDescription] (OptionSet -> TestTree -> Maybe (IO Bool)) |
tryIngredients :: [Ingredient] -> OptionSet -> TestTree -> Maybe (IO Bool)Source
Run the first Ingredient
that agrees to be run.
If no one accepts the task, return Nothing
. This is usually a sign of
misconfiguration.
ingredientOptions :: Ingredient -> [OptionDescription]Source
Return the options which are relevant for the given ingredient.
Note that this isn't the same as simply pattern-matching on
Ingredient
. E.g. options for a TestReporter
automatically include
NumThreads
.
ingredientsOptions :: [Ingredient] -> [OptionDescription]Source
Like ingredientOption
, but folds over multiple ingredients.
Standard console ingredients
Console test reporter
consoleTestReporter :: IngredientSource
A simple console UI
Tests list
listingTests :: IngredientSource
The ingredient that provides the test listing functionality
This option, when set to True
, specifies that we should run in the
«list tests» mod
testsNames :: OptionSet -> TestTree -> [TestName]Source
Obtain the list of all tests in the suite
Command line handling
optionParser :: [OptionDescription] -> Parser OptionSetSource
Generate a command line parser from a list of option descriptions
suiteOptionParser :: [Ingredient] -> TestTree -> Parser OptionSetSource
The command line parser for the test suite
defaultMainWithIngredients :: [Ingredient] -> TestTree -> IO ()Source
Parse the command line arguments and run the tests using the provided ingredient list
Running tests
Current status of a test
NotStarted | test has not started running yet |
Executing Progress | test is being run |
Exception SomeException | test threw an exception and was aborted |
Done Result | test finished with a given result |
A test result
Result | |
|
Test progress information.
This may be used by a runner to provide some feedback to the user while a long-running test is executing.
Progress | |
|
type StatusMap = IntMap (TVar Status)Source
Mapping from test numbers (starting from 0) to their status variables.
This is what an ingredient uses to analyse and display progress, and to detect when tests finish.
launchTestTree :: OptionSet -> TestTree -> IO StatusMapSource
Start running all the tests in a test tree in parallel. The number of
threads is determined by the NumThreads
option.
Return a map from the test number (starting from 0) to its status variable.
newtype NumThreads Source
Number of parallel threads to use for running tests.
Note that this is not included in coreOptions
.
Instead, it's automatically included in the options for any
TestReporter
ingredient by ingredientOptions
, because the way test
reporters are handled already involves parallelism. Other ingredients
may also choose to include this option.
Options
suiteOptions :: [Ingredient] -> TestTree -> [OptionDescription]Source
All the options relevant for this test suite. This includes the options for the test tree and ingredients, and the core options.
coreOptions :: [OptionDescription]Source
The list of all core options, i.e. the options not specific to any
provider or ingredient, but to tasty itself. Currently only contains TestPattern
.
Patterns
data TestPattern Source
A pattern to filter tests. For the syntax description, see http://documentup.com/feuerbach/tasty#using-patterns
parseTestPattern :: String -> TestPatternSource
Parse a pattern
noPattern :: TestPatternSource
A pattern that matches anything.
testPatternMatches :: TestPattern -> [String] -> BoolSource
Test a path (which is the sequence of group titles, possibly followed by the test title) against a pattern