h1. Test Framework

You can help improve this README with extra snippets and advice by using the "GitHub wiki":http://github.com/batterseapower/test-framework/wikis/readme.


h2. Installing

To just install the library:

<pre>
<code>runghc Setup.lhs configure
runghc Setup.lhs build
sudo runghc Setup.lhs install
</pre>
</code>

If you want to build the example, to check it's all working:

<pre>
<code>runghc Setup.lhs configure -fexample
runghc Setup.lhs build
dist/build/test-framework-example/test-framework-example
</pre>
</code>


h2. Description

A test framework, with built-in support for "HUnit":http://hackage.haskell.org/cgi-bin/hackage-scripts/package/HUnit and "QuickCheck":http://hackage.haskell.org/cgi-bin/hackage-scripts/package/QuickCheck tests. The main benefit of using this framework is that you get a nice console based test runner with the following features:

* Run tests in parallel but report results in a deterministic order (to aid diff-based analysis of test output)
* Progress reporting for individual QuickCheck properties and for the whole suite being run
* Filter the tests to be run using patterns specified on the command line
* Hierarchical, colored display of test results
* Reporting of test statistics (number of tests run, number failed etc.)
* Extensibility: add your own test providers above and beyond those provided
* Seed reporting upon a failed QuickCheck run, so you can reproduce the failure if necessary


h2. Example

An example testsuite is provided in the package, which you can build by supplying @-fexample@ to the Cabal @configure@ step as described above. You can also view the most recent version online at "GitHub":http://github.com/batterseapower/test-framework/tree/master/Test/Framework/Example.lhs.

There are two essential components to getting running with the test framework: setting up the tests to be run, and making the program run the tests in the provided console test runner.

You specify the tests to run in your code like so:

<pre>
<code>import Test.Framework
import Test.Framework.Providers.HUnit
import Test.Framework.Providers.QuickCheck

tests = [
        testGroup "Sorting Group 1" [
                testProperty "sort1" prop_sort1,
                testProperty "sort2" prop_sort2,
                testProperty "sort3" prop_sort3
            ],
        testGroup "Sorting Group 2" [
                testProperty "sort4" prop_sort4,
                testProperty "sort5" prop_sort5,
                testProperty "sort6" prop_sort6,
                testCase "sort7" test_sort7,
                testCase "sort8" test_sort8
            ]
    ]
</pre>
</code>

And set up the console runner by including this in your @Main@ module:

<pre><code>main = defaultMain tests</pre></code>


h2. Console Runner Manual

A description of the options available can be obtained by using the option @--help@ on the command line.

The test-selection syntax for use with the @-s@ command line option is based on that of shell globs or "Git .gitignore files":http://www.kernel.org/pub/software/scm/git/docs/gitignore.html. Test patterns are treated as follows:

* An optional prefix @!@ which negates the pattern
* If the pattern ends with a slash, it is removed for the purpose of the following description, but it would only find a match with a test group. In other words, @foo/@ will match a group called @foo@ and any tests underneath it, but will not match a regular test @foo@.
* If the pattern does not contain a slash @/@, the framework checks for a match against any single component of the path
* Otherwise, the pattern is treated as a glob, where the wildcard @*@ matches anything within a single path component (i.e. @foo@ but not @foo/bar@), two wildcards @**@ matches anything (i.e. @foo@ and @foo/bar@) and anything else matches exactly that text in the path (i.e. @foo@ would only match a component of the test path called @foo@ (or a substring of that form). For example, @group/*1@ matches @group/test1@ but not @group/subgroup/test1@, whereas both examples would be matched by @group/**1@.  A leading slash matches the beginning of the test path; for example, @/test*@ matches @test1@ but not @group/test1@.

A test will be run if it matches __any__ of the patterns supplied with @-s@.


h2. Linkage

* "Hackage":http://hackage.haskell.org/cgi-bin/hackage-scripts/package/test-framework/
* "Bug Tracker":http://bsp.lighthouseapp.com/projects/15661-hs-test-framework
* "GitHub":http://github.com/batterseapower/test-framework/