= TBC: Test by Convention = TBC provides two features: - It attempts to compile and run all tests, even if some do not compile or run. - Aspiring to the write-it-once principle, tests following conventions require a lot less boilerplate. Inspired by test-based development, our aim is not to displace existing tools such as QuickCheck and HUnit but to make a project's tests more useful when things are in an inconsistent state. TBC is also useful for supporting 'what-if' experiments, aiding program comprehension. TBC is presently alpha. It has proven useful to the authors but is embryonic in many ways. The directory 'Sample/' contains a sample project following TBC conventions. We suggest running 'tbc' in that directory as a quick way of understanding what it can do. == Conventions == Tests live in $PROJECT/Tests. Test-specific hierarchical modules can also exist there. Within a test file, lines beginning with _ should _ when evaluated: "exception_" / throw an exception. "hunit_" / be HUnit tests: TBC applies "runTestTT . test" to them. "ok_" / not throw an exception. "prop_" / be QuickCheck tests: TBC applies "Test.QuickCheck.quickCheck" to them. "test_" / be boolean tests (of type Bool or IO Bool or IO () with a print statement or ...): TBC runs them and expects the final line of output to be "True". The tests themselves must ensure the test framework (QuickCheck, etc.) is in scope, i.e. be executable as they are in GHCi. The 'tbc' executable will search upwards for a '.cabal' file and assumes the 'dist/' directory is in the same place. == Invocation == There are two ways to use TBC: - as a Cabal 'test' hook. This is not recommended as packages that do this cannot be uploaded to hackage. Perhaps changes in Cabal will make this more useful. - invoking the 'tbc' executable. The 'tbc' executable will search upwards in the directory hierarchy for a .cabal file, and then go looking for tests in the current directory and its children. Presently it is best to run 'tbc' in the directory containing the tests (or a subdirectory of it). The Sample/ directory contains a sample project. After installing TBC: $ cd Sample $ runghc Setup configure $ tbc $ cd Tests $ tbc == Gotchas == Say: tbc -v for a verbose session. It seems that the parts of Cabal that TBC depends on change frequently. TBC has only been tested with Cabal-1.10.1.0 in the Haskell Platform 2011.2.0.1. It is probably not difficult to make it work with other Cabals. Mostly this involves copying out the guts of some of the Cabal modules. Conventions must retain the full name of the test, otherwise we run into some nasty lexical issues, e.g. to avoid: hunit_prop_blah =tName equals= prop_blah We only support GHCi on *NIX (including OS X) presently. This: $ tbc >> GHCi died. Output << GHCi, version 7.0.3: http://www.haskell.org/ghc/ :? for help TBC: fd:4: hFlush: resource vanished (Broken pipe) probably means that you need to reconfigure your project as one of libraries it depends on has a different hash but not a new version number. (For example, Cabal calls QuickCheck-2.4.0.1 the overly-specific QuickCheck-2.4.0.1-5898cfb0116fc1d6ecef04b3f053c323.) == Changes == 0.0.1 Initial release 0.0.2 The tbc binary exits with status 1 if anything goes wrong, 0 otherwise Fixed GHCi scoping issue (say ":l *filename" instead of ":l filename" - perhaps new with GHC 7. Andy Morris: support for Bird tracks Works with the Haskell Platform 2011.2.0.1 - GHC 7.0.3 - QuickCheck 2