name: shelltestrunner version: 0.7 category: Testing synopsis: A tool for testing command-line programs. description: . Run a command-line program through \"shell tests\" defined in one or more test files. Each test specifies command-line arguments, some input, and expected output, stderr output and/or exit status. We use test-framework's test runner, so can run tests in parallel. shelltestrunner was inspired by the tests in John Wiegley's ledger project. . Usage: . > shelltestrunner [FLAGS] EXECUTABLE [TESTFILES ...] > > -? --help[=FORMAT] Show usage information (optional format) > -V --version Show version information > -v --verbose Higher verbosity > -q --quiet Lower verbosity > -d --debug show debug messages > --debug-parse show parsing debug messages and stop > -i --implicit=none|exit|all provide implicit tests (default=exit) . Any unrecognised options will be passed through to test-framework's runner. You may be able to get a big speedup by running tests in parallel: try -j16. . A test file contains one or more shell tests, each of which looks like this: . > # optional comment lines beginning with # > -opt1 -opt2 arg1 arg2 # one line of command line args, executable will be prepended > <<< > 0 or more lines of input > >>> [/regexp/] [optional comment beginning with #] > [..or 0 or more lines of expected output] > >>>2 [/regexp/] [optional comment] > [..or 0 or more lines of expected error output] > >>>= [expected exit status or /regexp/] [optional comment] > # optional comment lines . The expected output and expected error output fields can have either a regular expression match expression, in which case the test passes if the output is matched, or 0 or more following data lines, in which case the output must match these exactly. The expected exit status field can have either a numeric exit code or a regular expression match expression. A ! preceding a an exit code or \/regexp\/ negates the match. The regular expression syntax is that of the pcre-light library with the dotall flag. . The command line args field is required, all others are optional. By default there is an implicit test for exit status = 0, but no implicit test for stdout or stderr. You can change this with the -i/--implicit-tests flag. . Some issues to be aware of: . - uncompiled haskell scripts can't be tested due to http://hackage.haskell.org/trac/ghc/ticket/3890 . - unparseable test files and uncompilable regexps are counted as test failures . license: GPL license-file: LICENSE author: Simon Michael maintainer: Simon Michael homepage: http://joyful.com/darcsweb/darcsweb.cgi?r=shelltestrunner bug-reports: mailto:simon@joyful.com stability: beta tested-with: GHC==6.10 cabal-version: >= 1.6 build-type: Simple extra-tmp-files: extra-source-files: executable shelltestrunner main-is: shelltestrunner.hs ghc-options: -Wall -threaded build-depends: base >= 3 && < 5 ,parsec ,process ,HUnit ,test-framework ,test-framework-hunit >= 0.2 ,cmdargs >= 0.1 ,pcre-light >= 0.3.1 source-repository head type: darcs location: http://joyful.com/repos/hledger