shelltestrunner: A tool for testing command-line programs.

[ program, testing ] [ Propose Tags ]

Run a command-line program through "shell tests" defined in one or more test files. Each test specifies command-line arguments, some input, and expected output, stderr output and/or exit status. We use test-framework's test runner, so can run tests in parallel. shelltestrunner was inspired by the tests in John Wiegley's ledger project.


shelltestrunner [FLAGS] EXECUTABLE [TESTFILES ...]

  -? --help[=FORMAT]           Show usage information (optional format)
  -V --version                 Show version information
  -v --verbose                 Higher verbosity
  -q --quiet                   Lower verbosity
  -d --debug                   show debug messages
     --debug-parse             show parsing debug messages and stop
  -i --implicit=none|exit|all  provide implicit tests (default=exit)

Any unrecognised options will be passed through to test-framework's runner. You may be able to get a big speedup by running tests in parallel: try -j16.

A test file contains one or more shell tests, each of which looks like this:

# optional comment lines beginning with #
-opt1 -opt2 arg1 arg2 # one line of command line args, executable will be prepended
0 or more lines of input
>>> [/regexp/] [optional comment beginning with #]
[..or 0 or more lines of expected output]
>>>2 [/regexp/] [optional comment]
[..or 0 or more lines of expected error output]
>>>= [expected exit status or /regexp/] [optional comment]
# optional comment lines

The expected output and expected error output fields can have either a regular expression match expression, in which case the test passes if the output is matched, or 0 or more following data lines, in which case the output must match these exactly. The expected exit status field can have either a numeric exit code or a regular expression match expression. A ! preceding a an exit code or /regexp/ negates the match. The regular expression syntax is that of the pcre-light library with the dotall flag.

The command line args field is required, all others are optional. By default there is an implicit test for exit status = 0, but no implicit test for stdout or stderr. You can change this with the -i/--implicit-tests flag.

Some issues to be aware of:

Versions 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 1.1, 1.2, 1.2.1, 1.3, 1.3.1, 1.3.2, 1.3.3, 1.3.4, 1.3.5, 1.9
Dependencies base (>=3 && <5), cmdargs (>=0.1), HUnit, parsec, pcre-light (>=0.3.1), process, test-framework, test-framework-hunit (>=0.2) [details]
License LicenseRef-GPL
Author Simon Michael <>
Maintainer Simon Michael <>
Category Testing
Home page
Bug tracker
Source repo head: darcs get
Uploaded by SimonMichael at Sat Mar 6 05:34:22 UTC 2010
Distributions Debian:1.3.5, LTSHaskell:1.9, NixOS:1.9, Stackage:1.9, openSUSE:1.9
Executables shelltestrunner
Downloads 10088 total (67 in the last 30 days)
Rating (no votes yet) [estimated by rule of succession]
Your Rating
  • λ
  • λ
  • λ
Status Docs not available [build log]
All reported builds failed as of 2016-12-30 [all 7 reports]
Hackage Matrix CI


Maintainer's Corner

For package maintainers and hackage trustees