shelltestrunner: A tool for testing command-line programs.

[ program, testing ] [ Propose Tags ]

Run a command-line program through "shell tests" defined in one or more test files. Each test specifies command-line arguments, some input, and expected output, stderr output and/or exit status. We use test-framework's test runner, so can run tests in parallel. shelltestrunner was inspired by the tests in John Wiegley's ledger project.



  -? --help[=FORMAT]           Show usage information (optional format)
  -V --version                 Show version information
  -v --verbose                 Higher verbosity
  -q --quiet                   Lower verbosity
  -d --debug                   show debug messages
     --debug-parse             show parsing debug messages and stop
     --execdir                 run tests in same directory as test file
     --extension=EXT           extension of test files when dirs specified (default=.test)
  -i --implicit=none|exit|all  provide implicit tests (default=exit)
  -w --with=EXECUTABLE         alternate executable, replaces the first word of test commands
     =OTHER FLAGS              any other flags are passed to test runner

A test file contains one or more shell tests, which look like this:

 # optional comment lines
 a one-line shell command to be tested
 stdin lines
 >>> [/regexp to match in stdout/]
 [or expected stdout lines
 >>>2 [/regexp to match in stderr/]
 [or expected stderr lines]
 >>>= expected exit status or /regexp/

The command line is required; all other fields are optional.
The expected stdout (>>>) and expected stderr (>>>2) fields can have either
a regular expression match pattern, in which case the test passes if the
output is matched, or 0 or more data lines, in which case the output
must match these exactly. The expected exit status (>>>=) field can have
either a numeric exit code or a /regexp/. A ! preceding a /regexp/ or exit
code negates the match. The regular expression syntax is that of the
pcre-light library with the dotall flag.

By default there is an implicit test for exit status=0, but no implicit test
for stdout or stderr.  You can change this with -i/--implicit-tests.

The command runs in your current directory unless you use --execdir.
You can use --with/-w to replace the first word of command lines
(everything up to the first space) with something else, eg to test a
different version of your program. To prevent this, start the command line
with a space.

Any unrecognised options will be passed through to test-framework's runner.
You may be able to get a big speedup by running tests in parallel: try -j8.
Versions 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 1.1, 1.2, 1.2.1, 1.3, 1.3.1, 1.3.2, 1.3.3, 1.3.4, 1.3.5, 1.9
Dependencies base (>=3 && <5), cmdargs (>=0.1), directory, FileManipCompat, filepath, HUnit, parsec, pcre-light (>=0.3.1), process, test-framework, test-framework-hunit (>=0.2) [details]
License LicenseRef-GPL
Author Simon Michael <>
Maintainer Simon Michael <>
Category Testing
Home page
Bug tracker
Source repo head: darcs get
Uploaded by SimonMichael at Fri Apr 9 23:35:06 UTC 2010
Distributions Debian:1.3.5, LTSHaskell:1.9, NixOS:1.9, Stackage:1.9, openSUSE:1.9
Executables shelltest
Downloads 10026 total (36 in the last 30 days)
Rating (no votes yet) [estimated by rule of succession]
Your Rating
  • λ
  • λ
  • λ
Status Docs not available [build log]
All reported builds failed as of 2016-12-30 [all 7 reports]
Hackage Matrix CI


Maintainer's Corner

For package maintainers and hackage trustees