Safe Haskell | None |
---|
A black box test in the terminology of the HTF consists of a driver program that is run in various input files. For each input file, the HTF checks that the driver program exits with the correct exit code and that it produces the expected output. The samples directory of the HTF source tree shows an example for a black box test, see https://github.com/skogsbaer/HTF/tree/master/sample.
NOTE: If you use black box tests, you have to compile your program
with the -threaded
option. Otherwise, your program just blocks
indefinitely!
Documentation
Use a value of this datatype to customize various aspects of your black box tests.
BBTArgs | |
|
:: FilePath | root directory of the test hierarchy |
-> String | name of program under test |
-> String | filename suffix for input file |
-> BBTArgs | configuration |
-> IO [Test] |
Collects all black box tests with the given file extension stored in a specific directory. For example, the invocation
blackBoxTests "bbt-dir" "dist/build/sample/sample" ".num" defaultBBTArgs
returns a list of Test
values, one Test
for each .num
file found in
bbt-dir
and its subdirectories. (The samples directory of the HTF source tree
contains the example shown here,
see https://github.com/skogsbaer/HTF/tree/master/sample.)
Suppose that one of the .num
files
is bbt-dir/should-pass/x.num
. Running the corresponding Test
invokes
dist/build/sample/sample
(the program under test) with bbt-dir/should-pass/x.num
as input file. If bbt-dir/should-pass/x.num
existed, its content
would be used as stdin. The tests succeeds
if the exit code of the program is zero and
the output on stdout and stderr matches the contents of
bbt-dir/should-pass/x.out
and bbt-dir/should-pass/x.err
, respectively.
The directory bbt-dir/should-fail
contains the file BBTArgs
. If this file
exists, then its content specifies various
arguments for the test run. The file defines the arguments separated by newlines.
Supported arguments:
Skip
- Skips all tests in the same directory as the argument file.
Fail
- Specify that the test should succeed if it exits with a non-zero exit code.
Flags: flags
- Passes the given
flags
to the program under test.
type Diff = Maybe FilePath -> String -> IO (Maybe String)Source
The type of a function comparing the content of a file
against a string, similar to the unix tool diff
.
The first parameter is the name of the file containing the
expected output. If this parameter is Nothing
, then no output
is expected. The second parameter is the actual output produced.
If the result is Nothing
then no difference was found.
Otherwise, a Just
value contains a string explaining the
different.
A default value for the Diff
datatype that simple resorts to the
diff
commandline utility.