LLVM.Analysis.Util.Testing

Various functions to help test this library and analyses based on it.

The idea behind the test framework is that each TestDescriptor describes inputs for a test suite and automatically converts the inputs to a summary value, which it compares against an expected value. It reports how many such tests pass/fail.

More concretely, each test suite specifies:

• The test input files (via a shell glob)
• A function to conver a test input file name to a filename containing the expected outut.
• A summary function to reduce a Module to a summary value
• A comparison function (usually an assertion from HUnit)

With these components, the framework reads each input file and converts it to bitcode. It uses the summary function to reduce the Module to a summary value and reads the expected output (using the read function). These two types (the summary and expected output) must be identical. The comparison function is then applied. If it throws an exception, the test is considered to have failed.

NOTE 1: The result type of the summary function MUST be an instance of Read AND the same as the type found in the expected results file.

NOTE 2: The test inputs can be C, C++, bitcode, or LLVM assembly files.

Types

data TestDescriptor

A description of a set of tests.

Constructors

forall a . Read a => TestDescriptor

Fields

testPattern :: String

A shell glob pattern (relative to the project root) that collects all test inputs

testExpectedMapping :: FilePath -> FilePath

A function to apply to an input file name to find the file containing its expected results

testResultBuilder :: Module -> a

A function to turn a Module into a summary value of any type

testResultComparator :: String -> a -> a -> IO ()

A function to compare two summary values (throws on failure)

data BuildException

Constructors

ClangFailed FilePath ExitCode

NoBuildMethodForInput FilePath

OptFailed FilePath ExitCode

NoOptBinaryFound

Instances

Show BuildException

Typeable BuildException

Exception BuildException

Actions

testAgainstExpected

Arguments

:: [String]

Options for opt

-> (FilePath -> IO Module)

A function to turn a bitcode file bytestring into a Module

-> [TestDescriptor]

The list of test suites to run

-> IO ()

This is the main test suite entry point. It takes a bitcode parser and a list of test suites.

The bitcode parser is taken as an input so that this library does not have a direct dependency on any FFI code.

Helpers

buildModule

Arguments

:: [String]

Front-end options (passed to clang) for the module.

-> [String]

Optimization options (passed to opt) for the module. opt is not run if the list is empty

-> (FilePath -> IO Module)

A function to turn a bitcode file into a Module

-> FilePath

The input file (either bitcode or C/C++)

-> IO Module

Given an input file, bitcode parsing function, and options to pass to opt, return a Module. The input file can be C, C++, or LLVM bitcode.

Note that this function returns an Either value to report some kinds of errors. It can also raise IOErrors.

readInputAndExpected

Arguments

:: Read a

=> [String]

Arguments for opt

-> (FilePath -> IO Module)

A function to turn a bitcode file bytestring into a Module

-> (FilePath -> FilePath)

The function to map an input file name to the expected output file

-> FilePath

The input file

-> IO (FilePath, Module, a)

An intermediate helper to turn input files into modules and return the expected output.