Futhark.Test

Facilities for reading Futhark test programs. A Futhark test program is an ordinary Futhark program where an initial comment block specifies input- and output-sets.

Documentation

testSpecFromFile :: FilePath -> IO ProgramTest

Read the test specification from the given Futhark program. Note: will call error on parse errors.

testSpecsFromPaths :: [FilePath] -> IO [(FilePath, ProgramTest)]

Read test specifications from the given paths, which can be a files or directories containing .fut files and further directories. Calls error on parse errors, or if any of the immediately passed path names do not name a file that exists.

valuesFromByteString :: String -> ByteString -> Either String [Value]

Try to parse a several values from a byte string. The String parameter is used for error messages.

getValues :: MonadIO m => FilePath -> Values -> m [Value]

Get the actual core Futhark values corresponding to a Values specification. The FilePath is the directory which file paths are read relative to.

getValuesBS :: MonadIO m => FilePath -> Values -> m ByteString

Extract a pretty representation of some Values. In the IO monad because this might involve reading from a file. There is no guarantee that the resulting byte string yields a readable value.

compareValues :: [Value] -> [Value] -> [Mismatch]

Compare two sets of Futhark values for equality. Shapes and types must also match.

compareValues1 :: [Value] -> [Value] -> Maybe Mismatch

As compareValues, but only reports one mismatch.

testRunReferenceOutput :: FilePath -> Text -> TestRun -> FilePath

When/if generating a reference output file for this run, what should it be called? Includes the "data/" folder.

getExpectedResult :: MonadIO m => FilePath -> Text -> TestRun -> m (ExpectedResult [Value])

Get the values corresponding to an expected result, if any.

compileProgram :: (MonadIO m, MonadError [Text] m) => [String] -> FilePath -> String -> FilePath -> m (ByteString, ByteString)

runProgram :: MonadIO m => String -> [String] -> String -> Text -> Values -> m (ExitCode, ByteString, ByteString)

ensureReferenceOutput :: (MonadIO m, MonadError [Text] m) => FilePath -> String -> FilePath -> [InputOutputs] -> m ()

Ensure that any reference output files exist, or create them (by compiling the program with the reference compiler and running it on the input) if necessary.

determineTuning :: MonadIO m => Maybe FilePath -> FilePath -> m ([String], String)

Determine the --tuning options to pass to the program. The first argument is the extension of the tuning file, or Nothing if none should be used.

data Mismatch

Two values differ in some way. The Show instance produces a human-readable explanation.

Instances

Show Mismatch

Defined in Futhark.Test.Values

Methods

showsPrec :: Int -> Mismatch -> ShowS

show :: Mismatch -> String

showList :: [Mismatch] -> ShowS

data ProgramTest

Description of a test to be carried out on a Futhark program. The Futhark program is stored separately.

Constructors

ProgramTest

Fields

• testDescription :: Text
• testTags :: [Text]
• testAction :: TestAction

Instances

Show ProgramTest

Defined in Futhark.Test

Methods

showsPrec :: Int -> ProgramTest -> ShowS

show :: ProgramTest -> String

showList :: [ProgramTest] -> ShowS

data StructureTest

A structure test specifies a compilation pipeline, as well as metrics for the program coming out the other end.

Constructors

StructureTest StructurePipeline AstMetrics

Instances

Show StructureTest

Defined in Futhark.Test

Methods

showsPrec :: Int -> StructureTest -> ShowS

show :: StructureTest -> String

showList :: [StructureTest] -> ShowS

data StructurePipeline

How a program can be transformed.

Constructors

KernelsPipeline

SOACSPipeline

SequentialCpuPipeline

GpuPipeline

Instances

Show StructurePipeline

Defined in Futhark.Test

Methods

showsPrec :: Int -> StructurePipeline -> ShowS

show :: StructurePipeline -> String

showList :: [StructurePipeline] -> ShowS

data WarningTest

A warning test requires that a warning matching the regular expression is produced. The program must also compile succesfully.

Constructors

ExpectedWarning Text Regex

Instances

Show WarningTest

Defined in Futhark.Test

Methods

showsPrec :: Int -> WarningTest -> ShowS

show :: WarningTest -> String

showList :: [WarningTest] -> ShowS

data TestAction

How to test a program.

Constructors

CompileTimeFailure ExpectedError

RunCases [InputOutputs] [StructureTest] [WarningTest]

Instances

Show TestAction

Defined in Futhark.Test

Methods

showsPrec :: Int -> TestAction -> ShowS

show :: TestAction -> String

showList :: [TestAction] -> ShowS

data ExpectedError

The error expected for a negative test.

Constructors

AnyError

ThisError Text Regex

Instances

Show ExpectedError

Defined in Futhark.Test

Methods

showsPrec :: Int -> ExpectedError -> ShowS

show :: ExpectedError -> String

showList :: [ExpectedError] -> ShowS

data InputOutputs

Input and output pairs for some entry point(s).

Constructors

InputOutputs

Fields

• iosEntryPoint :: Text
• iosTestRuns :: [TestRun]

Instances

Show InputOutputs

Defined in Futhark.Test

Methods

showsPrec :: Int -> InputOutputs -> ShowS

show :: InputOutputs -> String

showList :: [InputOutputs] -> ShowS

data TestRun

A condition for execution, input, and expected result.

Constructors

TestRun

Fields

• runTags :: [String]
• runInput :: Values
• runExpectedResult :: ExpectedResult Success
• runIndex :: Int
• runDescription :: String

Instances

Show TestRun

Defined in Futhark.Test

Methods

showsPrec :: Int -> TestRun -> ShowS

show :: TestRun -> String

showList :: [TestRun] -> ShowS

data ExpectedResult values

How a test case is expected to terminate.

Constructors

Succeeds (Maybe values)

Execution suceeds, with or without expected result values.

RunTimeFailure ExpectedError

Execution fails with this error.

Instances

Show values => Show (ExpectedResult values)

Defined in Futhark.Test

Methods

showsPrec :: Int -> ExpectedResult values -> ShowS

show :: ExpectedResult values -> String

showList :: [ExpectedResult values] -> ShowS

data Success

The result expected from a succesful execution.

Constructors

SuccessValues Values

These values are expected.

SuccessGenerateValues

Compute expected values from executing a known-good reference implementation.

Instances

Show Success

Defined in Futhark.Test

Methods

showsPrec :: Int -> Success -> ShowS

show :: Success -> String

showList :: [Success] -> ShowS

data Values

Several Values - either literally, or by reference to a file, or to be generated on demand.

Constructors

Values [Value]

InFile FilePath

GenValues [GenValue]

Instances

Show Values

Defined in Futhark.Test

Methods

showsPrec :: Int -> Values -> ShowS

show :: Values -> String

showList :: [Values] -> ShowS

data GenValue

Constructors

GenValue [Int] PrimType

Generate a value of the given rank and primitive type. Scalars are considered 0-ary arrays.

GenInt Int32

A fixed non-randomised integer.

Instances

Show GenValue

Defined in Futhark.Test

Methods

showsPrec :: Int -> GenValue -> ShowS

show :: GenValue -> String

showList :: [GenValue] -> ShowS

data Value

An efficiently represented Futhark value. Use pretty to get a human-readable representation, and the instances of Get and Put to obtain binary representations

Instances

Show Value

Defined in Futhark.Test.Values

Methods

showsPrec :: Int -> Value -> ShowS

show :: Value -> String

showList :: [Value] -> ShowS

Binary Value

Defined in Futhark.Test.Values

Methods

put :: Value -> Put

get :: Get Value

putList :: [Value] -> Put

Pretty Value

Defined in Futhark.Test.Values

Methods

ppr :: Value -> Doc

pprPrec :: Int -> Value -> Doc

pprList :: [Value] -> Doc