Test.Sandwich.Options

Documentation

data Options

All the options controlling a test run.

defaultOptions :: Options

A reasonable default set of options.

Artifacts

optionsTestArtifactsDirectory :: Options -> TestArtifactsDirectory

Where to save test artifacts (logs, screenshots, failure reports, etc.).

data TestArtifactsDirectory

Control whether test artifacts are stored to a directory.

Constructors

TestArtifactsNone

Do not create a test artifacts directory.

TestArtifactsFixedDirectory

Use the test artifacts directory at the given path, creating it if necessary.

Fields

• testRootFixed :: FilePath

TestArtifactsGeneratedDirectory

Create a new test artifacts directory under '' test artifacts directory at the given path.

Fields

• runsRoot :: FilePath

  The root folder under which test run directories will be created.

• getTestRunDirectoryName :: IO FilePath

  Action to generate the new directory name.

Logging

optionsSavedLogLevel :: Options -> Maybe LogLevel

Minimum test log level to save (has no effect if optionsTestArtifactsDirectory is TestArtifactsNone).

optionsMemoryLogLevel :: Options -> Maybe LogLevel

Test log level to store in memory while tests are running. (These logs are presented in formatters, etc.).

optionsLogFormatter :: Options -> LogEntryFormatter

Formatter function for log entries.

type LogEntryFormatter = UTCTime -> Loc -> LogSource -> LogLevel -> LogStr -> ByteString

A callback for formatting a log entry to a ByteString.

Formatting

optionsFormatters :: Options -> [SomeFormatter]

Which formatters to use to output the results of the tests.

data SomeFormatter

An existential wrapper around Formatters

Constructors

forall f.(Formatter f, Show f, Typeable f) => SomeFormatter f

Instances

Show SomeFormatter

Defined in Test.Sandwich.Types.RunTree

Methods

showsPrec :: Int -> SomeFormatter -> ShowS

show :: SomeFormatter -> String

showList :: [SomeFormatter] -> ShowS

class Formatter f where

Methods

formatterName :: f -> String

Name of the formatter

runFormatter :: (MonadLoggerIO m, MonadUnliftIO m, MonadCatch m) => f -> [RunNode BaseContext] -> Maybe (CommandLineOptions ()) -> BaseContext -> m ()

The main function, executed while the test tree is running

finalizeFormatter :: (MonadIO m, MonadLogger m, MonadCatch m) => f -> [RunNode BaseContext] -> BaseContext -> m ()

Called after the test tree is completed, can be used to print final results

Instances

Formatter FailureReportFormatter

Defined in Test.Sandwich.Formatters.FailureReport

Methods

formatterName :: FailureReportFormatter -> String

runFormatter :: (MonadLoggerIO m, MonadUnliftIO m, MonadCatch m) => FailureReportFormatter -> [RunNode BaseContext] -> Maybe (CommandLineOptions ()) -> BaseContext -> m ()

finalizeFormatter :: (MonadIO m, MonadLogger m, MonadCatch m) => FailureReportFormatter -> [RunNode BaseContext] -> BaseContext -> m ()

Filtering

optionsFilterTree :: Options -> Maybe TreeFilter

Filter to apply to the text tree before running.

newtype TreeFilter

Constructors

TreeFilter String

Timing

optionsTestTimerType :: Options -> TestTimerType

Whether to enable the test timer. When the test timer is present, timing information will be emitted to the project root (if present).

Dry run

optionsDryRun :: Options -> Bool

Whether to skip actually launching the tests. This is useful if you want to see the set of the tests that would be run, or start them manually in the terminal UI.

Misc

optionsProjectRoot :: Options -> Maybe FilePath

An optional absolute path to the root of the project being tested (i.e. the folder where the cabal file is found). This is useful to provide when the current working directory does not match the project root, for example in multi-project Stack setups. We use this hint to connect CallStack paths (which are relative to the project root) to their actual path on disk.