EL.Test.RunTests

Run tests. This is meant to be invoked via a main module generated by EL.Test.GenerateRunTests.

Documentation

data Test

Constructors

Test

Fields

• testSymName :: Text

  Name of the test function.

• testRun :: IO ()

  Run the test.

• testFilename :: FilePath

  Test module filename.

• testLine :: Int

  Line of the test function declaration.

• testModuleMeta_ :: Maybe ModuleMeta

  Module-level metadata, declared as meta in the test module toplevel.

testModuleMeta :: Test -> ModuleMeta

testName :: Test -> Text

metaPrefix :: Text

data Flag

Constructors

CheckOutput

ClearDirs

Jobs !Jobs

List

Output !FilePath

Subprocess

Instances

Eq Flag

Defined in EL.Test.RunTests

Methods

(==) :: Flag -> Flag -> Bool

(/=) :: Flag -> Flag -> Bool

Show Flag

Defined in EL.Test.RunTests

Methods

showsPrec :: Int -> Flag -> ShowS

show :: Flag -> String

showList :: [Flag] -> ShowS

data Jobs

Constructors

Auto

NJobs !Int

Instances

Eq Jobs

Defined in EL.Test.RunTests

Methods

(==) :: Jobs -> Jobs -> Bool

(/=) :: Jobs -> Jobs -> Bool

Show Jobs

Defined in EL.Test.RunTests

Methods

showsPrec :: Int -> Jobs -> ShowS

show :: Jobs -> String

showList :: [Jobs] -> ShowS

options :: [OptDescr Flag]

run :: [String] -> [Test] -> IO ()

Called by the generated main function.

quitWithUsage :: [String] -> [String] -> IO a

runTests :: [Test] -> [Flag] -> [String] -> IO Bool

getJobs :: Jobs -> IO Int

runOutput :: FilePath -> Int -> [Test] -> Bool -> IO Bool

runInSubprocess :: Test -> IO ()

Isolate the test by running it in a subprocess. I'm not sure if this is necessary, but I believe at the time GUI-using tests would crash each other without it. Presumably they left some GUI state around that process exit will clean up.

parallel jobs

runParallel :: [FilePath] -> [Test] -> IO ()

Run tests in parallel, redirecting stdout and stderr to each output.

jobThread :: FilePath -> Queue (Text, [Test]) -> IO ()

Pull tests off the queue and feed them to a single subprocess.

subprocess :: [Test] -> IO ()

testsCompleteLine :: Text

Signal to the caller that the current batch of tests are done.

run tests

matchingTests :: [String] -> [Test] -> [Test]

Match all tests whose names match any regex, or if a test is an exact match, just that test.

runTest :: Test -> IO ()

isolate :: IO a -> IO a

Try to save and restore any process level state in case the test messes with it. Currently this just restores CWD, but probably there is more than that. For actual isolation probably a subprocess is necessary.

catch :: Text -> IO a -> IO ()

queue

newtype Queue a

This is a simple channel which is written to once, and read from until empty.

Constructors

Queue (MVar [a])

newQueue :: [a] -> IO (Queue a)

takeQueue :: Queue a -> IO (Maybe a)

whileJust :: Monad m => m (Maybe a) -> (a -> m ()) -> m ()

check output

clearDirectory :: FilePath -> IO ()

Empty the directory, but don't remove it entirely, in case it's /tmp or something.

checkOutputs :: [FilePath] -> IO Bool

readFileEmpty :: FilePath -> IO Text

extractStats

Arguments

:: Text

-> ([Text], Int, Int, Int)

(failureContext, failures, checks, tests)

extractFailures

Arguments

:: [Text]

-> [Maybe Text]

Just context for a failure, Nothing for a success.

Collect lines before and after each failure for context.

I collect before because that's where debugging info about that test is likely to show up, and I collect after because the failure output may have multiple lines.

It can be confusing that I can get the failure lines of the previous test as context for the current one. To fix that I'd have to explicitly mark all lines of the failure, or put some ending marker afterwards. It's not hard but maybe not worth it.