* subprocess communication is getting stuck . But why? I think this should be the same code as karya's version, which doesn't do that. . There's definitely something fishy going on in both cases though, since karya's version sometimes fails on travis. . dist/build/test/test --clear-dirs --output=dist/test-output . How to run GenerateRunTests? - use physical CPUs instead of getNumCapabilities . The cpuinfo package is meant to do this but it's linux only: https://github.com/TravisWhitaker/cpuinfo/issues/1 * There should be a standard set of "run from cabal" flags: --clear-dir --output=dist/test-output --check-output . / Makefile to update when out of date, cabal Setup hooks to run make. / Like Makefile, but a shell script that does the same thing. / Use Setup hooks directly. Does cabal know how to only rebuild when necessary? * HSpec uses a -pgmF program to generate the module. But I think it runs every single time you build tests. Of course, maybe that's actually pretty fast. . pgmF args: OrigSourceName.hs input.hs output.hs * remove the ioHuman and interactive stuff / argv0 is wrong, so parallelization probably doesn't work . Use Environment.getExecutablePath * fix tmpBaseDir . Surely cabal has some way to pass in a tmp dir? . Haha, it's cabal of course not. Use CPP then. . Well, it would have to come from the testee config, which means putting it in Testing.Config. Too much bother, just hardcode to dist/build/tmp-test Interpret the output . I need the equivalent of test/run_tests. . Can I embed that logic in RunTest itself? The problem is I want to write to stdout, and then check that stdout. It's hard to do that when cabal is redirecting stdout. . Does cabal have any more flexible way to run tests? Looks like no, just 'detailed', which is even less flexible, since it requires the tests to be some cabal data structure. . I would like a runner that runs the test binary, then runs something else with access to the log. . Can I redirect stdout to a file temporarily? I could do withStdout (open "output") $ runTests ... check "output" . I can have tests write to a file via 'success' and 'failure'. The problem is that then debugging isn't interleaved. But I could write to both the output file and stdout. Or write everything to sdout, only errors to stderr. . Or use the parallelization feature, run as a subprocess. . How this works is you call with --outputs. Then RunTests can have a flag to check the outputs afterwards.