doctest-parallel: Test interactive Haskell examples

This is a package candidate release! Here you can preview how this package release will appear once published to the main package index (which can be accomplished via the 'maintain' link below). Please note that once a package has been published to the main package index it cannot be undone! Please consult the package uploading documentation for more information.

[maintain] [Publish]

The doctest program checks examples in source code comments. It is modeled after doctest for Python (https://docs.python.org/3/library/doctest.html).

Documentation is at https://github.com/martijnbastiaan/doctest-parallel#readme.


[Skip to Readme]

Properties

Versions 0.1, 0.2, 0.2, 0.2.1, 0.2.2, 0.2.3, 0.2.4, 0.2.5
Change log CHANGES.markdown
Dependencies base (>=4.10 && <5), base-compat (>=0.7.0), Cabal, code-page (>=0.1), containers, deepseq, directory, doctest-parallel, exceptions, filepath, ghc (>=8.4 && <9.3), ghc-paths (>=0.1.0.9), Glob, pretty, process, syb (>=0.3), template-haskell, transformers, unordered-containers [details]
License MIT
Copyright (c) 2009-2018 Simon Hengel, 2021 Martijn Bastiaan
Author Martijn Bastiaan <martijn@hmbastiaan.nl>
Maintainer Martijn Bastiaan <martijn@hmbastiaan.nl>
Category Testing
Home page https://github.com/martijnbastiaan/doctest-parallel#readme
Bug tracker https://github.com/martijnbastiaan/doctest-parallel/issues
Source repo head: git clone https://github.com/martijnbastiaan/doctest-parallel
Uploaded by martijnbastiaan at 2021-12-12T20:10:09Z

Modules

[Index]

Downloads

Maintainer's Corner

For package maintainers and hackage trustees


Readme for doctest-parallel-0.2

[back to package description]

Doctest parallel: Test interactive Haskell examples

doctest-parallel is a library that checks examples in Haddock comments. It is similar to the popular Python module with the same name.

Installation

doctest-parallel is available from Hackage. It cannot be used as a standalone binary, rather, it expects to be integrated in a Cabal/Stack project. See examples/ for more information on how to integrate doctest-parallel into your project.

Usage

Below is a small Haskell module. The module contains a Haddock comment with some examples of interaction. The examples demonstrate how the module is supposed to be used.

module Fib where

-- | Compute Fibonacci numbers
--
-- Examples:
--
-- >>> fib 10
-- 55
--
-- >>> fib 5
-- 5
fib :: Int -> Int
fib 0 = 0
fib 1 = 1
fib n = fib (n - 1) + fib (n - 2)

A comment line starting with >>> denotes an expression. All comment lines following an expression denote the result of that expression. Result is defined by what a REPL (e.g. ghci) prints to stdout and stderr when evaluating that expression.

doctest-parallel will fail on comments that haddock also doesn't like. Sometimes (e.g., #251), this means that doctest-parallel will fail on input that GHC accepts.

Example groups

Examples from a single Haddock comment are grouped together and share the same scope. E.g. the following works:

-- |
-- >>> let x = 23
-- >>> x + 42
-- 65

If an example fails, subsequent examples from the same group are skipped. E.g. for

-- |
-- >>> let x = 23
-- >>> let n = x + y
-- >>> print n

print n is not tried, because let n = x + y fails (y is not in scope!).

Setup code

You can put setup code in a named chunk with the name $setup. The setup code is run before each example group. If the setup code produces any errors/failures, all tests from that module are skipped.

Here is an example:

module Foo where

import Bar.Baz

-- $setup
-- >>> let x = 23 :: Int

-- |
-- >>> foo + x
-- 65
foo :: Int
foo = 42

Note that you should not place setup code in between the module header (module ... where) and import declarations. GHC will not be able to parse it (issue #167). It is best to place setup code right after import declarations, but due to its declarative nature you can place it anywhere in between top level declarations as well.

Multi-line input

GHCi supports commands which span multiple lines, and the same syntax works for doctest:

-- |
-- >>> :{
--  let
--    x = 1
--    y = 2
--  in x + y + multiline
-- :}
-- 6
multiline = 3

Note that >>> can be left off for the lines following the first: this is so that haddock does not strip leading whitespace. The expected output has whitespace stripped relative to the :}.

Some peculiarities on the ghci side mean that whitespace at the very start is lost. This breaks the example broken, since the x and y aren't aligned from ghci's perspective. A workaround is to avoid leading space, or add a newline such that the indentation does not matter:

{- | >>> :{
let x = 1
    y = 2
  in x + y + works
:}
6
-}
works = 3

{- | >>> :{
 let x = 1
     y = 2
  in x + y + broken
:}
3
-}
broken = 3

Multi-line output

If there are no blank lines in the output, multiple lines are handled automatically.

-- | >>> putStr "Hello\nWorld!"
-- Hello
-- World!

If however the output contains blank lines, they must be noted explicitly with <BLANKLINE>. For example,

import Data.List ( intercalate )

-- | Double-space a paragraph.
--
--   Examples:
--
--   >>> let s1 = "\"Every one of whom?\""
--   >>> let s2 = "\"Every one of whom do you think?\""
--   >>> let s3 = "\"I haven't any idea.\""
--   >>> let paragraph = unlines [s1,s2,s3]
--   >>> putStrLn $ doubleSpace paragraph
--   "Every one of whom?"
--   <BLANKLINE>
--   "Every one of whom do you think?"
--   <BLANKLINE>
--   "I haven't any idea."
--
doubleSpace :: String -> String
doubleSpace = (intercalate "\n\n") . lines

Matching arbitrary output

Any lines containing only three dots (...) will match one or more lines with arbitrary content. For instance,

-- |
-- >>> putStrLn "foo\nbar\nbaz"
-- foo
-- ...
-- baz

If a line contains three dots and additional content, the three dots will match anything within that line:

-- |
-- >>> putStrLn "foo bar baz"
-- foo ... baz

QuickCheck properties

Haddock (since version 2.13.0) has markup support for properties. Doctest can verify properties with QuickCheck. A simple property looks like this:

-- |
-- prop> \xs -> sort xs == (sort . sort) (xs :: [Int])

The lambda abstraction is optional and can be omitted:

-- |
-- prop> sort xs == (sort . sort) (xs :: [Int])

A complete example that uses setup code is below:

module Fib where

-- $setup
-- >>> import Control.Applicative
-- >>> import Test.QuickCheck
-- >>> newtype Small = Small Int deriving Show
-- >>> instance Arbitrary Small where arbitrary = Small . (`mod` 10) <$> arbitrary

-- | Compute Fibonacci numbers
--
-- The following property holds:
--
-- prop> \(Small n) -> fib n == fib (n + 2) - fib (n + 1)
fib :: Int -> Int
fib 0 = 0
fib 1 = 1
fib n = fib (n - 1) + fib (n - 2)

If you see an error like the following, ensure that QuickCheck is a dependency of your test-suite.

<interactive>:39:3:
    Not in scope: ‘polyQuickCheck’
    In the splice: $(polyQuickCheck (mkName "doctest_prop"))

<interactive>:39:3:
    GHC stage restriction:
      ‘polyQuickCheck’ is used in a top-level splice or annotation,
      and must be imported, not defined locally
    In the expression: polyQuickCheck (mkName "doctest_prop")
    In the splice: $(polyQuickCheck (mkName "doctest_prop"))

Hiding examples from Haddock

You can put examples into named chunks, and not refer to them in the export list. That way they will not be part of the generated Haddock documentation, but Doctest will still find them.

-- $
-- >>> 1 + 1
-- 2

Using GHC extensions

You can enable GHC extensions using the following syntax:

-- >>> :set -XTupleSections

If you want to omit the information which language extensions are enabled from the Doctest examples you can use the method described in Hiding examples from Haddock, e.g.:

-- $
-- >>> :set -XTupleSections

Using GHC plugins

You can enable GHC plugins using the following syntax:

-- >>> :set -fplugin The.Plugin

Hiding Prelude

You hide the import of Prelude by using:

-- >>> :m -Prelude

Relation to doctest

This is a fork of sol/doctest that allows running tests in parallel and aims to provide a more robust project integration method. It is not backwards compatible and expects to be setup differently. At the time of writing it has a few advantages over the base project:

There are two downsides to using this project:

All in all, you can expect doctest-parallel to run about 1 or 2 orders of magnitude faster than doctest for large projects.

Relation to cabal-docspec

There is no direct relation between doctest-parallel and cabal-docspec. They are similar in some ways:

And different in others:

Development

To run the tests:

cabal run spectests
cabal run doctests

Future of this project

Of course, if you wish to add a feature that's not in this list, please feel free top open a pull request!

Contributors