cabal-doctest: A Setup.hs helper for running doctests

[ bsd3, distribution, library ] [ Propose Tags ]

As of now (end of 2021), there isn't cabal doctest command. Yet, to properly work, doctest needs plenty of configuration. This library provides the common bits for writing a custom Setup.hs.


[Skip to Readme]

Modules

[Index] [Quick Jump]

Downloads

Maintainer's Corner

Package maintainers

For package maintainers and hackage trustees

Candidates

  • No Candidates
Versions [RSS] 1, 1.0.1, 1.0.2, 1.0.3, 1.0.4, 1.0.5, 1.0.6, 1.0.7, 1.0.8, 1.0.9, 1.0.10
Change log changelog.md
Dependencies base (>=4.9 && <5), Cabal (>=1.10 && <3.14), directory (>=1.3 && <2), filepath (>=1.4 && <2) [details]
License BSD-3-Clause
Copyright (c) 2017-2020 Oleg Grenrus, 2020- package maintainers
Author Oleg Grenrus <oleg.grenrus@iki.fi>
Maintainer Max Ulidtko <ulidtko@gmail.com>
Category Distribution
Home page https://github.com/ulidtko/cabal-doctest
Source repo head: git clone https://github.com/ulidtko/cabal-doctest
Uploaded by ulidtko at 2024-06-26T21:59:54Z
Distributions Arch:1.0.9, Debian:1.0.8, Fedora:1.0.9, LTSHaskell:1.0.10, NixOS:1.0.9, Stackage:1.0.10, openSUSE:1.0.9
Reverse Dependencies 1 direct, 0 indirect [details]
Downloads 99158 total (334 in the last 30 days)
Rating 2.0 (votes: 1) [estimated by Bayesian average]
Your Rating
  • λ
  • λ
  • λ
Status Docs uploaded by user
Build status unknown [no reports yet]

Readme for cabal-doctest-1.0.10

[back to package description]

cabal-doctest

Hackage Haskell-CI

A Setup.hs helper for running doctests.

Simple example

Follow simple example for the common case of a single-library .cabal package with doctests.

To recap the example's code:

  1. specify build-type: Custom in your .cabal file;

  2. declare dependencies of Setup.hs:

    custom-setup
     setup-depends:
       base >= 4 && <5,
       cabal-doctest >= 1 && <1.1
    

    See Notes below for a caveat with cabal-install < 2.4.

  3. Populate Setup.hs like so:

    module Main where
    
    import Distribution.Extra.Doctest (defaultMainWithDoctests)
    
    main :: IO ()
    main = defaultMainWithDoctests "doctests"
    

    Assuming your test-suite is called doctests, this Setup will generate a Build_doctests module during package build. If your test-suite goes by name foo, defaultMainWithDoctests "foo" creates a Build_foo module.

  4. Use the generated module in a testsuite, simply like so:

    module Main where
    
    import Build_doctests (flags, pkgs, module_sources)
    import Data.Foldable (traverse_)
    import System.Environment (unsetEnv)
    import Test.DocTest (doctest)
    
    main :: IO ()
    main = do
        traverse_ putStrLn args -- optionally print arguments
        unsetEnv "GHC_ENVIRONMENT" -- see 'Notes'; you may not need this
        doctest args
      where
        args = flags ++ pkgs ++ module_sources
    

Ultimately, cabal test or stack test should run the doctests of your package.

Example with multiple cabal components

cabal-doctest also supports more exotic use cases where a .cabal file contains more components with doctests than just the main library, including:

  • doctests in executables,
  • doctests in internal libraries (if using Cabal-2.0 or later).

Unlike the simple example shown above, these examples involve named components. You don't need to change the Setup.hs script to support this use case. However, in this scenario Build_doctests will generate extra copies of the flags, pkgs, and module_sources values for each additional named component.

The simplest approach is to use x-doctest-components field in .cabal:

x-doctest-components: lib lib:internal exe:example

In that case, the test driver is generally:

module Main where

import Build_doctests (Component (..), components)
import Data.Foldable (for_)
import System.Environment (unsetEnv)
import Test.DocTest (doctest)

main :: IO ()
main = for_ components $ \(Component name flags pkgs sources) -> do
    print name
    putStrLn "----------------------------------------"
    let args = flags ++ pkgs ++ sources
    for_ args putStrLn
    unsetEnv "GHC_ENVIRONMENT"
    doctest args

There is also a more explicit approach: if you have an executable named foo, then Build_doctest will contain flags_exe_foo, pkgs_exe_foo, and module_sources_exe_foo. If the name has hyphens in it (e.g., my-exe), cabal-doctest will convert them to underscores (e.g., you'd get flags_my_exe, pkgs_my_exe, module_sources_my_exe). Internal library bar values will have a _lib_bar suffix.

An example testsuite driver for this use case might look like this:

module Main where

import Build_doctests
       (flags,            pkgs,            module_sources,
        flags_exe_my_exe, pkgs_exe_my_exe, module_sources_exe_my_exe)
import Data.Foldable (traverse_)
import System.Environment (unsetEnv)
import Test.DocTest

main :: IO ()
main = do
    unsetEnv "GHC_ENVRIONMENT"
    -- doctests for library
    traverse_ putStrLn libArgs
    doctest libArgs

    -- doctests for executable
    traverse_ putStrLn exeArgs
    doctest exeArgs
  where
    libArgs = flags            ++ pkgs            ++ module_sources
    exeArgs = flags_exe_my_exe ++ pkgs_exe_my_exe ++ module_sources_exe_my_exe

See the multiple-components-example.

Additional configuration

The cabal-doctest based Setup.hs supports a few extensions fields in pkg.cabal files to customise the doctest runner behaviour, without customising the default doctest.hs.

test-suite doctests:
  if impl(ghc >= 8.0)
    x-doctest-options: -fdiagnostics-color=never
  x-doctest-source-dirs: test
  x-doctest-modules: Servant.Utils.LinksSpec
  • x-doctest-options Additional arguments passed into doctest command.
  • x-doctest-modules Additional modules to doctest. May be useful if you have doctest in test or executables (i.e not default library component).
  • x-doctest-src-dirs Additional source directories to look for the modules.

Notes

  • If support for cabal-install < 2.4 is required, you'll have to add Cabal to setup-depends; see issue haskell/cabal#4288.

  • Some versions of Cabal (for instance, 2.0) can choose to build a package's doctest test suite before the library. However, in order for cabal-doctest to work correctly, the library must be built first, as doctest relies on the presence of generated files that are only created when the library is built. See #19.

    A hacky workaround for this problem is to depend on the library itself in a doctests test suite. See simple-example.cabal for a demonstration. (This assumes that the test suite has the ability to read build artifacts from the library, a separate build component. In practice, this assumption holds, which is why this library works at all.)

  • custom-setup section is supported starting from cabal-install-1.24. For older cabal-install's you have to install custom setup dependencies manually.

  • stack respects custom-setup starting from version 1.3.3. Before that you have to use explicit-setup-deps setting in your stack.yaml; stack#2094.

  • With base < 4.7 (GHC < 7.8, pre-2014), System.Environment.unsetEnv function will need to be imported from base-compat library. It is already in transitive dependencies of doctest. Simply declare the dependency upon base-compat, and then import System.Environment.Compat (unsetEnv) if you need that old GHC.

  • You can use x-doctest-options field in test-suite doctests to pass additional flags to the doctest.

  • For build-type: Configure packages, you can use defaultMainAutoconfWithDoctests function to make custom Setup.hs script.

  • If you use the default . in hs-source-dirs, then running doctests might fail with weird errors (ambiguous module errors). Workaround is to move sources under src/ or some non-top-level directory.

  • The extensions: field isn't supported. Upgrade your .cabal file to use at least cabal-version: >= 1.10 and use default-extensions or other-extensions.

  • If you use QuickCheck properties (prop>) in your doctests, the test-suite doctest should depend on QuickCheck and template-haskell. This is a little HACK: These dependencies aren't needed to build the doctests test-suite executable. However, as we let Cabal resolve dependencies, we can pass the resolved (and installed!) package identifiers to to the doctest command. This way, QuickCheck and template-haskell are available to doctest, otherwise you'll get errors like:

      Variable not in scope:
        mkName
          :: [Char]
             -> template-haskell-2.11.1.0:Language.Haskell.TH.Syntax.Name
    

    or

      Variable not in scope:
        polyQuickCheck
          :: Language.Haskell.TH.Syntax.Name -> Language.Haskell.TH.Lib.ExpQ
    
  • From version 2, Stack sets the GHC_ENVIRONMENT variable, and GHC (as invoked by doctest) will pick that up. This is undesirable: cabal-doctest passes all the necessary information on the command line already, and can lead to ambiguous module errors as GHC will load the environment in addition to what cabal-doctest instructs it to.

    Hence, cabal-doctest tells GHC to ignore package environments altogether on the command line. However, this is only possible since GHC 8.2. If you are using cabal-doctest with Stack 2 and GHC 8.0 or earlier and seeing ambiguous module errors or other mysterious failures, try manually unsetting GHC_ENVIRONMENT before invoking doctest.

  • If you are on Nix. doctest will not pick up your version of GHC if you don't point it towards it, and therefore will result in "cannot satisfy -package-id" errors. You will need to set NIX_GHC and NIX_GHC_LIBDIR within your environment in order for doctest to pick up your GHC. Put the following in shell.nix and run nix-shell.

    # shell.nix
    { pkgs ? import <nixpkgs> {} }:
    let
      myHaskell = (pkgs.haskellPackages.ghcWithHoogle (p: with p; [
        # Put your dependencies here
        containers
        hslogger
      ]));
    in
    pkgs.mkShell {
      name = "myPackage";
    
      # These environment variables are important. Without these,
      # doctest doesn't pick up nix's version of ghc, and will fail
      # claiming it can't find your dependencies
      shellHook = ''
        export NIX_GHC=${myHaskell}/bin/ghc
        export NIX_GHC_LIBDIR=${myHaskell}/lib/ghc-8.10.7
      '';
      buildInputs = with pkgs; [
        myHaskell
      ];
    }
    

Copyright 2017 Oleg Grenrus.

With contributions from:

  • Ryan Scott
  • Andreas Abel
  • Max Ulidtko

Available under the BSD 3-clause license.