allow per-package configuration options in config file

I am working on this, wish me luck :)

Since this is not a trivial refactoring and I am not a usual committer, here is a plan write-up.

Problem

We have ConfigFlags?, a data structure in typical database fashion, a dictionary of plain values composed of several fields. There are two main goals

a) Be able to serialize it (done now via ParseUtils?.FieldDescrs?)

b) Be able to manipulate it via command line flags (done via Command.Option)

For a), we want a format that is human readable and modifiable.

For b), we want to be able to specify the flags, with an expressivity comparable to the GetOpt? language.

Overall, we want to have a coherence and be able to reuse code. We may want to see these as two facets of the same problem and be able to specify them in one single place.

(and in Haskell 98 of course)

Solution 1

Specify b) via Distribution.Command.Option, and then convert those options to ParseUtils?.FieldDescrs? automatically, which can be serialized for b). The problem is that NoArg/OptArg? options cannot really be converted to FieldDescrs?. We are missing a proper name for the field in the serialized file, and since there are usually several Options of this kind associated to one field, we want to see them as defining the possible values of the field. But we cannot collect them, as the option description is missing a declarative reference to which record they modify in the data structure.

option :: [Char] -> [String] -> String -> c -> d
       -> (c -> d -> ArgDescr a) -> Option a
option ss ls d get set arg = Option ss ls d (arg get set)

There are getters and setters, but those are opaque functions. Hence, we can only automatically infer FieldDescrs? for ReqArg? options. The others must be inputed manually.

This seems error prone and tricky to maintain to me, although it has the advantage of being the simplest refactoring among all the options.

Solution 2

If we reify the record, then options can be given a more declarative type.

option :: Record data field -> [Char] -> String -> String -> ArgDescr field -> Option field

data Record data field = 
  { get  :: data -> field
  , set  :: field -> data -> data

This is good enough to enable us to collect all the NoArg? options related to one field, and condense a single FieldDescr? out of it.

Solution 3

But why stop there? We can put more metadata in the Record descriptor.

 interface :: OptInterface,
 desc      :: String,
 shape     :: Shape,
 ppr       :: field -> Doc
}

data OptInterface {
      		      shortFlags :: [String],	
 		      longFlags  :: [String]  }

data Shape field = StringValued (String -> ReadS field)
                 | Choice       (Map OptInterface field)

So now it is possible to extract an Option directly from this structure.

mkOption :: Record data field -> [Option data]

The definition of mkOption should be trivial. If the Record is StringValued? then we get a single Option, otherwise we get an Option for every choice.

And the same goes for FieldDescrs?.

mkFieldDescr :: Record data field -> FieldDescr data

For the row name in the file we just take the first longFlag, and if it has a Choice shape then we build the trivial parser for it. Otherwise we just use the parser provided in StringValued?.

Finally, we could just extend FieldDescr? to accomodate the role of Record above. But this poses the minor issue that FieldDescr? 'instances' are provided in CabalInstall, whereas they would be required by Cabal already in Distribution.Setup.hs So they will need to be moved there.

Thanks for keeping us posted, it sounds like a very sensible plan.

It would be useful to be able to specify flags in the config file for all the commands and not only for configure e.g. install's --root-cmd.

This poses the question if it should be possible to specify different defaults for the same flag for two different commands.

The config file now holds the options for the global command (things like repos, cache dir etc), the install/upgrade command and the configure and upload commands.

Thu Aug 21 22:05:55 BST 2008  Duncan Coutts <duncan@haskell.org>
  * Rearrange config settings and global command line settings
  The intention here is to have the config file content and parser
  be derivied from the command line types and flags. Now instead
  of having a saved config type that contains additional
  information to that kept in command line parameters we now put
  all the information in the command line settings type and make
  the saved config just the aggregation of the settings for
  various key commands like configure, install, upload. There's
  also an extended GlobalFlags type with the things like repos,
  cache dir etc. When we generate the initial config file we put
  in commented out versions of all (non-deprecated) valid fields,
  along with their default values. The aim is to make the config
  file self-documenting.

As before it also has two sets of saved install dirs.

Fri Aug 22 02:57:02 BST 2008  Duncan Coutts <duncan@haskell.org>
  * Add back separate user and global sets of install dirs
  The user install setting selects between the two sets of dirs.
  At the moment it's just the "user-" and "global-" prefixed field
  names to support existing config files.

Fri Aug 22 13:00:59 BST 2008  Duncan Coutts <duncan@haskell.org>
  * Add sections for user/global install-dirs to the config file
  So it looks like:
  install-dirs user
    prefix: /home/username/.cabal
    ...
  Rather than using user-prefix, global-prefix, etc etc for each
  field. The old field names are still recognised but not added
  into the initial config file.

The default config file now looks like:

-- version:
-- numeric-version:
-- config-file:
remote-repo: hackage.haskell.org:http://hackage.haskell.org/packages/archive
remote-repo-cache: /home/username/.cabal/packages
-- local-repo:
-- documentation: False
-- dry-run:
-- reinstall:
-- root-cmd:
-- cabal-lib-version:
-- log-builds:
-- build-reports:
-- only:
-- verbose: 1
-- distpref: dist
-- compiler: ghc
-- with-compiler:
-- with-hc-pkg:
-- scratchdir:
-- program-prefix: 
-- program-suffix: 
-- library-vanilla: True
-- library-profiling: False
-- shared: False
-- executable-profiling: False
-- optimization: True
-- library-for-ghci: True
-- split-objs: False
-- executable-stripping: True
-- configure-option:
user-install: True
-- package-db:
-- flags:
-- extra-include-dirs:
-- extra-lib-dirs:
-- constraint:
-- verbose: 1
-- check:
-- username:
-- password:

install-dirs user
  prefix: /home/username/.cabal
  -- bindir: $prefix/bin
  -- libdir: $prefix/lib
  -- libsubdir: $pkgid/$compiler
  -- libexecdir: $prefix/libexec
  -- datadir: $prefix/share
  -- datasubdir: $pkgid
  -- docdir: $datadir/doc/$pkgid
  -- htmldir: $docdir/html
  -- haddockdir: $htmldir

install-dirs global
  -- prefix: /usr/local
  -- bindir: $prefix/bin
  -- libdir: $prefix/lib
  -- libsubdir: $pkgid/$compiler
  -- libexecdir: $prefix/libexec
  -- datadir: $prefix/share
  -- datasubdir: $pkgid
  -- docdir: $datadir/doc/$pkgid
  -- htmldir: $docdir/html
  -- haddockdir: $htmldir

There are some fields there that come from command line flags that do not really apply to the config file and so should be filtered out.

This syntax will allow us to to per-package config options, like:

package foo
  extra-lib-dirs: /some/thing

But actually adding that feature is not the top priority. Having a more self-documenting config file that contains all the options that can be specified on the command line is top priority.

The new config seems to be working. The fields that do not apply in the config file are now excluded.

The remaining issue can be punted to a later milestone.