id,summary,reporter,owner,description,type,status,priority,milestone,component,version,resolution,keywords,cc,os,architecture,failure,difficulty,testcase,blockedby,blocking,related
3896,Make the GHC build system accommodate extra packages,simonpj,,"Simon and I discussed the extra-packages story in GHC's build system.  Here's a summary:

 * Simon added these 'extra-packages' stuff to help automate the
   process of downloading and building packages
   - that are not boot packages
   - but which we want to test regularly

 * DPH is another such package.  It's not really a boot package, in the
  sense that (a) it's not necessary to build GHC and (b) it can only
  be built with a stage-2 compiler.  The main reason it's treated as a
  boot package today is so that new patches are validated against it;
  it is a particularly good stress test.  But it seems a bit overkill
  for it to masquerade as a boot package for that reason alone.

 * The current extra-packages story is a bit of a mess:
   - The extra packages are listed in $(TOP)/packages
   - But they are, quite separately listed in a make variable
     EXTRA_PACKAGES, and then stuffed into PACKAGES by 'addPackage' in
     $(TOP)/ghc.mk.  
   - A series of addPackage calls in ghc.mk duplicates the list of
     boot packages in $(TOP)/packages.
   - Moreover, the list in EXTRA_PACKAGES must be in dependency order.

We propose the following plan

 * Make the list in $(TOP)/packages the authoritative list.

 * Make 'sh boot' construct the list $(PACKAGES) from $(TOP)/packages
  and stuff it somewhere appropriate.  ['sh boot' has to iterate over
  the packages anyhow, to boot them, but doesn't currently consult
  $(TOP)/packages to do so; it looks a the file system, which is less
  good.]

 * Make DPH into an extra package

 * And/or add a tag 'validated' to mean ""this is an extra package, 
  not a boot package, but it should be validated before pushing patches"".
  That doesn't necessarily apply to all extra packages (eg stm).

 * The list in $(TOP)/packages is really a list of '''repositories''' not a list
  of '''packages'''.  For example, utils/hcs2hs isn't a package.  We probably
  want a tag (""util""?) to distinguish non-packages from packages.

 * DPH is unusual because it is a '''single repository''' that contains '''several packages'''.
  We can tell this because it has a file ghc-packages in its root directory,
  that lists the packages it contains.  'sh boot' must understand this when
  constructing $(PACKAGES); indeed it already does understand this.

 * Packages must be built in dependency order, and the build system has
  no way of figuring out the right order.  But it would be easy to
  stipulate that $(TOP)/packages (and the ghc-packages file, if any,
  in the repo) should list the packages in dependency order.


We don't need to execute this plan right away, but would be a very
nice clean-up if anyone felt up to doing something about it.

",task,closed,normal,7.0.1,Build System,,fixed,,,Unknown/Multiple,Unknown/Multiple,None/Unknown,,,,,