Hackage rough To Do list

A basic interface to the Hackage Database is now online. To get the source:

Below is a list of items that still need to be done.

Haddock documentation

Currently:

  • As part of a Google Summer of Code project, Sascha Böhme is implementing a tool to build HackageDB packages using Cabal in a chroot jail.
    • Building using Cabal is more likely to work than some shortcut, and performs a useful test on the package.
    • For this to work, prerequisite packages must be built, haddocked and installed first.
    • The chroot jail is needed because Cabal builds can invoke arbitrary code.
    • The alternative of only building packages that use the simple build system would rule out too many important packages.
  • Trials are being run, generating documentation and build logs for the library packages in HackageDB. Soon it will run automatically.
  • Haddock documentation for a library package is placed in the directory archive/pkg/version/doc/html. If this directory exists, the package script renders exposed module names as links to the documentation for the corresponding modules.
  • The haddock documentation links to types, modules etc. in the packages it depends on.
  • Links across packages point to the latest version of the target package, unless the dependency has a constraint that forbids this. This way we don't have to worry about re-haddocking dependent packages whenever a package at the bottom of the tree is updated.
  • The --html-location option of the development version of Cabal is used to make haddock generate links pointing at archive/pkg/latest/doc/html, which will be a symbolic link to an html directory.
  • The build log for a package is placed in the file archive/pkg/version/log. If this file exists, the package script displays a link to it.

Should building happen during upload, between upload and arriving in the archive, or later? Probably the latter.

Problem: building may require foreign libraries, but these can't be specified in the .cabal file.

Other things we could do:

  • Generate a Hoogle database for searching.
  • Syntax highlighted source code (using hscolour or similar) with links from the haddock documentation (using the Haddock --source options)?
  • Test with different compiler & versions, report which ones work
  • Run test suites

Package properties

The .cabal file contains a number of internal properties of the package. There will be a need to add other attributes of uploaded packages, without modifying the .cabal file. This could be achieved by storing a mapping from attributes to values (both strings) outside of the Cabal package. (Suggested format: a plain text file with a mail header format like the .cabal file.) These attributes could be set by programs on the server, set by users via a web interface, shown by the package display and used for searching.

Example attributes:

  • who uploaded the package
  • when it was uploaded
  • which versions of compilers it has been built or tested with
  • packages that depend on this one (?)
  • user-supplied descriptions of the package
  • feedback from users' cabal-install runs (?)
  • overriding fields from the .cabal file, e.g. category

These attributes could also be used for a more sophisticated form of classification, as in debtags, but multiple categories should be enough for now. (Also, this use arguably belongs in the .cabal file.)

Queries

We already have Hoogle for searching for names in modules. As the package collection grows, we'll also need a package-level search facility. We might like to search by

  • fields of the .cabal file, e.g. package name, words in synopsis or description, module name, author / maintainer.
  • attributes in the tags file, e.g. uploader, ranges of upload date, compilers it builds with, user-supplied descriptions.
  • combinations of these

Darcs integration

  • Need Cabal field for darcs repository, perhaps 
    repository: darcs:http://darcs.haskell.org/ghc
  • should we also have a link to a URL for browsing the repo, that would eg. point to the darcsweb?
  • should there also be a way to jump to the repo browser for the tag of this particular version, for viewing the change log?
  • Problem: some repositories contain multiple packages.

Package page as home page

  • The page for a package could include information from standard README, INSTALL, NEWS etc files. Those files could contain wikitext-like markup. (Or haddock markup, since we already have to parse descriptions? Or HTML fragments?) cf hask-home.

Uploads

  • auto building (see above)
  • Haddock (see above)
  • Generate change information between package versions: functions added/removed/changed, modules added/removed.
  • When a new version of a package is uploaded, generate a list of packages that will be affected and possibly broken by the upload, and notify the uploader, and test the builds.
  • RSS feed could distinguish new packages from upgrades

Gateways to other packaging systems

Each of these is probably best done by someone familiar with a particular packaging system.