Quick Start hacking guide

This page describes the new Hackage server.

Installing

To build the latest Hackage code:

darcs get http://code.haskell.org/hackage-server/
cd hackage-server
cabal install http://code.haskell.org/~duncan/HTTP-4000.1.3.20111008.tar.gz
cabal install --only-dependencies [--dry-run]
cabal configure -O0
cabal build

Yes, you do need the snapshot of the HTTP package, at least until version 4000.1.3 or later is released. It has some fixes that the mirror client requires.

Running

The following commands run the server in the current build tree:

dist/build/hackage-server/hackage-server init --static-dir=static/
dist/build/hackage-server/hackage-server run  --static-dir=static/

By default the server runs on port 8080 with the following settings:

URL:      http://localhost:8080/
username: admin
password: admin

To specify something different, see hackage-server init --help for details.

The server can be killed by using:

Control-C

This will save the current state and shutdown cleanly. Running again will resume with the same state.

Resetting

To reset everything, kill the server and delete the server state:

rm -rf state/

Note that the static/ and state/ directories differ: static is for static html and other files. The state directory holds information about the database, using acid-state.

Creating users & uploading packages

List of registered users:

http://localhost:8080/users/

Register new users:

http://localhost:8080/users/register

Currently there is no restriction on registering, but only an admin user can grant privileges to registered users e.g. by adding them to other groups. In particular there are groups:

• admins  http://localhost:8080/users/admins/ -- administrators can do things with user accounts like disabling, deleting, changing other groups etc.
• trustees  http://localhost:8080/packages/trustees/ -- trustees can do janitorial work on all packages
• mirrors  http://localhost:8080/packages/mirrorers/ -- for special mirroring clients that are trusted to upload packages
• per-package maintainer groups  http://localhost:8080/package/foo/maintainers -- users allowed to upload packages

Currently any registered user can upload new packages i.e. new names (we may want to review this), but only people in the package maintainer group can upload new versions of existing packages.

Mirroring

There is a program included in the hackage-server package called hackage-mirror. This should now work OK. It's intended to enable syncing all the packages from one server to another, e.g. getting all the packages from the old hackage and uploading them to a local instance of a hackage-server. The mirror client basically works, though it does not yet copy over the name of the original uploader or original upload date. In addition, not all packages can be mirrored yet because the new server rejects packages that use non-standard tar formats. That will need to be fixed in the tar package and in the server (we want to keep strict checking for ordinary uploads, just not for old existing packages).

To try it out:

1. Make sure you have built the mirror-client against a version of HTTP with the latest patches applied, e.g.  http://code.haskell.org/~duncan/HTTP-4000.1.3.20111008.tar.gz
2. Add a user to the mirrorers group by navigating to  http://localhost:8080/packages/mirrorers/
3. Run the client with the two server URLs and credentials for the server you want to mirror to. For example, if the server is running locally and your mirrorer is the default "admin" user with password "admin" then do:

   hackage-mirror http://hackage.haskell.org http://admin:admin@localhost:8080
   

This will do a one-time sync, and will bail out at the first sign of trouble. You can also do more robust and continuous mirroring. Use the flag --continuous. It will sync every 30 minutes (configurable with --interval). In this mode it carries on even when some packages cannot be mirrored for some reason and remembers them so it doesn't try them again and again. You can force it to try again by deleting the state files it mentions.

Current TODOs

Currently there are a few remaining points to be dealt with before a migration to the new codebase can be considered,

• Active tickets against Hackage 2
• HackageDB and HackageToDo
• Documentation needs love

Some further context regarding the state of the project can be found in,

•  http://www.haskell.org/pipermail/cabal-devel/2011-October/007803.html
•  http://www.haskell.org/pipermail/haskell-cafe/2012-February/099422.html

Documentation

• HackageDB/2.0/OldURIs - the documents and scripts in hackage-scripts' directory tree
• HackageDB/2.0/URIs - the planned new URIs and how to interact with them
• HackageDB/2.0/Architecture - ideas and implementations for internal code organization
• HackageDB/2.0/CommandLine - how to use the command line interface
• HackageDB/2.0/Switchover - Roadmap for hackage-server and the eventual switch to it.

People to ask

You should join the  cabal-devel mailing list. There are a number of people there who are interested or are working on bits of hackage. That's the place to ask questions, post patches for review etc.

As for specific people, ask Duncan (dcoutts on irc) and Matt (Gracenotes on irc). Duncan knows the general design ideas and strategy. Matt did his GSoC project last year on improving the server.

There is a #hackage irc channel where you will find some interested people.