Improve update performance

[bos]

I have an automated package build set up using Hudson, and it tries to build my code as often as once every 5 minutes. Right now, that means running cabal update before every build, which takes at least 30 seconds. It would be great if cabal update used the ETag and If-Modified-Since HTTP headers to do conditional fetches of the metadata, so that there would be far less of a performance hit to both the Hackage server and my client.

[duncan]

More generally there is the incremental update idea as in ticket #428 so that it'll use much less bandwidth even when there are changes.

This is a rather weaker feature request so not strictly a duplicate. Even so it's probably better to go the whole hog on this one. It shouldn't be that much more work.