Distinguish speculative and required build-depends upper bound

[ezyang]

When a package adds a build-dependency on another package, it's good form to follow the Package Versioning Policy and add a speculative upper bound that is x.(y+1), if the current version is x.y. This is great, but in many cases the next version will /not/ break compatibility with your package. In that case, once you have tested that the dependency is fine, you can bump the upper bound.

As the list of dependencies for any respectable Cabal package can be quite long, it may be useful to have some sort of 'cabal whatsnew' command which checks the upper bounds of your build dependencies and notifies you when there are newer versions than your upper bounds availble. Furthermore, there should be a mechanism for telling Cabal that the upper bound is required, and that the package will break if you try to bump it.

[duncan]

I agree we should have tools that can help with checking if packages will in fact build with later versions of their deps. I'm not convinced at this stage that we need to record extra information in the .cabal file to distinguish "conservative" from "known" upper bounds.

[wk]

For testing whether a GHC bug had been fixed in HEAD, I just went through blindly bumping quite a few upper bounds in cabal dependency declarations in Agda and the packages it requires, since GHC HEAD of course contains versions that not everybody has tested against, yet. (The resulting Agda actually works.)

Abstractly, for each package in the dependencies of a another package, there is a dependently-typed function

  status  : \Pi version : Version
          . \powerset (flags version)
          -> {KnownToWork, KnownToBreak, Unknown}

(assuming no interaction between dependencies).

This function is only very poorly approximated by the current range sets.

The current scheme does not distinguish between KnownToBreak and UnKnown, and conservatively refuses to build for any version outside the declared range.

This is a real hassle for the use case described above, namely trying out new versions of anything very deep down the dependency graph (like the packages that come with GHC...).

If the dependency declaration allowed to distinguish between KnownToBreak and UnKnown, cabal could just issue a warning when compiling against a version with UnKnown status, and refuse (with an improved error message) when only finding KnownToBreak versions.

By the way, an automatic tool run can only be used to obtain KnownToBreak, but (almost) never KnownToWork --- a "KnownToCompile" is in general only an UnKnown, and in the best case might, via *Check, be turned into KnownToBreak.

I therefore strongly recommend to require a distinction between KnownToBreak and UnKnown in Cabal dependency declarations.

Wolfram

[igloo]

FWIW, this could make hackage regression testing with the GHC HEAD more useful.

[duncan]

Recording this extra information in the context of hackage is much easier. For one thing it can be discovered automatically. Adding this information to .cabal files is more tricky, we need a syntax for it and users have to specify it.

[wk]

> Recording this extra information in the context of hackage is much easier.

I don't understand exactly what you mean, but it seems to exclude recording this "extra information" for unpublished packages and local developments?

> For one thing it can be discovered automatically.

You can discover automatically whether it compiles. You cannot, in general, discover automatically whether it works correctly.

> Adding this information to .cabal files is more tricky, we need a syntax for it and users have to specify it.

I don't think the syntax question is tricky --- if you know the semantics you want to enable, you have tons of options.

One particularly easy option would interpret the current ranges as establishing mappings to KnownToWork, and add a separate notation (perhaps using a ! prefix) to establish mappings to KnownToBreak. Versions not mentioned in either would be interpreted as being mapped to UnKnown.

I estimate that most currently-listed excluded upper bounds are intended to mean UnKnown and not KnownToBreak. Therefore, this would even be backwards-compatible.

(The semantics of sequences of such positive or negative ranges would most naturally be Map.foldWithKey Map.insert, that is, a right-biased version of Map.union.)

If UnKnown gives rise only to a warning instead of to a refusal to even attempt compilation (as it does now), that will make it also much easier for package authors to be honest about old versions they did not check against.

Wolfram