identify and exclude broken or unusable packages when constructing and merging package dbs

[guest]

The following error appears like nonsense:

cabal: dependencies conflict: ghc-6.10.1 requires process ==1.0.1.0 however
process-1.0.1.0 was excluded because ghc-6.10.1 requires process ==1.0.1.1

It actually means that there's some package ghc depends on that now is built against the newer process. It should tell the name of the other package(s), too.

This is made worse by user-installed packages of the same version shadowing global ones, in this case there is no real error.

[duncan]

Replying to guest:

It actually means that there's some package ghc depends on that now is built against the newer process. It should tell the name of the other package(s), too.

That's true in this case, I'm not sure there is any particular package to identify in general. We can identify the whole chain of packages involved in the conflict, though I don't know if it would clarify the situation.

This is made worse by user-installed packages of the same version shadowing global ones, in this case there is no real error.

There is a real error but yes it is caused by the user package shadowing the global one. It is not just cabal-install's constraint solver that bumps into this error, ghc --make will do the exact same thing and try to link the ghc package against the wrong version of one of its dependencies (the one from the user package that is doing the shadowing).

The short term solution is to modify the code for overlaying package databases to mark packages as broken when one of their dependencies get shadowed. This requires extending the representation of the installed packages to record if a package is present but broken (which is generally useful info to have around since packages can be broken for a number of reasons like missing dependencies).

The long term solution is to identify installed packages by their ABI hash. Then this problem neatly goes away.

[duncan]

We should identify these broken and unusable packages when constructing and merging package dbs. Then when solving we should add exclude constraints on such packages. If those packages are then needed then we should report the reason they were excluded.

It may also be worth warning whenever we detect this situation at all, even if we do not need the packages in question. Something like ghc-pkg check perhaps to do a global package consistency check.

[kosmikus]

I think this is mostly fixed in the HEAD. Should try to verify once again.

[kosmikus]

This turned out to be trickier than I first expected.

The problem is that the underlying shadowing is an incredibly subtle thing.

Here are a few facts:

• The new modular solver produces a better error message.
• Unlike the old (topdown) solver, the modular solver will just consider multiple installed instances of a package to all be available, even if they have the same version.
• It's possible to install packages with cabal-install that ghc/ghci will not be able to load with a simple -package flag, because GHC applies shadowing in such a situation. However, explicitly selecting the packages with -package-id should help in such a case.
• It's possible for Cabal to fail at build time because currently, the interface between cabal-install and the build process is suboptimal. We only pass version constraints to the individual package builds, not the exact package identities, even if we have them.
• I've added functionality to the modular solver so that it can, if desired, apply shadowing itself, by disabling packages from the installed package index.
• Broken packages with missing dependencies should be disabled now in the solver.
• Applying shadowing seems like the wrong thing to do, though, and despite the lossy flag passing to the builder, the situations that fail are rare (fewer than would be rejected if shadowing was enabled in the index). So I've disabled shadowing for now.

[kosmikus]

Also see #937.

[kosmikus]

I'm closing this one, as the original problems, the error message and the handling of broken packages, are fixed.