Template Haskell does not respect -package and -hide constraints

[guest]

(I am using 6.8.2-2ubuntu1, packaged for Ubuntu Hardy Heron, on amd64.)

Recently sjanssen added some patches to the Lambdabot darcs repository which replace a preprocessor, BotPP, with some Template Haskell. This seems to work well, however I ran into a problem with it - everytime TH was called, it would error out with a message about Bytestring 0.9.1.0.

This is far from the first time I've run into this problem, but it was not fixed by forcibly removing 0.9.1.0 & recompiling affected packages against bytestring 09.0.1!

After even more debugging, I finally figured it out:

The module TH was erroring out on has an 'import Text.Regex'. Text.Regex is installed by the regex-compat package. I had two regex-compats installed. regex-compat-0.91 was linked against bytestring0.9.0.1. regex-compat-0.92 was linked against (the removed) bytestring-0.9.1.0.

Now, normally Cabal detects that using 0.92 would lead to errors, and it selects 0.91, and it adds to the GHC options '-hide-all-package -package regex-compat-0.91', and everything compiles fine and the user is happy.

But! in this case, Template Haskell ignores the flags, and it uses the most recent regex-compat. And it is this that breaks the build. I ultimately resolved this issue by removing regex-compat-0.92 and reinstalling and linking it against 0.9.0.1, although I fear this may yet lead to bugs in the future.

Short summary: Cabal is smarter than TH and picks better package versions to compile against, and yet TH ignores Cabal's picks. This leads to unnecessary & difficult to figure out compile failures.

[simonpj]

Simon and I discussed this, and we can't understand enough about the problem to see what is going on. So we need a way to reproduce it. Can you give us some code, and a recipe for making it happen?

Simon

[guest]

I think I described it thoroughly; reproducing it requires carefully installing a package or two, and I'm not going to risk screwing up my installation yet again.

[simonpj]

OK. If you come across it again, please let us know and we'll take the opportunity to get more specific info. Meanwhile, we're stuck.

Simon

[simonmar]

We don't dispute there's a bug here, but it can't be as simple as just "TH ignores the package flags". The compiler only has one package environment which it constructs using the package flags on the command line, there's no separate package environment just for TH. TH uses exactly the same compilation and linking code as GHCi, so if something were wrong for TH it should also be wrong for GHCi. I've tried a few simple tests with the base package (we have both base-3 and base-4 now), and I can't see any evidence that TH or GHCi are "ignoring package flags".

If we even had a copy of the precise error message, that might help shed some light.

[duncan]

Replying to simonmar:

We don't dispute there's a bug here, but it can't be as simple as just "TH ignores the package flags".

My current guess is that stale .hi files refer to another version of a package, so despite what the -package flags say we pull in two versions of a package when TH runs.

That said, this should only happen if there are multiple versions of a package in the package graph. So perhaps this doesn't explain what gwern reported.