seemingly too liberal behaviour on overlapping dependency constraints across packages

[kowey]

There's a high chance this is not actually a bug, but I'm filing one anyway.

In  http://bugs.darcs.net/isuse1763 we have darcs-2.4 which depends on mmap >= 0.2 and on hashed-storage >= 0.4.7 && < 0.5

The hashed-storage 0.4.x line depends on mmap == 0.4.*

The bug in darcs happened when darcs was linked against mmap-0.5 while hashed-storage against mmap-0.4.

Is it reasonable to expect that if I depend on a package, that the package can indirectly impose at least soft constraints (on the cabal-install level) on our shared dependencies? Should cabal-install have been smart enough to work out that darcs should actually only pick up mmap 0.4? I think I could be quite happy with "no"

[kowey]

Apologies for the broken  http://bugs.darcs.net/issue1763 link

[kowey]

Replying to kowey:

Apologies for the broken  http://bugs.darcs.net/issue1763 link

Gah! That should be  http://bugs.darcs.net/issue1753

[duncan]

The very short answer is no, you cannot rely on the behaviour you want.

The long answer is, that it depends on the method you use.

Yes, "cabal install darcs" will always ensure that a single version of shared dependencies is used.

If you use the lower level "runghc Setup configure" interface then it will pick the latest mmap rather than the one used by hashed-storage. It will however warn that it is indirectly using two versions of the same package.

If you use "cabal configure" then it will usually pick the same version of mmap as used by hashed-storage however this may not possible without e.g. rebuilding one or other of the packages to get consistent deps. If it was not possible then "cabal configure" will also just pick the latest version of mmap and issue a warning.

The "cabal install" command is in a position to rebuild dependencies so as to achieve consistent deps and indeed it cannot produce solutions that use inconsistent deps (many people see this as a limitation in the dependency resolver).

In general, in future you should not expect cabal to always pick the same versions of shared dependencies, in particular where those dependencies are "private" dependencies (that is where no types from the private dep are exposed through the public interface). The ability to pick different versions of private dependencies in different places is actually important from the point of view of encapsulation. On the other hand, for public dependencies the cabal tool will continue to pick common shared versions.

The reason the other commands allow you to pick inconsistent dependencies is as follows. The reason we try to pick consistent dependencies is to avoid compilation failure, not to avoid runtime failure as in your case. Since picking consistent deps is a conservative approximation it is important occasionally for user to be able to indicate that they want inconsistent dependencies because they happen to know it will work. Your example is pretty much the worst case, the compilation worked but for some reason using two versions, or at least the wrong version.

We should make sure we understand the reasons for the mmap failure. We should distinguish whether the problem was using the wrong version, or if the problem was using a different version.

If we feel we need to be stricter about using common versions then perhaps we should make the commands that currently do allow inconsistent deps do so only with a special override flag.

[duncan]

So we believe the problem is that the mmap package contains C code and so when multiple versions of that package are linked into one program then the linker just picks the first occurrence of the C functions. GHC packages are designed to allow linking multiple versions however this property is broken by including unversioned C symbols in the package.

There are a number of possible solutions and workarounds:

• Solution: avoid using C code in the mmap package
• Solution: use versioned C symbol names (versioned with the package name/version)
• Workadound: avoid linking multiple versions of the mmap package.

See ticket #701 for some more general ideas.

[mornfall]

As for solution 1, is there a way to use C finalizers without having C code (and C symbols)? The workaround is what we already did (we copied the mmap dependency from hashed-storage into darcs.cabal), but I don't think it fixed all version mixups (see also  http://irclog.perlgeek.de/darcs/2010-06-14#i_2434140). Would solution 2 involve manual renaming of all C symbols in each version? (That is quite inconvenient and error-prone...) The C functions are supposed to be private, so maybe a hiding solution would work better? Is that doable with current Cabal?

[duncan]

Replying to mornfall:

As for solution 1, is there a way to use C finalizers without having C code (and C symbols)?

I don't think so.

Would solution 2 involve manual renaming of all C symbols in each version? (That is quite inconvenient and error-prone...)

To do it sensibly you'd want to use a CPP macro to construct the C function name. The macro would depend on a CPP symbol which you'd define to the package name and version. That define would have to be generated by the Setup.hs.

The C functions are supposed to be private, so maybe a hiding solution would work better? Is that doable with current Cabal?

I don't think that's trivial to implement outside of Cabal. It'd involve linking objects together using ld -r -x to get one .o file with all internal symbols hidden.

[kosmikus]

There seems to be an open issue here, so I'm keeping this around. But the issue should probably be renamed. I don't think the issue that has originally being reported (about different versions being chosen or not) will be changed or fixed.