cabal building broken libraries when module list is not complete

[m@…]

to reproduce:

• write a *.cabal file for a library, but omit a module in extra-modules list
• install that library
• use the library, and you will get linking errors, pointing out that the left out modules functions are missing (undefined reference)

to workaround:

• make shure that all modules are noticed in *.cabal

to fix (suggestion):

• don't build a library when some of its modules are not in *.cabal

[duncan]

This requires cabal to do proper dep chasing.

[claus]

installing broken libraries is no good at all - compare #306 for the version I ran into. looks like a bug to me, even if it needs more infrastructure before it can be fixed properly. in the mean time, is there no way to have cabal make a test install before packaging/upload, to make sure that the package is at least complete, builds, and loads (ie., using some compiler's dependency chasing to raise errors before distribution/installation)?

[duncan]

Sure it's a bug but it's a bug that depends on a huge new feature.

The problem is exactly that we're using the compiler's dependency chasing. It can find modules that we did not know about which makes it work at build time but then we miss things in the link phase.

[claus]

Replying to duncan:

Sure it's a bug but it's a bug that depends on a huge new feature.

Is Cabal dependency chasing scheduled to be available after the summer? Actually, it is a bit confusing that there are two separate "dependency chasing" issues floating around (between packages and inside packages; why not have one chaser, then extract either packages or modules?). This ticket seems to be blocked on #15, but #159 claimed to have implemented that 9 months ago?

The problem is exactly that we're using the compiler's dependency chasing. It can find modules that we did not know about which makes it work at build time but then we miss things in the link phase.

Which is why I asked for "test installs", ie, temporarily install package, then change to tmp dir and try to load installed package. If that fails, there was an unrecorded local dependency and, to some extent, vice-versa. Of course, that would depend on a working "uninstall" command (#106, #234)..

Btw, even while you're using the compiler's dependency chasing, doesn't that tell you which modules it finds and uses (GHC does)?

[claus]

Replying to claus:

Replying to duncan:

Sure it's a bug but it's a bug that depends on a huge new feature. The problem is exactly that we're using the compiler's dependency chasing. It can find modules that we did not know about which makes it work at build time but then we miss things in the link phase.

Btw, even while you're using the compiler's dependency chasing, doesn't that tell you which modules it finds and uses (GHC does)?

simply adding -v2 to build should give dependency info to stdout, among other info, eg, for some mini-dummy package p that only imports its own Paths_p we get:

$ runhaskell.exe Setup.hs build -v --ghc-options="-v2"
Creating dist\build (and its parents)
Creating dist\build\autogen (and its parents)
Preprocessing library p-0...
Building p-0...
Building library...
Creating dist\build (and its parents)
c:\ghc\ghc-6.9.20080514\bin\ghc.exe -v2 -package-name p-0 --make -hide-all-packages -i -idist\build
-i. -idist\build\autogen -Idist\build -odir dist\build -hidir dist\build -stubdir dist\build -packag
e base-3.0 -O P Paths_p
Glasgow Haskell Compiler, Version 6.9.20080514, for Haskell 98, stage 2 booted by GHC version 6.6.1
Using package config file: C:\ghc\ghc-6.9.20080514\package.conf
wired-in package ghc-prim mapped to ghc-prim-0.1
wired-in package integer mapped to integer-0.1
wired-in package base mapped to base-3.0
wired-in package rts mapped to rts-1.0
wired-in package haskell98 mapped to haskell98-1.0.1
wired-in package template-haskell mapped to template-haskell-2.2
wired-in package ndp not found.
*** Chasing dependencies:
Chasing modules from: P.hs,Paths_p
Stable obj: [P, Paths_p]
Stable BCO: []
Ready for upsweep
  [NONREC
      ModSummary {
         ms_hs_date = Wed Jun 25 15:05:33 GMT Daylight Time 2008
         ms_mod = p-0:Paths_p,
         ms_imps = [System.Environment, Data.Version]
         ms_srcimps = []
      },
   NONREC
      ModSummary {
         ms_hs_date = Tue Jun 24 15:00:42 GMT Daylight Time 2008
         ms_mod = p-0:P,
         ms_imps = [Paths_p]
         ms_srcimps = []
      }]

The ms_imps in the ModSummarys seem to give the basic dependency info you're looking for. There's also a --include-pkg-deps option, but I can't figure out how to make that make a difference:

 http://www.haskell.org/ghc/docs/latest/html/users_guide/separate-compilation.html#makefile-dependencies

[duncan]

To be honest I'd much rather spend time working on a proper solution than on intermediate hacky solutions like reading the output of ghc -v2. If someone else wants to implement that then fine I'm not totally against it but I don't really see it as a good use of our limited development time.

The module/file dependency chasing is scheduled to be done by the end of the summer however that project only aims to make a prototype and demonstrate it on the case of pre-processors. I expect that it will take a bit longer before we are able to use it to replace the use of ghc --make.

As for ticket #159, it's one of two existing prototypes for the make-style dependency problem. The GSoC project is pursuing a slightly different approach.

I don't think package dependency resolution is the same problem as file/module dependency chasing despite initial similarities.

[duncan]

When confirming a solution to this one, also check #586.

[elga]

*  rio bouygues