ghc panic with mutually recursive modules and template haskell

[guest]

When compiling the files below using ghc --make Main.hs I get the following error:

[1 of 5] Compiling ModuleA[boot]    ( ModuleA.hs-boot, nothing )
[2 of 5] Compiling ModuleC          ( ModuleC.hs, ModuleC.o )
[3 of 5] Compiling ModuleB          ( ModuleB.hs, ModuleB.o )
Loading package base ... linking ... done.
Loading package template-haskell ... linking ... done.
ghc-6.6: panic! (the 'impossible' happened)
  (GHC version 6.6 for powerpc-apple-darwin):
        Maybe.fromJust: Nothing

ModuleA.hs:

module ModuleA where

import ModuleB

ModuleA.hs-boot:

module ModuleA where

ModuleB.hs:

{-# OPTIONS -fth #-}
module ModuleB where

import ModuleC

$(nothing)

ModuleC.hs:

module ModuleC where

import Language.Haskell.TH

import {-# SOURCE #-} ModuleA

nothing = return [] :: Q [Dec]

Main.hs:

module Main.hs

import ModuleA

main = return ()

[igloo]

Urk, this looks tricky. It will probably want to wait until we can use code spliced in earlier in a module in later splices in the same module.

[simonmar]

This now gives a slightly improved error message, as a result of the fix for #936:

[1 of 5] Compiling A[boot]          ( A.hs-boot, nothing )
[2 of 5] Compiling C                ( C.hs, C.o )
[3 of 5] Compiling B                ( B.hs, B.o )
Loading package base ... linking ... done.
module main:A cannot be linked; it is only available as a boot module

Simon: please feel free to edit the error message if you can think of a way to improve it (compiler/ghci/Linker.hs). Should we close this bug? Perhaps document the shortcoming in the TH docs?

[simonpj]

I've updated the docs. Ian, please merge.

Wed Jan 31 09:14:51 GMT 2007 simonpj@…

• Add note about Template Haskell and mutual recursion

Simon

[igloo]

Merged.

[fons]

Don't know if this deserves its own ticket but it seems that ghci (or ghc --make) fails when running a compile-time function imported from a mutually-recursive group of modules which _doesn't_ include the module currently being compiled.

ModuleA.hs-boot

module ModuleA

ModuleA.hs

module ModuleA where
import ModuleC

ModuleC.hs

odule ModuleC where

import Language.Haskell.TH

import {-# SOURCE #-} ModuleA

nothing = return [] :: Q [Dec]

ModuleB.hs (makes use of a function of C but is _not_ included in the group of recursive modules)

{-# LANGUAGE TemplateHaskell #-}
module ModuleB where

import ModuleC

$(nothing)

$ ghci ModuleB.hs
GHCi, version 6.8.1: http://www.haskell.org/ghc/  :? for help
Loading package base ... linking ... done.
[1 of 4] Compiling ModuleA[boot]    ( ModuleA.hs-boot, interpreted )
[2 of 4] Compiling ModuleC          ( ModuleC.hs, interpreted )
[3 of 4] Compiling ModuleB          ( ModuleB.hs, interpreted )
module main:ModuleA cannot be linked; it is only available as a boot module
> 

Furthermore, it would be really useful if the error message indicated the compilation failed due to Template Haskell. (It took me a while to figure out why it was caused)

[fons]

curiously enough compiling the following module doesn't cuase any problems.

Main.hs

module Main where

import ModuleB

main = return ()

$ ghc --make Main.hs
[1 of 5] Compiling ModuleA[boot]    ( ModuleA.hs-boot, ModuleA.o-boot )
[2 of 5] Compiling ModuleC          ( ModuleC.hs, ModuleC.o )
[3 of 5] Compiling ModuleA          ( ModuleA.hs, ModuleA.o )
[4 of 5] Compiling ModuleB          ( ModuleB.hs, ModuleB.o )
Loading package base ... linking ... done.
Loading package array-0.1.0.0 ... linking ... done.
Loading package packedstring-0.1.0.0 ... linking ... done.
Loading package containers-0.1.0.0 ... linking ... done.
Loading package pretty-1.0.0.0 ... linking ... done.
Loading package template-haskell ... linking ... done.
[5 of 5] Compiling Main             ( Main.hs, Main.o )
Linking Main ...

[simonmar]

Since ModuleB is outside the ModuleA/ModuleC loop, it can import ModuleA without creating any new loops, and I bet this will work around the problem and get you unblocked.

What's happening is that the dependency analysis isn't figuring out that the real ModuleA must be compiled before ModuleB. I think the solution is something along the lines of: every module that depends on a module in a cycle, but is not a member of that cycle, should have an implicit dependency on each of the modules in the cycle.

[simonpj]

Simon is right: B.hs depends on C.hs, which depends on A.hs-boot. I think it's quite accidental that A happens to be compiled before B in your "curiously enough" case, but after B in your original case.

All you need do is to make B depend on A, thus

{-# LANGUAGE TemplateHaskell #-}
module ModuleB where

import ModuleC
import ModuleA

$(nothing)

and it all works fine. Use -v to see the compilation order.

I will clarify the documentation, thus:

You can only run a function at compile time if it is imported from another module that is not part of a mutually-recursive group of modules that includes the module currently being compiled. Furthermore, all of the modules of the mutually-recursive group must be reachable by non-SOURCE imports from the module where the splice is to be run.

Fri Jan  4 12:19:39 GMT 2008  simonpj@microsoft.com
  * Document SOURCE pragma; clarify TH behavior for 
    mutually-recurive modules (Trac #1012)

It's not clear to me how to improve the error message, at least not without adding more plumbing to say "I'm in a Template Haskell splice". Let's see if it happens again.

Meanwhile I'll close the bug.

Simon

[fons]

Replying to simonpj:

Simon is right: B.hs depends on C.hs, which depends on A.hs-boot. I think it's quite accidental that A happens to be compiled before B in your "curiously enough" case, but after B in your original case. All you need do is to make B depend on A, thus {{{ {-# LANGUAGE TemplateHaskell #-} module ModuleB where import ModuleC import ModuleA $(nothing) }}} and it all works fine. Use -v to see the compilation order.

I finally managed to solve the problem "turning around" the circular dependencies of my project (tranforming the SOURCE imports of the loop in normal onces and vice versa). However, the workaround suggested by Simon should work as you said.

Meanwhile I'll close the bug.

As far as I understand, ghc's dependency analysis could be improved (otherwise a workaround would not be needed). I don't personally think this bug should be closed and include it as a limitation in the docs before considering simonmar's proposal, that is:

What's happening is that the dependency analysis isn't figuring out that the real ModuleA must be compiled before ModuleB. I think the solution is something along the lines of: every module that depends on a module in a cycle, but is not a member of that cycle, should have an implicit dependency on each of the modules in the cycle.

[igloo]

I agree that it is still a bug, but I've put it back in the _|_ milestone as it's easy to work-around, and I assume that if it was trivial to fix then SPJ would have done so.

[simonpj]

Fair enough. I have taken a little look at this, based on fons's suggestion "every module M that depends on a module C in a cycle, but is not a member of that cycle, should have an implicit dependency on each of the modules C1.. Cn in the cycle.". Yes, I think that would not be too hard to do. There are two places to think about:

• ghc --make: When deciding the up-sweep order, first do a SCC analysis finding strongly connected components of modules, and top-sort those components. Then linearise each component. That gives a linear order that respects fons's suggestion.

• ghc -M: similar story, but less neat. We have to emit lots of extra dependencies in the makefile, so that M depends on C1..Cn.

Not very hard, but more than an hours work. Let's do it for 6.10.

Simon

[dleuschner]

Just a note: Of course this is not an important bug/limitation. It would be still nice if it would just work. It took me some time to realise what the problem is, find this bug report and the section in the manual and reorganise the code to accomodate for the limitation. Before that I did several clean/rebuild, rebuild without "make -j" cycles to be sure that it's not a problem with the build system. It's a bit unexpected because normally GHC just does everything I want (and even more). :-)