ghc: panic! (the 'impossible' happened)

[EricKow]

I downloaded the beta of the Haskell Platform installer from  http://hackage.haskell.org/platform/2009.2.0.1/haskell-platform-2009.2.0.1-beta1-i386.dmg

and when I tried to cabal install the HEAD version of darcs, I got this error message:

[ 57 of 134] Compiling Darcs.Repository.Prefs ( src/Darcs/Repository/Prefs.lhs, dist/build/Darcs/Repository/Prefs.o )
ghc: panic! (the 'impossible' happened)
  (GHC version 6.10.3 for i386-apple-darwin):
	cat_evals
    base:GHC.Arr.Array{d ras}
    [ww{v a2ZC6} [lid], ww1{v a2ZC7} [lid], ww2{v a2ZC8} [lid]]
    [!, !, _, _]

Please report this as a GHC bug:  http://www.haskell.org/ghc/reportabug

Darcs repo version (last patch; see attached for context file):

Mon Jun  1 10:19:28 BST 2009  Petr Rockai <me@mornfall.net>
  * Remove tentativelyMerge from Gorsvet, as it's unused and confusing.

[EricKow]

version of darcs I tried to cabal install

[EricKow]

In the event that this is at all relevant, I was simultaneously trying to compile HaXml? 1.13

[EricKow]

Also, I'm pleased to report that this error does not go away if I type "cabal install" a second (or Nth time). So I'll be glad to do any debugging steps you request.

[igloo]

I can't reproduce this on my OS X/x86 machine, with the 6.10.3 installer from the GHC download page and a hand-compiled cabal-install.

I also can't install that beta. It says

The following install step failed: run postinstall script for hp-postflight. Con
tact the software manufacturer for assistance.

Eric, can you see if you can reproduce it by running ./Setup build, or even ghc --make, by hand please?

If you could boil it down to a minimal testcase then that would be very useful indeed. Just try replacing right hand sides of top-level things with undefined, and remove functions and imports that are no longer used as a result (warnings are a great help here).

In particular, removing dependencies on other packages, and removing any special preprocessing etc, is valuable, so that the bug can be reproduced with just ghc --make ....

[GregoryCollins]

I can reproduce this also -- hoping it isn't an issue with the platform libraries.

[EricKow]

Success! Here's the smallest I can get it

{-# OPTIONS_GHC -O2 #-}
module Foo where

import Text.Regex ( matchRegex )
import Data.Maybe ( isJust )

foo :: IO (FilePath -> FilePath)
foo = do
    regexes <- return undefined
    let isbin f = or $ map (\r -> isJust $ matchRegex r f) regexes
        ftf f = if isbin f then undefined else undefined
    return ftf

[igloo]

Great stuff, thanks Eric! I can now reproduce this:

{-# OPTIONS_GHC -O2 #-}
module Foo where

import Text.Regex

isJust :: Maybe a -> Bool
isJust (Just _) = True
isJust Nothing = False

foo :: IO (FilePath -> FilePath)
foo = do
    regexes <- return undefined
    let isbin f = or $ map (\r -> isJust $ matchRegex r f) regexes
        ftf f = if isbin f then undefined else undefined
    return ftf

$ ghc --make q               
[1 of 1] Compiling Foo              ( q.hs, q.o )
ghc: panic! (the 'impossible' happened)
  (GHC version 6.10.3 for i386-apple-darwin):
        cat_evals
    base:GHC.Arr.Array{d rfk}
    [ww{v aLo} [lid], ww1{v aLp} [lid], ww2{v aLq} [lid]]
    [!, !, _, _]

Please report this as a GHC bug:  http://www.haskell.org/ghc/reportabug

It looks like despite the installer failing, I'm still getting regex-compat from /Library/Frameworks/HaskellPlatform.framework. There is also a copy in /Library/Frameworks/GHC.framework.

Core lint says:

$ ghc --make q -dcore-lint
[1 of 1] Compiling Foo              ( q.hs, q.o )
ghc: panic! (the 'impossible' happened)
  (GHC version 6.10.3 for i386-apple-darwin):
        Iface Lint failure
    Unfolding of regex-compat-0.71.0.1:Text.Regex.matchRegex{v r3n}
      <no location info>:
          In a case alternative: (base:Data.Maybe.Nothing{d r8})
          In a case alternative, data constructor isn't in scrutinee type:
          Scrutinee type constructor: ghc-prim:GHC.Types.[]{(w) tc 317}
          Data con: base:Data.Maybe.Nothing{d r8}

Please report this as a GHC bug:  http://www.haskell.org/ghc/reportabug

In

ghc --show-iface /Library/Frameworks/HaskellPlatform.framework/lib/regex-compat-0.71.0.1/ghc-6.10.3/Text/Regex.hi

is

3170b173e0108c4d0f4e4a6c458e962b
  matchRegex :: Text.Regex.Posix.Wrap.Regex
                -> GHC.Base.String
                -> Data.Maybe.Maybe [GHC.Base.String]
    {- Arity: 2 Strictness: LL
       Unfolding: (\ p :: Text.Regex.Posix.Wrap.Regex
                     str :: GHC.Base.String ->
                   case @ (Data.Maybe.Maybe
                               [GHC.Base.String]) Text.Regex.Posix.String.a4 p str of wild {
                     Data.Maybe.Nothing -> Data.Maybe.Nothing @ [GHC.Base.String]
                     Data.Maybe.Just preMApost
                     -> Data.Maybe.Just
                          @ [GHC.Base.String]
                          (case @ [[GHC.Types.Char]] preMApost of w { (ww, ww1, ww2) ->
                           case @ [[GHC.Types.Char]] Text.Regex.Base.Context.$wlvl3
                                                       @ [GHC.Types.Char]
                                                       ww
                                                       ww1
                                                       ww2 of ww3 { (# ww4, ww5, ww6, ww7 #) ->
                           ww7 } }) }) -}
5497cf2d6b99de31ced55bda7d2a1265

In

ghc --show-iface /Library/Frameworks/HaskellPlatform.framework/lib/regex-posix-0.72.0.3/ghc-6.10.3/Text/Regex/Posix/String.hi

is

270f3a6a066ef52e8035aeae31f61c08
  a4 :: Text.Regex.Posix.Wrap.Regex
        -> GHC.Base.String
        -> [Text.Regex.Base.RegexLike.MatchText GHC.Base.String]
    {- Arity: 2 Strictness: LL -}
cd1e8839869c9407d5b2d1f9af8a4d06

In

ghc --show-iface /Library/Frameworks/GHC.framework/Versions/610/usr/lib/ghc-6.10.3/regex-posix-0.72.0.3/Text/Regex/Posix/String.hi

is

340c19d7034daffc230c61fd7b7524a7
  a4 :: Text.Regex.Posix.Wrap.Regex
        -> GHC.Base.String
        -> Data.Maybe.Maybe
               (GHC.Base.String,
                Text.Regex.Base.RegexLike.MatchText GHC.Base.String,
                GHC.Base.String)
    {- Arity: 2 Strictness: LL -}
57f7a827fcdc372c03316de2b8a05865

so it looks like the problem is that regex-compat got built against the regex-posix that comes with GHC rather than the one that comes with the Haskell platform.

So, two questions:

Why did this happen?

Isn't the fingerprinting meant to catch this?

[GregoryCollins]

As a workaround, you can run:

    sudo cabal install -O2 --reinstall regex-base regex-compat regex-posix

which seems to clear up the problem for me. The underlying issue still remains.

[GregoryCollins]

Sorry, I meant to say:

sudo cabal install -O2 --reinstall --global regex-base regex-compat regex-posix

[GregoryCollins]

Haskell platform is tracking this as  http://trac.haskell.org/haskell-platform/ticket/71, I'll cc: this message to the ticket there.

There must be some subtle ABI incompatibility that is triggering this. Actually, I bet I know what's causing this. Roughly speaking, here's the sequence of events when I build the installer:

• I install GHC from binary

• I build the libraries in LISTED order (we'll return to this):

  ...
  regex-base    ==0.72.0.2,
  regex-compat  ==0.71.0.1,
  regex-posix   ==0.72.0.3,
  ...
  

• Each library gets packaged into a .pkg file which, when installed, copies the generated libs to the destination and runs "ghc-pkg register"

So regex-compat gets built against GHC's regex-posix instead of the platform's, then we build a NEW regex-posix against the platform libraries and GHC panics when you try to use -compat. The two regex-posix libraries have the same version number but are installed in different places and obviously aren't binary-identical, probably some autogenerated symbol names are different.

The obvious workaround is to build and install the platform libraries in strict topological order -- could we re-order haskell-platform.cabal so that each library's dependencies are listed before it? Or could someone tell me how to do this topological sort using the Cabal library?

Hopefully this message will create an "aha!" moment for someone who knows the GHC internals better than I do (which is to say, "not at all").

[igloo]

OK, that does indeed explain why it happened. You'll need to either build the packages in topological order, or to just not rebuild the packages that GHC comes with anyway. This will be less of an issue in 6.12 when the GHC installer won't install any extralibs.

And to answer my other question: No, fingerprints aren't meant to catch this problem. We will doubtless think more about this when we come to working out the details of dealing with shared libraries.

I'll therefore close this ticket. As Gregory said, the problem in the OS X platform installer is being tracked here:  http://trac.haskell.org/haskell-platform/ticket/71