performance problem compiling large file

As well as triggering the internal error, I found it necessary to increase the heap size, and it took *AGES*. So, I guess it's two bugs - the internal error and the performance bug.

cheers, Tom

I'm 99% certain the crash has been fixed; see #787. However I tried the attached file, and it loaded without error on my 6.4.2 installation here.

Changing the title to reflect the performance problem, this is definitely an issue.

Still a problem in 6.6 and the HEAD.

See also #1875, #1136

With a slightly simpler variant of this:

module Foo where

foo :: Double -> Int
foo x | x == 1 = 1
foo x | x == 2 = 2
...
foo x | x == 500 = 500

compiling with

ghc -c large.hs -fforce-recomp +RTS -p -h

one problem is that we get something like

lit1 = ...
lit2 = ...
...

foo = case ... of
          False ->
              case ... of
                  False -> ...
                  True -> lit2
          True -> lit1

Each of the case expressions has an SRT for its alternatives. The outer one has all 500 lit's, the one inside 499, and so on, so we have quadratic space usage. The SRTs are SRTEntries cafs, where cafs is an IdSet (which is ultimately a UniqFM), so no sharing is possible.

In this particular case using -O "fixes" it as == etc get specialised and the lits disappear, but (a) this won't be the case in general and (b) I don't think you should need to use -O to get reasonable compiler space usage.

OK I have fixed the immediate problem.

Thu Mar  6 13:47:34 GMT 2008  simonpj@microsoft.com
  * Fix Trac #783: improve short-cutting literals in the type checker

Ian can you merge, and (somehow) add a test case to the test suite?

I'm not sure how to fully solve the quadratic behaviour in general. The reason that the lits were appearing in the SRT is because the dictionary for Num Float has a CAF in it. So, as Ian says, there really are N sub-SRTs for the various stages of evaluation. However things are now less bad than before:

• Ian has made the SRTs use bitmaps:

  Wed Feb 27 06:45:05 PST 2008  Ian Lynagh <igloo@earth.li>
    * Add and use seqBitmap when constructing SRTs
  

• The above patch from me makes the literal problem go away for Float and Double (common cases)

However, as Simon M says, I'm not sure why the N sub-SRTS don't all share a single table, and that should really kill off the quadratic behaviour in this case. That is the remaining bit that would be worth investigating. Maybe leave the ticket open for that.

Simon

I've merged the patch.

Also, just to clarify, the patch Add and use seqBitmap when constructing SRTs is actually in a later stage of the compiler (although I did find that problem while looking at this bug). Going directly to a bitmap, rather than first making the SRTEntries, might be a way to fix this bug.