Segmentation fault on long input (list of pairs)

[cathper]

When compiling LongList?.hs with

ghc --make LongList.hs

then running the output gives a segfault:

$ ./LongList
8595
Segmentation fault

It also segfaults with ghc 6.12.1 (without printing 8595) and with ghc 6.12.3.

On ghc 6.12.3 running on Mac OS 10.5, no segfault is seen. (I haven't tried much longer input, since it already eats up approx. 1.2 GB of RAM.)

[cathper]

A file with a very long line.

[simonmar]

Did you manage to compile this file? I left it compiling overnight last night, and it still hasn't finished.

[cathper]

Yes. The compilation time varies quite much, though.

> ghc --version; wget -q http://hackage.haskell.org/trac/ghc/raw-attachment/ticket/4505/LongList.hs && time nice ghc --make LongList.hs
The Glorious Glasgow Haskell Compilation System, version 6.12.1
[1 of 1] Compiling Main             ( LongList.hs, LongList.o )
Linking LongList ...
nice ghc --make LongList.hs  63,91s user 1,11s system 99% cpu 1:05,28 total
> rm LongList*
> /tmp/ghc7/bin/ghc --version; wget -q http://hackage.haskell.org/trac/ghc/raw-attachment/ticket/4505/LongList.hs && time nice /tmp/ghc7/bin/ghc --make LongList.hs
The Glorious Glasgow Haskell Compilation System, version 7.0.1
[1 of 1] Compiling Main             ( LongList.hs, LongList.o )
nice /tmp/ghc7/bin/ghc --make LongList.hs  515,28s user 2,06s system 99% cpu 8:40,01 total
> uname -a
Linux freja 2.6.32-25-server #44-Ubuntu SMP Fri Sep 17 21:13:39 UTC 2010 x86_64 GNU/Linux
> cat /proc/cpuinfo|grep "model name"|head -n 1
model name      : Intel(R) Xeon(R) CPU           E5420  @ 2.50GHz

It uses > 1 GB of RAM.

[simonmar]

I understand what the problem is here. The program compiles to code that allocates >1MB in one go, and the runtime doesn't currently support that (we support allocating >1MB only for a single object, such as an array, not for multiple objects). This is a zero-day bug: it's been around for ever, but nobody has run into it until now.

Worst case we can make the RTS emit an error message rather than just segfaulting, but I hope we'll be able to find a way to fix it properly. I'll look into it.

Meanwhile, it seems the HEAD can't compile the test program at all, unless a type signature is added, and 7.0.1 is significantly slower than 6.12.3 to compile it. I've created #4801 for the compiler performance regression.

[simonmar]

Not a blocker, because it's not a regression.

[simonmar]

I improved the error message. Please merge these for 7.0.2, and then re-milestone to 7.2.1 for a proper fix.

Wed Dec  8 08:32:12 PST 2010  Simon Marlow <marlowsd@gmail.com>
  * Tweak the "sorry" message a bit
  
  -		"sorry! (this is work in progress)\n"
  +		"sorry! (unimplemented feature or known bug)\n"

Thu Dec  9 03:40:05 PST 2010  Simon Marlow <marlowsd@gmail.com>
  * Catch too-large allocations and emit an error message (#4505)
  
  This is a temporary measure until we fix the bug properly (which is
  somewhat tricky, and we think might be easier in the new code
  generator).
  
  For now we get:
  
  ghc-stage2: sorry! (unimplemented feature or known bug)
    (GHC version 7.1 for i386-unknown-linux):
          Trying to allocate more than 1040384 bytes.
  
  See: http://hackage.haskell.org/trac/ghc/ticket/4550
  Suggestion: read data from a file instead of having large static data
  structures in the code.

Thu Dec  9 12:04:04 GMT 2010  Simon Marlow <marlowsd@gmail.com>
  * fix ticket number (#4505)

[simonmar]

(the priority bump was an accident, in case you were wondering)

[cathper]

Replying to simonmar:

I improved the error message. Please merge these for 7.0.2, and then re-milestone to 7.2.1 for a proper fix.

I'm a Trac novice. You mean, merge it as fixed (so the ticket is closed), and open a new ticket that has 7.2.1 as milestone?

See:  http://hackage.haskell.org/trac/ghc/ticket/4550

s/4550/4505/ I guess.

[simonmar]

Replying to cathper:

I'm a Trac novice. You mean, merge it as fixed (so the ticket is closed), and open a new ticket that has 7.2.1 as milestone?

Sorry for the confusion, that message was intended for Ian, who does all the merging.

See:  http://hackage.haskell.org/trac/ghc/ticket/4550

s/4550/4505/ I guess.

Yes, I fixed that in a subsequent patch.

[cathper]

Just perfect :-)

[simonmar]

One further patch to merge:

Fri Dec 10 09:40:02 GMT 2010  Simon Marlow <marlowsd@gmail.com>
  * warning fix: don't redefine BLOCKS_PER_MBLOCK

[igloo]

First 3 merged.

[igloo]

Last one also merged.

[simonmar]

no proper fix in sight until we have the new code generator.