Heap used by compacting collector larger than heap used without.

[r6]

When you run a program that slowly accumulates memory with +RTS -M100M -c, the memory space used by the application will grow to around 200MB and it will crash with an out of heap error when the live data is around 100MB.

On the other hand if you run the program with +RTS -M100M, the memory space used by the application will grow to around 100MB and it will crash with an out of heap error when the live data is around 50MB.

This different interpretation of -Mfoo is confusing for the user. I find the latter interpretation more natural than the former.

[r6]

Here is a small example program:

module Main where

import qualified Data.ByteString as BS

l = map (BS.replicate (1024^2 `div` 2)) [0..]

main = do
  print (map BS.head l)
  print (map BS.head l)

For me, ghc --make -O2 Alloc.hs && ./Alloc +RTS -M100M -c -S runs until the heap is exhasted and outputs:

Heap exhausted;
Current maximum heap size is 104857600 bytes (100 MB);

      104,220,800 bytes allocated in the heap
             198 MB total memory in use (98 MB lost due to fragmentation)

On the other hand, ghc --make -O2 Alloc.hs && ./Alloc +RTS -M100M -S runs until the heap is exhasted and outputs:

Heap exhausted;
Current maximum heap size is 104857600 bytes (100 MB);
      52,397,636 bytes allocated in the heap
             100 MB total memory in use (49 MB lost due to fragmentation)

[simonmar]

I suspect this is fixed. Your program didn't work, because [0..] has type [Word8] and thus only has 256 elements, so I had to modify it slightly:

l = map (BS.replicate (1024^2 `div` 2) . fromIntegral) [(0::Int)..]

with this change, I get the same behaviour with and without -c. First, with +RTS -M100m -c -s:

eap exhausted;
Current maximum heap size is 104857600 bytes (100 MB);
use `+RTS -M<size>' to increase it.
     155,883,256 bytes allocated in the heap
     103,920,168 bytes copied during GC
     103,930,080 bytes maximum residency (9 sample(s))
         878,368 bytes maximum slop
             105 MB total memory in use (0 MB lost due to fragmentation)

and with +RTS -M100m -s:

Heap exhausted;
Current maximum heap size is 104857600 bytes (100 MB);
use `+RTS -M<size>' to increase it.
     155,883,256 bytes allocated in the heap
     162,745,832 bytes copied during GC
     103,930,064 bytes maximum residency (9 sample(s))
       1,046,320 bytes maximum slop
             105 MB total memory in use (0 MB lost due to fragmentation)

I got the same results with 7.0.3 and the current master.