sum and product thunkpile

[Bart Massey]

Do "ulimit -v 524288" at the shell so your machine won't hang. Run ghci, bring in Data.List, and execute "sum [1..10**8]". Notice that you run out of memory. Now try again with "foldl1' (+) [1..10**8]". Note that you get an answer.

The definition of sum and product in the Report use foldl. However, the versions currently being used are open-coded for some reason. They should just use foldl1' (after handling the special case of the empty list appropriately). As it is, the functions thunkpile; this seems pointless, as they are ultimately completely strict in their argument.

This kind of bug is grim for new users, which is why the severity is marked "major".

Patch attached. This is just a context unidiff against current Darcs head; a Darcs-format patch with full context (at 350KB) is at  http://po8.org/bart/sumprod.patch

[Bart Massey]

Oh, BTW, maybe product needs to be open-coded (but not the current way!) so that it can be properly lazy on encountering a 0 in the list. I have mixed feelings about this; on the one hand, it's in the spirit of "maximal laziness", on the other hand, it's a bit strange.

Should I file another bug report or proposal for this?

[simonmar]

We can't apply this patch, because it changes the semantics of sum & product. Note that sum is overloaded, and there's no requirement that (+) is strict for any given instance of Num.

When used at Int or Integer, we do indeed have strict specialised versions of sum, but these don't get used inside GHCi, because GHCi doesn't do optimisation. Perhaps there's a case that we should be doing this kind of optimisation in GHCi, but that's another story (and should be another ticket).