Optimisation sometimes decreases sharing in IO code

I don't think we're going to fix this any time soon. The simplifier is eta-expanding State# lambdas on purpose, because it's often critical for good performance of IO code, but it does risk re-computation in some cases, and this is one of those cases. Leaving the bug open because it is a real bug, someday we might implement something a bit more refined (e.g. arity analysis).

See also #1957.

Changing the subject to indicate the real cause of the problem.

This just came up on the mailing list again. The program was slow unless (-fno-state-hack) was used.

Is it possible to have GHC emit a warning (as part of -Wall) to let the use know that GHC is applying a potentially risky simplification?

A warning would be good, but I see two potential problems:

• there would be a lot of false positives

• the warning has to be generated by the simplifier, and source-location information has been stripped off by this point, so we don't have a good way to report the location of the warning.

Adding a link to the recent email thread http://www.haskell.org/pipermail/glasgow-haskell-users/2008-February/014259.html

Adding Adam Jenkins to the cc list. At the end of the above thread he writes (reasonably enough):

Thank you for the explanation. Inlining does explain the behavior I was seeing, and -fno-state-hack does make the program behave the way I'd expect.

I would like to humbly submit that perhaps this optimization should be off by default. It seems to me that any compiler optimization which can potentially increase your program's runtime by an order of complexity is dangerous. Also, while the Haskell Report doesn't seem to specify anything about lazy evaluation behavior, it seems to me that in order to program in Haskell, assuming that a computation assigned to a variable won't be recomputed multiple times is almost as necessary an assumption as assuming that the compiler will optimize away tail recursion. For instance GHC FAQ entry 7.2 implicitly assumes this is true:

http://haskell.org/haskellwiki/GHC:FAQ#Does_GHC_do_common_subexpression_elimination.3F

In my real program where I ran into this problem it took me quite a while to figure out why my program's performance wasn't making any sense, and why seemingly irrelevant changes to my code made the performance problem go away.

I agree that the current behaviour sometimes bites badly, as it did for you here. Trouble is, removing the hack has (or used to have) a significant negative effect on performance of lots of programs.

Still, I think that there are various other possibilities to try. For example, compiling the I/O libraries (only) with the state-hack on, and with it off by default elsewhere, might gather most of the benefits without the costs. Or restricting it to top level functions. Or something like that.

It needs someone who's willing to do some measurements and try variations.

See also #2284