Obscure loop in interaction between arity and case-of-bottom

[simonpj]

John Meacham discovered that this program makes the simplifier loop:

newtype T = V T deriving(Eq,Ord)

It's a silly program, but it shouldn't kill GHC! I investigated and the reason is because we get this:

  rec { compare :: T -> T -> Ordering
        compare [arity 2] = compare }
   
  max :: T -> T -> T
  max x y = case compare x y of { GT -> x; other -> y }

The real bug is that compare has been eta-reduced, but still has arity 2. That's wrong. Anyway, since compare diverges, we get

  max x y = compare `cast` UnsafeCoerce (T->T->T) T

and now we try to eta-expand the RHS of max, and that is a stupid thing to do (and loops in CoreUtils.etaExpand.

The real bug is the unsound eta-reducion for compare, which I knew about (comments in Note [Add IdInfo back onto a let-bound Id] in SimplEnv).

It's a dark corner case, and I'm not going to fix it today.

[simonpj]

Actually this is fixed by the same patch as #1947, namely:

Tue Dec  4 14:58:03 GMT 2007  simonpj@microsoft.com
  * Make eta reduction check more carefully for bottoms (fix Trac #1947)

I added a test case simpl-T1370 to make sure it stays fixed.

Simon