Obscure loop in interaction between arity and case-of-bottom

John Meacham discovered that this program makes the simplifier loop:

newtype T = V T deriving(Eq,Ord)

It's a silly program, but it shouldn't kill GHC! I investigated and the reason is because we get this:

  rec { compare :: T -> T -> Ordering
        compare [arity 2] = compare }
   
  max :: T -> T -> T
  max x y = case compare x y of { GT -> x; other -> y }

The real bug is that compare has been eta-reduced, but still has arity 2. That's wrong. Anyway, since compare diverges, we get

  max x y = compare `cast` UnsafeCoerce (T->T->T) T

and now we try to eta-expand the RHS of max, and that is a stupid thing to do (and loops in CoreUtils.etaExpand.

The real bug is the unsound eta-reducion for compare, which I knew about (comments in Note [Add IdInfo back onto a let-bound Id] in SimplEnv).

It's a dark corner case, and I'm not going to fix it today.

Trac metadata

Trac field	Value
Version	6.6.1
Type	Bug
TypeOfFailure	OtherFailure
Priority	normal
Resolution	Unresolved
Component	Compiler
Test case
Differential revisions
BlockedBy
Related
Blocking
CC
Operating system	Unknown
Architecture	Unknown