Representation polymorphism check doesn't reduce type family
Here is (a variant on) a program from @AndreasK which runs afoul of representation polymorphism restrictions, in my opinion needlessly:
```hs
{-# LANGUAGE MagicHash #-}
{-# LANGUAGE TypeFamilies #-}
{-# LANGUAGE ViewPatterns #-}
{-# LANGUAGE TypeFamilyDependencies #-}
{-# LANGUAGE PatternSynonyms #-}
{-# LANGUAGE UnboxedTuples #-}
{-# LANGUAGE KindSignatures #-}
{-# LANGUAGE FunctionalDependencies #-}
{-# LANGUAGE UnliftedDatatypes #-}
{-# LANGUAGE UnliftedNewtypes #-}
module WorkerWrapper where
import GHC.Exts
import Data.Proxy
import Data.Kind
type UnboxedRep :: TYPE r -> RuntimeRep
type family UnboxedRep b
type instance UnboxedRep Int = IntRep
type HasUnboxed :: forall {r}. TYPE r -> Constraint
class HasUnboxed b where
type Unboxed b :: TYPE (UnboxedRep b)
box :: Unboxed b -> b
unbox :: b -> Unboxed b
instance HasUnboxed Int where
type Unboxed Int = Int#
box x = I# x
unbox (I# x) = x
ok :: Int# -> Int
ok = box
bad :: Int# -> Int
bad x = box x
```
The problem is in the application `box x`:
```
• The argument ‘x’ of ‘box’
does not have a fixed runtime representation.
Its type is:
Unboxed Int :: TYPE (UnboxedRep Int)
```
Here `x :: Int# :: TYPE IntRep` has a fixed runtime representation, but GHC instead works with the type family `UnboxedRep Int` which it fails to reduce and thus considers non-concrete.
I think the problem is simply that we have a [phase 1 FRR check](https://gitlab.haskell.org/ghc/ghc/-/wikis/FixedRuntimeRep#phase-1) somewhere in the compiler where we should have a phase 2 FRR check. My old MR !7988 was trying to move the remaining phase 1 checks to phase 2 checks but I got bogged down with casts inside data constructor return types. It shouldn't be too hard to identify which FRR check is causing the issue and at least migrating that one to phase 2.
issue