GHC does not reject overlapping instances

This example was mentioned on the mailing list by Daniel Díaz at https://mail.haskell.org/pipermail/haskell-cafe/2020-October/132815.html

GHC does not reject this program:

{-# LANGUAGE FlexibleInstances #-}
newtype Foo = Foo Int
instance {-# OVERLAPS #-} Show (Maybe Foo) where
  show _ = "foo"

newtype W a = W (Maybe a)
instance Show a => Show (W a) where
  show (W a) = show a -- This should not be allowed. Which `Show (Maybe a)` should be used?

foo :: Show a => W a -> String
foo a = show a

main :: IO ()
main = pure ()

There's two overlapping instances here:

instance Show a => Show (Maybe a) -- Defined in ‘GHC.Show’
instance [overlap ok] Show (Maybe Foo) -- Defined at ex1.hs:3:27

If we tried to solve for Show (Maybe a) a different way, GHC complains as it should:

foo' :: Show a => Maybe a -> String
foo' a = show a

ex1.hs:6:8: error:
    • Overlapping instances for Show (Maybe a)
        arising from a use of ‘show’
      Matching instances:
        instance Show a => Show (Maybe a) -- Defined in ‘GHC.Show’
        instance [overlap ok] Show (Maybe Foo) -- Defined at ex1.hs:3:27
      (The choice depends on the instantiation of ‘a’
       To pick the first instance above, use IncoherentInstances
       when compiling the other instance declarations)
    • In the expression: show
      In an equation for ‘foo'’: foo' = show
  |
6 | foo' = show

It seems to me, that one of two things must be true:

• The algorithm described in https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#overlapping-instances is incorrect and the algorithm behaves differently for instance selection inside instances.
• GHC makes an incoherent instance selection here