GHC internal error when running haddock with -fprefer-byte-code

When running haddock for the code built with -fprefer-byte-code, the haddock crashes with GHC internal error:

<no location info>: error:
    panic! (the 'impossible' happened)
  GHC version 9.6.1:
        Unexpected DT_Dyn state when writing simple interface
  CallStack (from HasCallStack):
    panic, called at compiler/GHC/Driver/Main.hs:1267:29 in ghc:GHC.Driver.Main

Please report this as a GHC bug:  https://www.haskell.org/ghc/reportabug

[...]

haddock: Cannot typecheck modules

This seems to be the same issue with --dynamic-too.