GHC Panic: loadArchive trying to load Nix's clang++ wrapper (darwin-aarch64)

It seems something has changed about archive loading in the 9.x series (at least 9.2.7 and 9.4.4) that makes GHC not like the shell environment that Nix provides on darwin-aarch64. As is the case strangely often for me, accelerate is the only package I've found that reproduces this. Please forgive the annoying repro, I'm still searching for a smaller one.

First, clone the loadarchive-bug branch of my fork of accelerate here. All I've done here is bump base in accelerate.cabal, and provided two minimal shell.nix files that give GHC 9.2.7 or GHC 9.4.4, plus Nixpkgs' current stdenv for darwin-aarch64. Then, all you have to do (with either the 927-shell.nix or 944-shell.nix) is:

$ cat shell.nix
let pkgsrc = builtins.fetchGit
    {
        url = "https://github.com/nixos/nixpkgs";
        ref = "master";
        rev = "38637c5c8fddf9c5b454918acf309c55e3809af1";
    };
in with import pkgsrc {};
let thisghc = haskell.packages.ghc927.ghcWithPackages (p: [p.cabal-install]);
in mkShell
{
    packages = [thisghc];
}
$ nix-shell --pure

[nix-shell:~/sources/accelerate]$ cabal clean

[nix-shell:~/sources/accelerate]$ cabal build
Resolving dependencies...
Build profile: -w ghc-9.2.7 -O1
In order, the following will be built (use -v for more details):
 - accelerate-1.3.0.0 (lib:accelerate) (first run)
[1 of 1] Compiling Main             ( /Users/twhitaker/sources/accelerate/dist-newstyle/build/aarch64-osx/ghc-9.2.7/accelerate-1.3.0.0/setup/setup.hs, /Users/twhitaker/sources/accelerate/dist-newstyle/build/aarch64-osx/ghc-9.2.7/accelerate-1.3.0.0/setup/Main.o )
Linking /Users/twhitaker/sources/accelerate/dist-newstyle/build/aarch64-osx/ghc-9.2.7/accelerate-1.3.0.0/setup/setup ...
Configuring accelerate-1.3.0.0...
Preprocessing library for accelerate-1.3.0.0..
Building library for accelerate-1.3.0.0..
[  1 of 109] Compiling Crypto.Hash.XKCP ( src/Crypto/Hash/XKCP.hs, /Users/twhitaker/sources/accelerate/dist-newstyle/build/aarch64-osx/ghc-9.2.7/accelerate-1.3.0.0/build/Crypto/Hash/XKCP.o, /Users/twhitaker/sources/accelerate/dist-newstyle/build/aarch64-osx/ghc-9.2.7/accelerate-1.3.0.0/build/Crypto/Hash/XKCP.dyn_o )
ghc: loadArchive: Neither an archive, nor a fat archive: `/nix/store/wqjp99m9d4dzi4ydwj87xr0z2dfygv9b-clang-wrapper-11.1.0/bin/clang++'
ghc: panic! (the 'impossible' happened)
  (GHC version 9.2.7:
        loadArchive "/nix/store/wqjp99m9d4dzi4ydwj87xr0z2dfygv9b-clang-wrapper-11.1.0/bin/clang++": failed

Please report this as a GHC bug:  https://www.haskell.org/ghc/reportabug

Initially I was surprised that compiling modules needs to load archives in the first place, but then I realized it's because of the Template Haskell in src/Crypto/Hash/XKCP.hs. I'm not sure why it's picking on clang++ in particular. I think it might have something to do with the fact that accelerate depends on double-conversion, which in turn depends on some C++ something or other.

Both #16590 (comment 195984) and #16063 seem related, but not exactly the same (and at least partially solved on the 9.x series).