## Summary Nightly bindist for Windows contains hard-coded path to Clang and other tools. Making it practically unusable. ## Steps to reproduce I encountered this issue when testing my library on GHA's Windows runner against a nightly GHC ([log here](https://github.com/minoki/haskell-simdy/actions/runs/31392795530/job/93468330043)). It is locally reproducible: ``` # I ran these commands on WSL2: $ curl -Lo archive.zip https://gitlab.haskell.org/ghc/ghc/-/jobs/2612371/artifacts/download?file_type=archive $ unzip archive.zip $ tar xf ghc-x86_64-windows-validate.tar.xz # and this one on Windows: > ghc-10.1.20260810-x86_64-unknown-mingw32\bin\ghc.exe Hello.hs [1 of 2] Compiling Main ( Hello.hs, Hello.o ) <no location info>: error: Assembler: could not execute: C:/GitLabRunner/builds/0/2612371/_build/bindist/ghc-10.1.20260810-x86_64-unknown-mingw32/mingw//bin/clang.exe ``` ## Expected behavior The path should be relative to `$topdir` or something. ## Environment * GHC version used: ghc-10.1.20260810 * Operating System: Windows * System Architecture: x86-64 ## Analysis by Claude <details> <summary>Not (yet) verified by human </summary> ### Windows bindists ship `default.target` with hard-coded CI toolchain paths #### Summary On `master`, the Windows binary distributions (including the nightly bindists that ghcup consumes) ship a `lib/targets/default.target` in which every toolchain `prgPath` is an **absolute path into the GitLab runner's build directory**, e.g. ``` C:/GitLab-Runner/builds/0/2603709/_build/bindist/ghc-10.1.20260803-x86_64-unknown-mingw32/mingw//bin/clang.exe ``` instead of the expected `$topdir/../mingw/bin/clang.exe`. Windows bindists are relocatable and are not `configure`d by the user — ghcup simply unpacks the tarball — so the installed compiler ends up pointing its C compiler, C++ compiler, CPP, Cmm CPP, linker, `ranlib`, `llc`, `opt`, `llvm-windres` and `llvm-dlltool` at paths that only ever existed on the CI machine. Anything that needs the toolchain (compiling a `.c` file, linking, `hsc2hs`, TH, …) fails. The `$tooldir` substitution that is supposed to rewrite those paths silently fails; the two root causes are described below. Both were introduced by 7d378476c3 (*"Read Toolchain.Target files rather than 'settings'"*, 2025-07-08), which is on `master` only. `ghc-9.14` and `ghc-9.12` do not contain `m4/subst_tooldir.m4` at all and are unaffected. #### Evidence `distrib/configure.ac.in` prints `default.target` right after applying the substitution, so the failure is visible in every Windows CI job log. From the nightly build of `10.1.20260803` (job 2603709, `https://gitlab.haskell.org/ghc/ghc/-/jobs/2603709/raw`): ``` sed: -e expression #1, char 2: extra characters after command ... config.status: creating default.target Applied tooldir substitution to default.target: Target { tgtArchOs = ArchOS {archOS_arch = ArchX86_64, archOS_OS = OSMinGW32} ... , tgtCCompiler = Cc {ccProgram = Program {prgPath = "C:/GitLab-Runner/builds/0/2603709/_build/bindist/ghc-10.1.20260803-x86_64-unknown-mingw32/mingw//bin/clang.exe", prgFlags = ["--target=x86_64-unknown-windows-gnu","--rtlib=compiler-rt","-D_UCRT","-IC:/GitLab-Runner/builds/0/2603709/_build/bindist/ghc-10.1.20260803-x86_64-unknown-mingw32/mingw//include","-Qunused-arguments"]}} , tgtCxxCompiler = Cxx {cxxProgram = Program {prgPath = "C:/GitLab-Runner/builds/0/2603709/.../mingw//bin/clang++.exe", ...}} , tgtCCompilerLink = CcLink { ccLinkProgram = Program {prgPath = "C:/GitLab-Runner/builds/0/2603709/.../mingw//bin/clang.exe", ...} ... ``` The `sed` error and the unchanged output show that `SUBST_TOOLDIR` did nothing. `hadrian/bindist/Makefile` then just copies the file into place, so this is exactly what lands in the tarball: ```make lib/targets/default.target : config.mk default.target @rm -f $@ @echo "Copying the bindist-configured default.target to lib/targets/default.target" cp default.target $@ ``` #### Root cause 1: `sed` argument order in `SUBST_TOOLDIR` `m4/subst_tooldir.m4:44` ```sh sed -i.bkp $1 's%'"$mingw_prefix"'%'"$mingw_install_prefix"'%g' ``` With `-i`, `sed` takes the *script* first and the *files* afterwards. Here the filename is passed as the script, so `sed` tries to parse `default.target` as a command list, fails with ``` sed: -e expression #1, char 2: extra characters after command ``` and exits 4 without touching the file. `configure` does not run under `set -e`, so the failure is not fatal and the build proceeds. Up to `ghc-9.14`, `SUBST_TOOLDIR` took a *shell variable name*, not a filename: ```sh $1=`echo "$$1" | sed 's%'"$mingw_prefix"'%'"$mingw_install_prefix"'%g'` ``` and was invoked as `SUBST_TOOLDIR([SettingsCCompilerCommand])`. When 7d378476c3 changed it to operate on `default.target` directly, `$1` stayed in the position it had before, which is now the script position. #### Root cause 2: `$$topdir` is no longer halved by `make` `distrib/configure.ac.in:106` ``` FP_SETUP_WINDOWS_TOOLCHAIN([$hardtop/mingw/], [\$\$topdir/../mingw/]) ``` `\$\$topdir` yields a literal `$$topdir` (two dollars) in `mingw_install_prefix`. That was correct while the paths flowed through `hadrian/bindist/Makefile`: ```make @echo ',("C compiler command", "$(SettingsCCompilerCommand)")' >> $@ ``` where `make` expands `$$` to `$`. Since 7d378476c3 the target file is `cp`'d verbatim, so nothing halves the `$$` any more. `expandPathVar` (`libraries/ghc-boot/GHC/BaseDir.hs:41`) only recognises `$topdir`: ```haskell expandPathVar var value str | Just str' <- stripPrefix ('$':var) str , maybe True isPathSeparator (listToMaybe str') = value ++ expandPathVar var value str' expandPathVar var value (x:xs) = x : expandPathVar var value xs expandPathVar _ _ [] = [] ``` so `$$topdir/../mingw/bin/clang.exe` would expand to `$C:/ghcup/ghc/<ver>/lib/../mingw/bin/clang.exe` — note the leading `$`. In other words, fixing root cause 1 alone is not enough; it would replace one broken path with another. #### Proposed fix ```diff --- a/m4/subst_tooldir.m4 +++ b/m4/subst_tooldir.m4 @@ -41,5 +41,5 @@ AC_DEFUN([SUBST_TOOLDIR], [ - sed -i.bkp $1 's%'"$mingw_prefix"'%'"$mingw_install_prefix"'%g' + sed -i.bkp 's%'"$mingw_prefix"'%'"$mingw_install_prefix"'%g' $1 ]) --- a/distrib/configure.ac.in +++ b/distrib/configure.ac.in @@ -103,7 +103,7 @@ if test "$HostOS" = "mingw32" -a "$EnableDistroToolchain" = "NO"; then - FP_SETUP_WINDOWS_TOOLCHAIN([$hardtop/mingw/], [\$\$topdir/../mingw/]) + FP_SETUP_WINDOWS_TOOLCHAIN([$hardtop/mingw], [\$topdir/../mingw]) fi ``` The trailing slash is dropped as well: `FP_SETUP_WINDOWS_TOOLCHAIN` builds `mingwbin="$mingw_prefix/bin/"` (`m4/fp_setup_windows_toolchain.m4:92`), which is why the current paths contain `mingw//bin`. `configure.ac:313` already passes `$hardtop/inplace/mingw` without a trailing slash, so this also makes the two call sites consistent. I verified the fix by extracting the equivalent m4 into a standalone `configure.ac`, running `autoconf`, and applying it to the exact `default.target` emitted by job 2603709. Result: ``` , tgtCCompiler = Cc {ccProgram = Program {prgPath = "$topdir/../mingw/bin/clang.exe", prgFlags = ["-I$topdir/../mingw/include", ...]}} ``` which `expandTopDir` resolves correctly at GHC startup. #### Suggested follow-up `SUBST_TOOLDIR` failing is invisible today. Since the resulting bindist is unusable, it would be worth making the substitution failure fatal, e.g. ```sh sed -i.bkp 's%'"$mingw_prefix"'%'"$mingw_install_prefix"'%g' $1 \ || AC_MSG_ERROR([failed to apply tooldir substitution to $1]) ``` and/or having the bindist `configure` assert that no occurrence of `$hardtop` remains in `default.target`. #### Not affected * `--enable-distro-toolchain` bindists — the basename rewriting loop in `distrib/configure.ac.in` is a separate, working code path. * `ghc-9.14` / `ghc-9.12` and earlier — `m4/subst_tooldir.m4` does not exist there and the settings file is still produced by `hadrian/bindist/Makefile` from `$(Settings*)` variables. </details>