9.2.2 bootstrap fails on macOS 12

changed the description

assigned to @bgamari

on Linux attempting to build 9.2.2 from source

$ cat /etc/slackware-version
$ Slackware 15.0
$

extract ghc-9.2.2-x86_64-deb10-linux.tar.xz to /tmp/ghc-9.2.2 and cd to /tmp/ghc-9.2.2
run './configure --prefix=/tmp/ghc-9.2.2-bootstrap/usr' and 'make install'

run 'rm -r /tmp/ghc-9.2.2'

run 'export PATH=${PATH}:/tmp/ghc-9.2.2-bootstrap/usr/bin'
extract ghc-9.2.2-src.tar.xz to /tmp/ghc-9.2.2
cd to /tmp/ghc-9.2.2
run './configure --prefix=/usr'

output as below (doesn't occur using ghc-9.2.1-x86_64-deb10-linux.tar.xz + ghc-9.2.2-src.tar.xz)

<command line>: error: directory portion of "libraries/template-haskell/dist-boot/build/Language/Haskell/TH.o" does not exist (used with "-o" option.)
compiler/ghc.mk:287: compiler/stage1/build/.depend-v.haskell: No such file or directory
make[1]: *** [libraries/template-haskell/ghc.mk:3: libraries/template-haskell/dist-boot/build/Language/Haskell/TH.o] Error 1
make: *** [Makefile:126: all] Error 2

using ghc-9.2.2-x86_64-deb10-linux.tar.xz (i.e. /tmp/ghc-9.2.2-bootstrap)

$ ls ghc-9.2.2/libraries/template-haskell/dist-boot/build
autogen
$ ls -d ghc-9.2.2/compiler/stage1/build/.[^.]*
ghc-9.2.2/compiler/stage1/build/.depend-v.c_asm
$

using ghc-9.2.1-x86_64-deb10-linux.tar.xz (i.e. /tmp/ghc-9.2.1-bootstrap)

$ ls ghc-9.2.2/libraries/template-haskell/dist-boot/build/Language/Haskell/*.o
ghc-9.2.2/libraries/template-haskell/dist-boot/build/Language/Haskell/TH.o
$ ls -d ghc-9.2.2/compiler/stage1/build/.[^.]*
ghc-9.2.2/compiler/stage1/build/.depend-v.c_asm
ghc-9.2.2/compiler/stage1/build/.depend-v.haskell
ghc-9.2.2/compiler/stage1/build/.depend-v.haskell.tmp
ghc-9.2.2/compiler/stage1/build/.depend-v.haskell.tmp2
$

mentioned in issue #21320 (closed)

marked #21320 (closed) as a duplicate of this issue

marked this issue as related to #21320 (closed)

We are also seeing this in !7867 (closed)

It appears that in principle the make build system is responsible for creating the output directory. In particular, I believe the relevant rule is this in rules/build-dependencies.mk:

$$($1_$2_depfile_haskell) : $$($1_$2_HS_SRCS) $$($1_$2_HS_BOOT_SRCS) $$$$($1_$2_HC_MK_DEPEND_DEP) | $$$$(dir $$$$@)/.
	$$(call removeFiles,$$@.tmp)
ifneq "$$($1_$2_HS_SRCS)" ""
	"$$($1_$2_HC_MK_DEPEND)" -M \
	    $$($1_$2_$$(firstword $$($1_$2_WAYS))_MOST_DIR_HC_OPTS) \
	    $$($1_$2_MKDEPENDHS_FLAGS) \
	    $$($1_$2_HS_SRCS)
endif
	echo "$1_$2_depfile_haskell_EXISTS = YES" >> $$@.tmp
ifneq "$$($1_$2_SLASH_MODS)" ""
	for dir in $$(sort $$(foreach mod,$$($1_$2_SLASH_MODS),$1/$2/build/$$(dir $$(mod)))); do \
		if test ! -d $$$$dir; then mkdir -p $$$$dir; fi \
	done
endif

The problem here appears to be that make is building template-haskell's object code without first executing its depfile rule. The dependency on the objects appears to be due to utils/deriveConstants/dist/build/Main.o, which seems to for some reason depend upon libraries/template-haskell/dist-boot/build/libHStemplate-haskell-2.18.0.0.a.

This is very strange since make -d suggests that make does attempt to build libraries/template-haskell/dist-boot/build/.depend-v.haskell. The rule evaluation does not fail, yet somehow does not create the target file. That is, with this patch,

diff --git a/rules/build-dependencies.mk b/rules/build-dependencies.mk
index cc438c80eb..3056b44641 100644
--- a/rules/build-dependencies.mk
+++ b/rules/build-dependencies.mk
@@ -34,6 +34,7 @@ ifneq "$$(NO_GENERATED_MAKEFILE_RULES)" "YES"
 $$($1_$2_depfile_haskell) : $$(includes_$3_H_CONFIG) $$(includes_$3_H_PLATFORM)
 
 $$($1_$2_depfile_haskell) : $$($1_$2_HS_SRCS) $$($1_$2_HS_BOOT_SRCS) $$$$($1_$2_HC_MK_DEPEND_DEP) | $$$$(dir $$$$@)/.
+       echo "===== Building dependencies $1 $2 $$($1_$2_SLASH_MODS)"
        $$(call removeFiles,$$@.tmp)
 ifneq "$$($1_$2_HS_SRCS)" ""
        "$$($1_$2_HC_MK_DEPEND)" -M \
diff --git a/rules/include-dependencies.mk b/rules/include-dependencies.mk
index 7d4c38b73b..091ebadaa1 100644
--- a/rules/include-dependencies.mk
+++ b/rules/include-dependencies.mk
@@ -31,6 +31,9 @@ ifeq "$$($1_$2_INCLUDE_DEPFILES)" "YES"
 ifneq "$$(strip $$($1_$2_HS_SRCS) $$($1_$2_HS_BOOT_SRCS))" ""
 ifneq "$$(NO_STAGE$3_DEPS)" "YES"
 include $$($1_$2_depfile_haskell)
+$$(warning haskell deps: $$(shell cat $$($1_$2_depfile_haskell)))
+else
+$$(warning not building haskell dependencies in $1)
 endif
 endif
 include $$($1_$2_depfile_c_asm)

I see:

$ make DEBUG=YES TRACE=YES -d
...
libraries/template-haskell/ghc.mk:3: dependencies(libraries/template-haskell,dist-boot,0)
libraries/template-haskell/ghc.mk:3: build-dependencies(libraries/template-haskell,dist-boot,0)
libraries/template-haskell/ghc.mk:3: include-dependencies(libraries/template-haskell,dist-boot,0)
Reading makefile 'libraries/template-haskell/dist-boot/build/.depend-v.haskell' (search path) (no ~ expansion)...
cat: libraries/template-haskell/dist-boot/build/.depend-v.haskell: No such file or directory
libraries/template-haskell/ghc.mk:3: haskell deps:
Reading makefile 'libraries/template-haskell/dist-boot/build/.depend-v.c_asm' (search path) (no ~ expansion)...
libraries/template-haskell/ghc.mk:3: build-package-way(libraries/template-haskell,dist-boot,v)

Intriguingly, the `==== Building dependencies" string appears nowhere in the log.

It appears that the problem is that the libraries/template-haskell/dist-boot/build/.depend-v.haskell rule itself is somehow depending upon libraries/template-haskell/dist-boot/build/Language/Haskell/TH.o as asking make to build only the libraries/template-haskell/dist-boot/build/.depend-v.haskell target reproduces the error.

The dependency chain appears to be:

libraries/template-haskell/dist-boot/build/.depend-v.haskell
compiler/stage1/build/.depend-v.haskell
compiler/stage1/build/GHC/Platform/Constants.hs
inplace/bin/deriveConstants
inplace/lib/bin/deriveConstants
utils/deriveConstants/dist/build/tmp/deriveConstants
utils/deriveConstants/dist/build/Main.o
libraries/template-haskell/dist-boot/build/libHStemplate-haskell-2.18.0.0.a
libraries/template-haskell/dist-boot/build/Language/Haskell/TH.o

Indeed somehow deriveConstants is appearing in utils/deriveConstants_dist_TRANSITIVE_DEP_NAMES:

utils/deriveConstants_dist_TRANSITIVE_DEP_NAMES = process directory unix time containers bytestring template-haskell pretty ghc-boot-th filepath deepseq array base ghc-bignum ghc-prim rts

It actually seems that template-haskell appears in the TRANSITIVE_DEP_NAMES of nearly all boot packages:

Dependencies

$ grep -R 'DEP_NAMES.*template-haskell' .
./libraries/ghc-boot/dist-boot/package-data.mk:libraries/ghc-boot_dist-boot_TRANSITIVE_DEP_NAMES = ghc-boot-th directory unix time filepath binary containers bytestring template-haskell pretty ghc-boot-th deepseq array base ghc-bignum ghc-prim rts
./libraries/exceptions/dist-boot/package-data.mk:libraries/exceptions_dist-boot_DEP_NAMES = base mtl stm template-haskell transformers
./libraries/exceptions/dist-boot/package-data.mk:libraries/exceptions_dist-boot_TRANSITIVE_DEP_NAMES = template-haskell stm pretty mtl transformers ghc-boot-th filepath deepseq array base ghc-bignum ghc-prim rts
./libraries/hpc/dist-boot/package-data.mk:libraries/hpc_dist-boot_TRANSITIVE_DEP_NAMES = directory unix time filepath containers bytestring template-haskell pretty ghc-boot-th deepseq array base ghc-bignum ghc-prim rts
./libraries/Cabal/Cabal/dist-boot/package-data.mk:libraries/Cabal/Cabal_dist-boot_TRANSITIVE_DEP_NAMES = process Cabal-syntax parsec text mtl transformers directory unix time binary containers bytestring template-haskell pretty ghc-boot-th filepath deepseq array base ghc-bignum ghc-prim rts
./libraries/Cabal/Cabal-syntax/dist-boot/package-data.mk:libraries/Cabal/Cabal-syntax_dist-boot_TRANSITIVE_DEP_NAMES = parsec text mtl transformers directory unix time binary containers bytestring template-haskell pretty ghc-boot-th filepath deepseq array base ghc-bignum ghc-prim rts
./libraries/text/dist-boot/package-data.mk:libraries/text_dist-boot_DEP_NAMES = array base binary bytestring deepseq ghc-prim template-haskell
./libraries/text/dist-boot/package-data.mk:libraries/text_dist-boot_TRANSITIVE_DEP_NAMES = binary containers bytestring template-haskell pretty ghc-boot-th filepath deepseq array base ghc-bignum ghc-prim rts
./libraries/binary/dist-boot/package-data.mk:libraries/binary_dist-boot_TRANSITIVE_DEP_NAMES = containers bytestring template-haskell pretty ghc-boot-th deepseq array base ghc-bignum ghc-prim rts
./libraries/parsec/dist-boot/package-data.mk:libraries/parsec_dist-boot_TRANSITIVE_DEP_NAMES = text mtl transformers binary containers bytestring template-haskell pretty ghc-boot-th filepath deepseq array base ghc-bignum ghc-prim rts
./libraries/ghci/dist-boot/package-data.mk:libraries/ghci_dist-boot_DEP_NAMES = array base binary bytestring containers deepseq filepath ghc-boot ghc-heap ghc-prim rts template-haskell transformers unix
./libraries/ghci/dist-boot/package-data.mk:libraries/ghci_dist-boot_TRANSITIVE_DEP_NAMES = transformers ghc-heap ghc-boot directory unix time binary containers bytestring template-haskell pretty ghc-boot-th filepath deepseq array base ghc-bignum ghc-prim rts
./ghc/stage1/package-data.mk:ghc_stage1_TRANSITIVE_DEP_NAMES = ghc terminfo process hpc ghci ghc-heap ghc-boot exceptions stm mtl transformers directory unix time binary containers bytestring template-haskell pretty ghc-boot-th filepath deepseq array base ghc-bignum ghc-prim rts
./utils/hsc2hs/dist/package-data.mk:utils/hsc2hs_dist_TRANSITIVE_DEP_NAMES = process directory unix time containers bytestring template-haskell pretty ghc-boot-th filepath deepseq array base ghc-bignum ghc-prim rts
./utils/ghc-pkg/dist/package-data.mk:utils/ghc-pkg_dist_TRANSITIVE_DEP_NAMES = terminfo ghc-boot Cabal process Cabal-syntax parsec text mtl transformers directory unix time binary containers bytestring template-haskell pretty ghc-boot-th filepath deepseq array base ghc-bignum ghc-prim rts
./utils/deriveConstants/dist/package-data.mk:utils/deriveConstants_dist_TRANSITIVE_DEP_NAMES = process directory unix time containers bytestring template-haskell pretty ghc-boot-th filepath deepseq array base ghc-bignum ghc-prim rts
./compiler/stage1/package-data.mk:compiler_stage1_DEP_NAMES = array base binary bytestring containers deepseq directory exceptions filepath ghc-boot ghc-heap ghci hpc process stm template-haskell terminfo time transformers unix
./compiler/stage1/package-data.mk:compiler_stage1_TRANSITIVE_DEP_NAMES = terminfo process hpc ghci ghc-heap ghc-boot exceptions stm mtl transformers directory unix time binary containers bytestring template-haskell pretty ghc-boot-th filepath deepseq array base ghc-bignum ghc-prim rts

I'm beginning to suspect that this due to a change in Cabal's behavior. Somehow none of deriveConstants's immediate dependencies have a dependency on template-haskell yet template-haskell nevertheless ends up in the installedPkgs of LocalBuildInfo. Frankly, the fact that we are relying on installedPkgs to depend only the dependencies of the target package seems a bit suspicious.

It seems there is more to this than just a Cabal change. Specifically, I have attempted to work around the extra entries in installedPkgs by teaching ghc-cabal to construct a new InstalledPackageIndex which only contains the transitive dependencies of the install plan:

diff --git a/utils/ghc-cabal/Main.hs b/utils/ghc-cabal/Main.hs
index 20ee539e0a..f4b1b8b363 100644
--- a/utils/ghc-cabal/Main.hs
+++ b/utils/ghc-cabal/Main.hs
@@ -34,6 +34,7 @@ import Distribution.Utils.Path (getSymbolicPath)
 import Control.Exception (bracket)
 import Control.Monad
 import Control.Applicative ((<|>))
+import Data.Foldable as Foldable
 import Data.List (nub, intercalate, isPrefixOf, isSuffixOf)
 import Data.Maybe
 import Data.Char (isSpace)
@@ -341,7 +342,17 @@ generate directory distdir config_args
           -- XXX Another Just...
           Just ghcProg = lookupProgram ghcProgram (withPrograms lbi)
 
-          dep_pkgs = PackageIndex.topologicalOrder (packageHacks (installedPkgs lbi))
+          dep_pkgs = PackageIndex.topologicalOrder (packageHacks dep_pkg_idx)
+          dep_pkg_idx :: PackageIndex.InstalledPackageIndex
+          dep_pkg_idx = PackageIndex.fromList $
+            [ ipi
+            | clbi <- Foldable.toList (componentGraph lbi)
+            , Just ipi <- pure $ PackageIndex.lookupUnitId (installedPkgs lbi) (componentUnitId clbi)
+            ] ++
+            [ ipi
+            | (_, ipis) <- PackageIndex.lookupPackageName (installedPkgs lbi) (mkPackageName "rts")
+            , ipi <- ipis
+            ]
           forDeps f = concatMap f dep_pkgs
 
           -- copied from Distribution.Simple.PreProcess.ppHsc2Hs
`@@ -425,6 +436,7 @@ generate directory distdir config_args
           injectDistInstall x | takeBaseName x == "include" = [x, takeDirectory x ++ "/dist-install/build/" ++ takeBaseName x]
           injectDistInstall x = [x]
 `
+      print $ map (display . Installed.sourcePackageId) $ PackageIndex.allPackages (installedPkgs lbi)
       -- See Note [Msys2 path translation bug].
       wrappedIncludeDirs <- wrap $ map normalise $ concatMap injectDistInstall $ forDeps Installed.includeDirs

While doing this eliminates the unexpected transitive dependencies from package-data.mk. However, the boot GHC still appears to be attempting to link against template-haskell:

$ "/opt/ghc/9.2.2/bin/ghc" -o utils/deriveConstants/dist/build/tmp/deriveConstants -hisuf hi -osuf  o -hcsuf hc -static  -O0 -H64m -Wall   -package-db libraries/bootstrapping.conf  -hide-all-packages -package-env - -i -iutils/deriveConstants/. -iutils/deriveConstants/dist/build -Iutils/deriveConstants/dist/build -iutils/deriveConstants/dist/build/deriveConstants/autogen -Iutils/deriveConstants/dist/build/deriveConstants/autogen   -optP-include -optPutils/deriveConstants/dist/build/deriveConstants/autogen/cabal_macros.h -package-id base-4.16.1.0 -package-id containers-0.6.5.1 -package-id directory-1.3.6.2 -package-id filepath-1.4.2.2 -package-id process-1.6.13.2 -XHaskell2010  -no-user-package-db -rtsopts        -outputdir utils/deriveConstants/dist/build    -static  -O0 -H64m -Wall   -package-db libraries/bootstrapping.conf  -hide-all-packages -package-env - -i -iutils/deriveConstants/. -iutils/deriveConstants/dist/build -Iutils/deriveConstants/dist/build -iutils/deriveConstants/dist/build/deriveConstants/autogen -Iutils/deriveConstants/dist/build/deriveConstants/autogen   -optP-include -optPutils/deriveConstants/dist/build/deriveConstants/autogen/cabal_macros.h -package-id base-4.16.1.0 -package-id containers-0.6.5.1 -package-id directory-1.3.6.2 -package-id filepath-1.4.2.2 -package-id process-1.6.13.2 -XHaskell2010  -no-user-package-db -rtsopts        utils/deriveConstants/dist/build/Main.o     
/usr/bin/ld.gold: error: cannot find -lHStemplate-haskell-2.18.0.0
/usr/bin/ld.gold: error: cannot find -lHSghc-boot-th-9.3
collect2: error: ld returned 1 exit status
`gcc' failed in phase `Linker'. (Exit code: 1)

Dumping the bootstrap package database confirms that none of the direct dependencies requested in the -package-id flags depend upon template-haskell. Specifically, the only packages that depend upon template-haskell in the bootstrap package database are ghc, ghci, and exceptions.

My suspicion is that this dependency is arising due to GHC.Unit.State.implicitPackageDeps, which introduces a dependency on template-haskell if TemplateHaskellQuotes is enabled. This behavior was introduced in 6cc80766.

It turns out that this commit is not in %9.2.2. However, there is another place where a template-haskell dependency may be introduced: GHC.HsToCore.Usage.mkDependencies.

This indeed appears to be a regression between %9.2.1 and %9.2.2:

$ ghc-9.2.1 utils/deriveConstants/Main.hs -v3 |& grep template-haskell
wired-in package template-haskell mapped to template-haskell-2.18.0.0
wired-in package template-haskell mapped to template-haskell-2.18.0.0
$ ghc-9.2.2 utils/deriveConstants/Main.hs -v3 |& grep template-haskell
wired-in package template-haskell mapped to template-haskell-2.18.0.0
wired-in package template-haskell mapped to template-haskell-2.18.0.0
/nix/store/8pbwywcj6vbswz7xmy2dh716x8blgh8w-gcc-wrapper-9.3.0/bin/cc '-Wl,--hash-size=31' -Wl,--reduce-memory-overheads -Wl,--no-as-needed -o utils/deriveConstants/Main -no-pie -fno-PIC -Wl,--gc-sections utils/deriveConstants/Main.o -L/nix/store/vywjxkhlmr31fidyfn7qihfkp46nniil-ghc-9.2.2/lib/ghc-9.2.2/lib/../lib/x86_64-linux-ghc-9.2.2/process-1.6.13.2 -L/nix/store/vywjxkhlmr31fidyfn7qihfkp46nniil-ghc-9.2.2/lib/ghc-9.2.2/lib/../lib/x86_64-linux-ghc-9.2.2/directory-1.3.6.2 -L/nix/store/vywjxkhlmr31fidyfn7qihfkp46nniil-ghc-9.2.2/lib/ghc-9.2.2/lib/../lib/x86_64-linux-ghc-9.2.2/unix-2.7.2.2 -L/nix/store/vywjxkhlmr31fidyfn7qihfkp46nniil-ghc-9.2.2/lib/ghc-9.2.2/lib/../lib/x86_64-linux-ghc-9.2.2/bytestring-0.11.3.0 -L/nix/store/vywjxkhlmr31fidyfn7qihfkp46nniil-ghc-9.2.2/lib/ghc-9.2.2/lib/../lib/x86_64-linux-ghc-9.2.2/template-haskell-2.18.0.0 -L/nix/store/vywjxkhlmr31fidyfn7qihfkp46nniil-ghc-9.2.2/lib/ghc-9.2.2/lib/../lib/x86_64-linux-ghc-9.2.2/pretty-1.1.3.6 -L/nix/store/vywjxkhlmr31fidyfn7qihfkp46nniil-ghc-9.2.2/lib/ghc-9.2.2/lib/../lib/x86_64-linux-ghc-9.2.2/ghc-boot-th-9.2.2 -L/nix/store/vywjxkhlmr31fidyfn7qihfkp46nniil-ghc-9.2.2/lib/ghc-9.2.2/lib/../lib/x86_64-linux-ghc-9.2.2/time-1.11.1.1 -L/nix/store/vywjxkhlmr31fidyfn7qihfkp46nniil-ghc-9.2.2/lib/ghc-9.2.2/lib/../lib/x86_64-linux-ghc-9.2.2/filepath-1.4.2.2 -L/nix/store/vywjxkhlmr31fidyfn7qihfkp46nniil-ghc-9.2.2/lib/ghc-9.2.2/lib/../lib/x86_64-linux-ghc-9.2.2/containers-0.6.5.1 -L/nix/store/vywjxkhlmr31fidyfn7qihfkp46nniil-ghc-9.2.2/lib/ghc-9.2.2/lib/../lib/x86_64-linux-ghc-9.2.2/deepseq-1.4.6.1 -L/nix/store/vywjxkhlmr31fidyfn7qihfkp46nniil-ghc-9.2.2/lib/ghc-9.2.2/lib/../lib/x86_64-linux-ghc-9.2.2/array-0.5.4.0 -L/nix/store/vywjxkhlmr31fidyfn7qihfkp46nniil-ghc-9.2.2/lib/ghc-9.2.2/lib/../lib/x86_64-linux-ghc-9.2.2/base-4.16.1.0 -L/nix/store/vywjxkhlmr31fidyfn7qihfkp46nniil-ghc-9.2.2/lib/ghc-9.2.2/lib/../lib/x86_64-linux-ghc-9.2.2/ghc-bignum-1.2 -L/nix/store/fphpbj8jpyibz0l2xspidg6s7zm7xyb5-gmp-6.2.0/lib -L/nix/store/vywjxkhlmr31fidyfn7qihfkp46nniil-ghc-9.2.2/lib/ghc-9.2.2/lib/../lib/x86_64-linux-ghc-9.2.2/ghc-prim-0.8.0 -L/nix/store/vywjxkhlmr31fidyfn7qihfkp46nniil-ghc-9.2.2/lib/ghc-9.2.2/lib/../lib/x86_64-linux-ghc-9.2.2/rts-1.0.2 /tmp/ghc2591472_0/ghc_5.o /tmp/ghc2591472_0/ghc_8.o -Wl,-u,hs_atomic_add64 -Wl,-u,hs_atomic_sub64 -Wl,-u,hs_atomic_and64 -Wl,-u,hs_atomic_nand64 -Wl,-u,hs_atomic_or64 -Wl,-u,hs_atomic_xor64 -Wl,-u,hs_cmpxchg64 -Wl,-u,hs_atomicread64 -Wl,-u,hs_atomicwrite64 -Wl,-u,base_GHCziTopHandler_runIO_closure -Wl,-u,base_GHCziTopHandler_runNonIO_closure -Wl,-u,ghczmprim_GHCziTuple_Z0T_closure -Wl,-u,ghczmprim_GHCziTypes_True_closure -Wl,-u,ghczmprim_GHCziTypes_False_closure -Wl,-u,base_GHCziPack_unpackCString_closure -Wl,-u,base_GHCziWeak_runFinalizzerBatch_closure -Wl,-u,base_GHCziIOziException_stackOverflow_closure -Wl,-u,base_GHCziIOziException_heapOverflow_closure -Wl,-u,base_GHCziIOziException_allocationLimitExceeded_closure -Wl,-u,base_GHCziIOziException_blockedIndefinitelyOnMVar_closure -Wl,-u,base_GHCziIOziException_blockedIndefinitelyOnSTM_closure -Wl,-u,base_GHCziIOziException_cannotCompactFunction_closure -Wl,-u,base_GHCziIOziException_cannotCompactPinned_closure -Wl,-u,base_GHCziIOziException_cannotCompactMutable_closure -Wl,-u,base_GHCziIOPort_doubleReadException_closure -Wl,-u,base_ControlziExceptionziBase_nonTermination_closure -Wl,-u,base_ControlziExceptionziBase_nestedAtomically_closure -Wl,-u,base_GHCziEventziThread_blockedOnBadFD_closure -Wl,-u,base_GHCziConcziSync_runSparks_closure -Wl,-u,base_GHCziConcziIO_ensureIOManagerIsRunning_closure -Wl,-u,base_GHCziConcziIO_interruptIOManager_closure -Wl,-u,base_GHCziConcziIO_ioManagerCapabilitiesChanged_closure -Wl,-u,base_GHCziConcziSignal_runHandlersPtr_closure -Wl,-u,base_GHCziTopHandler_flushStdHandles_closure -Wl,-u,base_GHCziTopHandler_runMainIO_closure -Wl,-u,ghczmprim_GHCziTypes_Czh_con_info -Wl,-u,ghczmprim_GHCziTypes_Izh_con_info -Wl,-u,ghczmprim_GHCziTypes_Fzh_con_info -Wl,-u,ghczmprim_GHCziTypes_Dzh_con_info -Wl,-u,ghczmprim_GHCziTypes_Wzh_con_info -Wl,-u,base_GHCziPtr_Ptr_con_info -Wl,-u,base_GHCziPtr_FunPtr_con_info -Wl,-u,base_GHCziInt_I8zh_con_info -Wl,-u,base_GHCziInt_I16zh_con_info -Wl,-u,base_GHCziInt_I32zh_con_info -Wl,-u,base_GHCziInt_I64zh_con_info -Wl,-u,base_GHCziWord_W8zh_con_info -Wl,-u,base_GHCziWord_W16zh_con_info -Wl,-u,base_GHCziWord_W32zh_con_info -Wl,-u,base_GHCziWord_W64zh_con_info -Wl,-u,base_GHCziStable_StablePtr_con_info -Wl,-u,hs_atomic_add8 -Wl,-u,hs_atomic_add16 -Wl,-u,hs_atomic_add32 -Wl,-u,hs_atomic_sub8 -Wl,-u,hs_atomic_sub16 -Wl,-u,hs_atomic_sub32 -Wl,-u,hs_atomic_and8 -Wl,-u,hs_atomic_and16 -Wl,-u,hs_atomic_and32 -Wl,-u,hs_atomic_nand8 -Wl,-u,hs_atomic_nand16 -Wl,-u,hs_atomic_nand32 -Wl,-u,hs_atomic_or8 -Wl,-u,hs_atomic_or16 -Wl,-u,hs_atomic_or32 -Wl,-u,hs_atomic_xor8 -Wl,-u,hs_atomic_xor16 -Wl,-u,hs_atomic_xor32 -Wl,-u,hs_cmpxchg8 -Wl,-u,hs_cmpxchg16 -Wl,-u,hs_cmpxchg32 -Wl,-u,hs_xchg8 -Wl,-u,hs_xchg16 -Wl,-u,hs_xchg32 -Wl,-u,hs_xchg64 -Wl,-u,hs_atomicread8 -Wl,-u,hs_atomicread16 -Wl,-u,hs_atomicread32 -Wl,-u,hs_atomicwrite8 -Wl,-u,hs_atomicwrite16 -Wl,-u,hs_atomicwrite32 -lHSprocess-1.6.13.2 -lHSdirectory-1.3.6.2 -lHSunix-2.7.2.2 -lHSbytestring-0.11.3.0 -lHStemplate-haskell-2.18.0.0 -lHSpretty-1.1.3.6 -lHSghc-boot-th-9.2.2 -lHStime-1.11.1.1 -lHSfilepath-1.4.2.2 -lHScontainers-0.6.5.1 -lHSdeepseq-1.4.6.1 -lHSarray-0.5.4.0 -lHSbase-4.16.1.0 -lHSghc-bignum-1.2 -lHSghc-prim-0.8.0 -lHSrts-1.0.2 -lrt -lutil -ldl -lpthread -lgmp -lc -lm -lm -lrt -ldl -lffi -lpthread

The difference between 9.2.1 and 9.2.2 is that bytestring depends on template-haskell with 9.2.2 I believe.

Indeed it looks like @mpickering is right; since 8d87975e deriveConstants depends upon directory, which depends upon unix, which depends upon bytestring, which depends upon template-haskell. I hadn't noticed this earlier as I was only looking at the stage0 package database, which didn't have bytestring installed in it yet.

It's not immediately clear to me how to fix this. In principle deriveConstants should be linked against the bootstrap compiler's directory, unix, bytestring, and template-haskell libraries. However, convincing the make build system to do this is quite tricky. The closest thing that we have to this is the build rule for ghc-cabal itself, which invokes the bootstrap compiler directly.

I also hit this trying to build at ghc-9.5-start on OpenBSD.

For better or worse I have my doubts that we will fix this issue before make is completely removed. As of 9.4 all GHC binary distributions will be produced by Hadrian and I do hope that we can drop make before %9.6.1

marked this issue as related to #21484 (closed)

mentioned in issue #21484 (closed)

marked #21484 (closed) as a duplicate of this issue

mentioned in issue #21491

closed

As of 9.4 all GHC binary distributions will be produced by Hadrian

Thank you @bgamari.

I hope that as part of this transition there will be bootstrap from source instructions for both Hadrian and ghc.

@essandess Yes, we provide scripts to bootstrap both hadrian and GHC (see the test=bootstrap CI job).

mentioned in commit dc790cc2

mentioned in merge request !8441 (merged)

marked this issue as related to #21666

marked this issue as related to #21883 (closed)

mentioned in issue #21883 (closed)

I realize this a one-off case where maintaining the old build system is not affordable, but I want to document the complexity I have to manage due to this bug deemed unfixable.

We moved from ghc 8.10 to 9.2 in OpenBSD 7.2 development cycle. I used to rely on the fact that ghc can be bootstrapped with a bindist from the same version. I no longer have an easy way to produce a bindist from ghc 8.10, which I need for bootstrapping 9.2. At the same time OpenBSD is changing the ABI such that the old 8.10 bootstrap binaries will no longer work. So, I now have to manage an 8.10 bindist production on the side to keep 9.2 going across OpenBSD releases. I also will need to set up a 9.2 bindist somewhere so that I can move on to ghc 9.4. An extra wrinkle here is we only made Hadrian build work on OpenBSD in 9.4. This precludes me from making 9.2 self-hosting with Hadrian.

Having a bit more slack would've helped me greatly.

9.2.2 bootstrap fails on macOS 12

Summary

Steps to reproduce

Expected behavior

Environment

Child items ...

Activity

9.2.2 bootstrap fails on macOS 12

Summary

Steps to reproduce

Expected behavior

Environment

Relates to

Activity