Released long ago. Closing.
We have observed in Debian that GHC can produce a erratically behaving version of ghc-pkg which causes issues when piping it's output to another program.
Example:
make_setup_recipe
Running ghc --make Setup.hs -o debian/hlibrary.setup
[1 of 1] Compiling Main ( Setup.hs, Setup.o )
Linking debian/hlibrary.setup ...
. /usr/share/haskell-devscripts/Dh_Haskell.sh && \
configure_recipe
Running debian/hlibrary.setup configure --ghc -v2 --package-db=/var/lib/ghc/package.conf.d --prefix=/usr --libdir=/usr/lib/haskell-packages/ghc/lib --libexecdir=/usr/lib --builddir=dist-ghc --ghc-option=-optl-Wl\,-z\,relro --haddockdir=/usr/lib/ghc-doc/haddock/base-prelude-/ --datasubdir=base-prelude --htmldir=/usr/share/doc/libghc-base-prelude-doc/html/ --enable-library-profiling
Configuring base-prelude-1.2.1...
Warning: cannot determine version of /usr/bin/ghc-pkg :
""
hlibrary.setup: The program 'ghc-pkg' is required but the version of
/usr/bin/ghc-pkg could not be determined.
Examining the behavior of the affected ghc-pkg binary on m68k shows what's going on:
root@pacman:~# uname -a
Linux pacman 4.16.0-2-m68k #1 Debian 4.16.16-1 (2018-06-19) m68k GNU/Linux
root@pacman:~# ghc-pkg --version
GHC package manager version 8.2.2
root@pacman:~# ghc-pkg --version | cat
root@pacman:~#
Comparing that to an x86_64 machine shows that piping to cat should actually output something:
glaubitz@epyc:~$ uname -a
Linux epyc 4.16.0-2-amd64 #1 SMP Debian 4.16.12-1 (2018-05-27) x86_64 GNU/Linux
glaubitz@epyc:~$ ghc --version
The Glorious Glasgow Haskell Compilation System, version 8.2.2
glaubitz@epyc:~$ ghc --version | cat
The Glorious Glasgow Haskell Compilation System, version 8.2.2
glaubitz@epyc:~$
Further investigation shows that the problem is partially resolved when building GHC with reduced optimization:
echo "SRC_HC_OPTS += -O0 -H64m" >> mk/build.mk
echo "GhcStage1HcOpts = -O" >> mk/build.mk
echo "GhcStage2HcOpts = -O0" >> mk/build.mk
echo "GhcLibHcOpts = -O" >> mk/build.mk
The above issue goes away, but ghc-pkg itself still shows some strange behavior:
make_setup_recipe
Running ghc --make Setup.hs -o debian/hlibrary.setup
[1 of 1] Compiling Main ( Setup.hs, Setup.o )
Linking debian/hlibrary.setup ...
. /usr/share/haskell-devscripts/Dh_Haskell.sh && \
configure_recipe
Running debian/hlibrary.setup configure --ghc -v2 --package-db=/var/lib/ghc/package.conf.d --prefix=/usr --libdir=/usr/lib/haskell-packages/ghc/lib --libexecdir=/usr/lib --builddir=dist-ghc --ghc-option=-optl-Wl\,-z\,relro --haddockdir=/usr/lib/ghc-doc/haddock/microlens-ghc-0.4.8.0/ --datasubdir=microlens-ghc --htmldir=/usr/share/doc/libghc-microlens-ghc-doc/html/ --enable-library-profiling
Configuring microlens-ghc-0.4.8.0...
hlibrary.setup: ghc-pkg dump failed: dieVerbatim: user error (hlibrary.setup:
'/usr/bin/ghc-pkg' exited with an error:
ghc-pkg: <stdout>: commitBuffer: invalid argument (invalid character)
)
To reproduce the issue, GHC can be tested using QEMU which has pretty good support for the m68k target these days.
Trac field | Value |
---|---|
Version | 8.4.3 |
Type | Bug |
TypeOfFailure | OtherFailure |
Priority | normal |
Resolution | Unresolved |
Component | Compiler |
Test case | |
Differential revisions | |
BlockedBy | |
Related | |
Blocking | |
CC | |
Operating system | |
Architecture |
Yeah, NixOS
's nixpkgs
will eventually migrate to hadrian
: https://github.com/NixOS/nixpkgs/issues/137127. But it will take some time to move over to it.
ghc-master
from today's 75f0091b fails to install as:
installing
install flags: SHELL=/<<NIX>>/bash-5.1-p16/bin/bash pkgconfigdir=/<<NIX>>/ghc-9.3.20220406/lib/pkgconfig m4datadir=/<<NIX>>/ghc-9.3.20220406/share/aclocal aclocaldir=/<<NIX>>/ghc-9.3.20220406/share/aclocal install
===--- building phase 0
make --no-print-directory -f ghc.mk phase=0 phase_0_builds
make[1]: Nothing to be done for 'phase_0_builds'.
===--- building phase 1
make --no-print-directory -f ghc.mk phase=1 phase_1_builds
make[1]: Nothing to be done for 'phase_1_builds'.
===--- building final phase
make --no-print-directory -f ghc.mk phase=final install
/<<NIX>>/coreutils-9.1/bin/install -c -m 755 -d "/<<NIX>>/ghc-9.3.20220406/lib/ghc-9.5.20220625/bin"
for i in utils/unlit/dist-install/build/tmp/unlit utils/hp2ps/dist-install/build/tmp/hp2ps utils/hp2ps/dist-install/build/tmp/hp2ps utils/haddock/dist/build/tmp/haddock utils/haddock/dist/build/tmp/haddock utils/hsc2hs/dist-install/build/tmp/hsc2hs utils/hsc2hs/dist-install/build/tmp/hsc2hs utils/ghc-pkg/dist-install/build/tmp/ghc-pkg utils/ghc-pkg/dist-install/build/tmp/ghc-pkg utils/hpc/dist-install/build/tmp/hpc utils/hpc/dist-install/build/tmp/hpc utils/runghc/dist-install/build/tmp/runghc utils/runghc/dist-install/build/tmp/runghc ghc/stage2/build/tmp/ghc-stage2 ghc/stage2/build/tmp/ghc-stage2 utils/iserv/stage2/build/tmp/ghc-iserv utils/iserv/stage2_p/build/tmp/ghc-iserv-prof utils/iserv/stage2_dyn/build/tmp/ghc-iserv-dyn; do \
/<<NIX>>/coreutils-9.1/bin/install -c -m 755 $i "/<<NIX>>/ghc-9.3.20220406/lib/ghc-9.5.20220625/bin"; \
done
"mv" "/<<NIX>>/ghc-9.3.20220406/lib/ghc-9.5.20220625/bin/ghc-stage2" "/<<NIX>>/ghc-9.3.20220406/lib/ghc-9.5.20220625/bin/ghc"
make[1]: *** No rule to make target 'mk/system-cxx-std-lib-1.0.conf.install', needed by 'install_packages'. Stop.
I think it's a recent regression caused by 0ef249aa where system-cxx-std-lib-1.0.conf
is created (somewhat manually), but not the .install varianlt of it
.
ghc-master
Optional:
/cc @bgamari
Or we can avoid .install
file entirely and always use inplace file as it's not expected to change. Proposed change: !8532 (closed)
ghc.mk: fix 'make install' (mk/system-cxx-std-lib-1.0.conf.install
does not exist)
before the change make install
was failing as:
"mv" "/<<NIX>>/ghc-9.3.20220406/lib/ghc-9.5.20220625/bin/ghc-stage2" "/<<NIX>>/ghc-9.3.20220406/lib/ghc-9.5.20220625/bin/ghc"
make[1]: *** No rule to make target 'mk/system-cxx-std-lib-1.0.conf.install', needed by 'install_packages'. Stop.
I think it's a recent regression caused by 0ef249aa where system-cxx-std-lib-1.0.conf
is created (somewhat manually), but not the .install varianlt of it.
The fix is to consistently use mk/system-cxx-std-lib-1.0.conf
everywhere.
Closes: #21784
Sergei Trofimovich (1f540289) at 25 Jun 08:45
ghc.mk: fix 'make install' (`mk/system-cxx-std-lib-1.0.conf.install...
... and 6765 more commits
I think we need a lighter variant of
$(eval $(call manual-package-config,rts,dist-install))
# from rules/manual-package-config.mk:
# ...
# This is actually a real file, but we need to recreate it on every
# "make install", so we declare it as phony
.PHONY: $1/$2/package.conf.install
$1/$2/package.conf.install : $1/package.conf.in | $$$$(dir $$$$@)/.
$$(HS_CPP) -P \
-DINSTALLING \
-DLIB_DIR='"$$(if $$(filter YES,$$(RelocatableBuild)),$$$$topdir,$$(ghclibdir))/$1"' \
-DINCLUDE_DIR='"$$(if $$(filter YES,$$(RelocatableBuild)),$$$$topdir,$$(ghclibdir))/$1/include"' \
$$($1_$2_PACKAGE_CPP_OPTS) \
-x c $1/package.conf.in -o $$@.raw
grep -v '^#pragma GCC' $$@.raw | \
sed -e 's/""//g' -e 's/:[ ]*,/: /g' >$$@
endef
ghc-master
from today's 75f0091b fails to install as:
installing
install flags: SHELL=/<<NIX>>/bash-5.1-p16/bin/bash pkgconfigdir=/<<NIX>>/ghc-9.3.20220406/lib/pkgconfig m4datadir=/<<NIX>>/ghc-9.3.20220406/share/aclocal aclocaldir=/<<NIX>>/ghc-9.3.20220406/share/aclocal install
===--- building phase 0
make --no-print-directory -f ghc.mk phase=0 phase_0_builds
make[1]: Nothing to be done for 'phase_0_builds'.
===--- building phase 1
make --no-print-directory -f ghc.mk phase=1 phase_1_builds
make[1]: Nothing to be done for 'phase_1_builds'.
===--- building final phase
make --no-print-directory -f ghc.mk phase=final install
/<<NIX>>/coreutils-9.1/bin/install -c -m 755 -d "/<<NIX>>/ghc-9.3.20220406/lib/ghc-9.5.20220625/bin"
for i in utils/unlit/dist-install/build/tmp/unlit utils/hp2ps/dist-install/build/tmp/hp2ps utils/hp2ps/dist-install/build/tmp/hp2ps utils/haddock/dist/build/tmp/haddock utils/haddock/dist/build/tmp/haddock utils/hsc2hs/dist-install/build/tmp/hsc2hs utils/hsc2hs/dist-install/build/tmp/hsc2hs utils/ghc-pkg/dist-install/build/tmp/ghc-pkg utils/ghc-pkg/dist-install/build/tmp/ghc-pkg utils/hpc/dist-install/build/tmp/hpc utils/hpc/dist-install/build/tmp/hpc utils/runghc/dist-install/build/tmp/runghc utils/runghc/dist-install/build/tmp/runghc ghc/stage2/build/tmp/ghc-stage2 ghc/stage2/build/tmp/ghc-stage2 utils/iserv/stage2/build/tmp/ghc-iserv utils/iserv/stage2_p/build/tmp/ghc-iserv-prof utils/iserv/stage2_dyn/build/tmp/ghc-iserv-dyn; do \
/<<NIX>>/coreutils-9.1/bin/install -c -m 755 $i "/<<NIX>>/ghc-9.3.20220406/lib/ghc-9.5.20220625/bin"; \
done
"mv" "/<<NIX>>/ghc-9.3.20220406/lib/ghc-9.5.20220625/bin/ghc-stage2" "/<<NIX>>/ghc-9.3.20220406/lib/ghc-9.5.20220625/bin/ghc"
make[1]: *** No rule to make target 'mk/system-cxx-std-lib-1.0.conf.install', needed by 'install_packages'. Stop.
I think it's a recent regression caused by 0ef249aa where system-cxx-std-lib-1.0.conf
is created (somewhat manually), but not the .install varianlt of it
.
ghc-master
Optional:
/cc @bgamari
.hs-boot
make rules: add missing order-only dependency on target directory
Noticed missing target directory dependency as a build failure in
make --shuffle
mode (added in https://savannah.gnu.org/bugs/index.php?62100):
"cp" libraries/base/./GHC/Stack/CCS.hs-boot libraries/base/dist-install/build/GHC/Stack/CCS.hs-boot
cp: cannot create regular file 'libraries/base/dist-install/build/GHC/Stack/CCS.hs-boot': No such file or directory
libraries/haskeline/ghc.mk:4: libraries/haskeline/dist-install/build/.depend-v-p-dyn.haskell: No such file or directory
make[1]: *** [libraries/base/ghc.mk:4: libraries/base/dist-install/build/GHC/Stack/CCS.hs-boot] Error 1 shuffle=1656129254
make: *** [Makefile:128: all] Error 2 shuffle=1656129254
Note that cp
complains about inability to create target file.
The change adds order-only dependency on a target directory (similar to the rest of rules in that file).
The bug is lurking there since 2009 commit 34cc75e1 (GHC new build system megapatch
.) where upfront directory creation was never added to
.hs-boot
files.
Sergei Trofimovich (406c800a) at 25 Jun 07:43
.hs-boot
make rules: add missing order-only dependency on target ...
... and 6765 more commits
UNREG: implement 64-bit mach ops for 32-bit targets
Noticed build failures like
```
ghc-stage1: panic! (the 'impossible' happened)
GHC version 9.3.20210721:
pprCallishMachOp_for_C: MO_x64_Ne not supported!
```
on `--tagget=hppa2.0-unknown-linux-gnu`.
The change does not fix all 32-bit unreg target problems,
but at least allows linking final ghc binaries.
Sergei Trofimovich (1ec03ca7) at 21 Jul 21:33
UNREG: implement 64-bit mach ops for 32-bit targets
... and 4964 more commits
GHC's allocateExec()
fails on libffi-3.4, corrupts memory.
The problem is observed on ghc
built against libffi-3.4-rc1
:
$ ghci
GHCi, version 8.10.5: https://www.haskell.org/ghc/ :? for help
ghc: freeHaskellFunctionPtr: not for me, guv! 0x7f0417a1efe8
ghc: freeHaskellFunctionPtr: not for me, guv! 0x7f0417a1efc8
Here ghc
tells us that freeHaskellFunctionPtr
sees corrupted
code memory where trampoline should be stored.
That happens because ghc
has very strong assumptions about libffi
's
void * ret = ffi_closure_alloc (size_t size, void **code)
semantics:
code
has at least size
bytes of executable memoryret
data is not used by libffi#if defined(linux_HOST_OS) || defined(netbsd_HOST_OS)
// On Linux we need to use libffi for allocating executable memory,
// because it knows how to work around the restrictions put in place
// by SELinux. The same goes for NetBSD where it is prohibited to
// mark a page mapping both writable and executable at the same time.
AdjustorWritable allocateExec (W_ bytes, AdjustorExecutable *exec_ret)
{
void **ret, **exec;
ACQUIRE_SM_LOCK;
ret = ffi_closure_alloc (sizeof(void *) + (size_t)bytes, (void**)&exec);
RELEASE_SM_LOCK;
if (ret == NULL) return ret;
/// breaks [1.] assumption: writes into libffi's space:
*ret = ret; // save the address of the writable mapping, for freeExec().
*exec_ret = exec + 1;
/// breaks [2.] assumption:
return (ret + 1);
}
Starting from https://github.com/libffi/libffi/commit/9ba559217bea0803263a9a9a0bafcf9203606f5b
libffi-3.4
neither of [1.]
or [2.]
is true:
code
now is a small trampoline of very specific form, user can't write
arbitrary instructions there.ret
now keeps a pointer to the trampoline used later by ffi_closure_free()
.I think ghc needs to implement it's own form of allocateExec()
and should stop
relying on libffi
's internals to provide executable scratch space.
As a workaround https://github.com/libffi/libffi/pull/647 adds --disable-exec-static-tramp
build-time libffi flag to allow ghc
to limp along for a while.
8.10.5
and HEAD
, both have a problem.Optional: