Loading libraries with FFI exports may cause segfaults in the compiler if they are loaded far from the rts in memory.
Original report below.
In this case we compile aeson which uses TH triggering dynamic loading of a number of libraries.
Some libraries (eg base) have FFI exports which require us to place a relative jump to the RTS in order to register a stable name. Now an issue arises if base is placed more than 2G from the RTS as we can't have relative jumps are limited to a 2GB range.
In the particular case this caused the jump target to underflow, resulting in a jump to unallocated memory and a segfault.
In more detail the PE linker (PEi386.c:ocResolve_PEi386) fails to detect, or properly deal with the bounds violation.
There seems to be some code in place to deal with an overflow already but fails to detect it.
I haven't had any luck with reproducing it outside of building the aeson package with cabal yet. So for now just documenting the fact.
build.mk used
GhcLibHcOpts += -g3
GhcRtsHcOpts += -g3
STRIP_CMD = :
BUILD_PROF_LIBS = NO
SplitObjs = NO
SplitSections = NO
HADDOCK_DOCS = NO
BUILD_SPHINX_HTML = NO
BUILD_SPHINX_PDF = NO
BUILD_MAN = NO
Error log:
"E:/ghc_dwarf/inplace/bin/ghc-stage2.exe" "--make" "-fbuilding-cabal-package" "-O" "-outputdir" "C:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build" "-odir" "C:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build" "-hidir" "C:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build" "-stubdir" "C:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build""-i" "-iC:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build" "-i." "-iattoparsec-iso8601/" "-ipure" "-iC:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build\autogen" "-iC:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build\global-autogen" "-IC:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build\autogen" "-IC:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build\global-autogen" "-IC:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build" "-Iinclude" "-IC:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build\include" "-optP-include" "-optPC:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build\autogen\cabal_macros.h" "-this-unit-id" "aeson-1.4.1.0-inplace" "-hide-all-packages" "-Wmissing-home-modules" "-no-user-package-db" "-package-db" "C:\Users\Andi\AppData\Roaming\cabal\store\ghc-8.7.20181025\package.db" "-package-db" "C:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\packagedb\ghc-8.7.20181025" "-package-db" "C:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\package.conf.inplace" "-package-id" "attoparsec-0.13.2.2-8913a968506e9e83757f6fe696d6fe61e0d4b4a8" "-package-id" "base-4.12.0.0" "-package-id" "base-compat-0.10.5-34e11ceb2d98e0262d1d958bca2afc3184e70c60" "-package-id" "bytestring-0.10.8.2" "-package-id" "containers-0.6.0.1" "-package-id" "deepseq-1.4.4.0" "-package-id" "dlist-0.8.0.5-681a0f929505417757ba9f9981a50ab1d7c8a0e0" "-package-id" "ghc-prim-0.5.3" "-package-id" "hashable-1.2.7.0-50f89c5dee92df34fc2d6540cfde1983f26d8e31" "-package-id" "primitive-0.6.4.0-c08c185073660c1604acdddfe5c369afae583ba2" "-package-id" "scientific-0.3.6.2-4bea197b4523e02da61c34a1eed01432d9fefff6" "-package-id" "tagged-0.8.6-d3cce1acba663b646f565adb64d80579664d8caa" "-package-id" "template-haskell-2.14.0.0" "-package-id" "text-1.2.3.1" "-package-id" "th-abstraction-0.2.8.0-e197ba78a6de8bf8fc5d00ecb5a358a8b27bcc92" "-package-id" "time-1.8.0.2" "-package-id" "time-locale-c_-0.1.1.5-7549537073e62ce01921c89c30cc6cafeed99b5b" "-package-id" "unordered-con_-0.2.9.0-f5cd33176070f516c88b1aac3ef61959b09fcfa6" "-package-id" "uuid-types-1.0.3-f68643250767dce83d2c227104d15a0aa9c3c77f" "-package-id" "vector-0.12.0.1-3a9a26f81a463f0efefa41528af3e27d3a88cc7d" "-XHaskell2010" "Data.Aeson" "Data.Aeson.Encoding" "Data.Aeson.Parser" "Data.Aeson.Text" "Data.Aeson.Types" "Data.Aeson.TH" "Data.Aeson.QQ.Simple" "Data.Aeson.Encoding.Internal" "Data.Aeson.Internal" "Data.Aeson.Internal.Time" "Data.Aeson.Parser.Internal" "Data.Aeson.Encode" "Data.Aeson.Compat" "Data.Aeson.Encoding.Builder" "Data.Aeson.Internal.Functions" "Data.Aeson.Parser.Unescape" "Data.Aeson.Parser.Time" "Data.Aeson.Types.FromJSON" "Data.Aeson.Types.Generic" "Data.Aeson.Types.ToJSON" "Data.Aeson.Types.Class" "Data.Aeson.Types.Internal" "Data.Attoparsec.Time" "Data.Attoparsec.Time.Internal" "Data.Aeson.Parser.UnescapePure" "-Wall" "-O2" "-hide-all-packages" "-g13"
[ 2 of 25] Compiling Data.Aeson.Internal.Functions ( Data\Aeson\Internal\Functions.hs, C:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build\Data\Aeson\Internal\Functions.o ) [Data.HashMap.Strict changed]
[ 5 of 25] Compiling Data.Aeson.Types.Generic ( Data\Aeson\Types\Generic.hs, C:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build\Data\Aeson\Types\Generic.o ) [Prelude.Compat changed]
[ 6 of 25] Compiling Data.Aeson.Types.Internal ( Data\Aeson\Types\Internal.hs, C:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build\Data\Aeson\Types\Internal.o ) [Data.Vector changed]
[ 7 of 25] Compiling Data.Aeson.Parser.Internal ( Data\Aeson\Parser\Internal.hs, C:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build\Data\Aeson\Parser\Internal.o ) [Data.Scientific changed]
[ 8 of 25] Compiling Data.Aeson.Parser ( Data\Aeson\Parser.hs, C:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build\Data\Aeson\Parser.o ) [Data.Aeson.Parser.Internal changed]
[ 9 of 25] Compiling Data.Attoparsec.Time.Internal ( attoparsec-iso8601\Data\Attoparsec\Time\Internal.hs, C:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build\Data\Attoparsec\Time\Internal.o ) [Prelude.Compat changed]
attoparsec-iso8601\Data\Attoparsec\Time\Internal.hs:24:1: warning: [-Wunused-imports]
The import of `Unsafe.Coerce' is redundant
except perhaps to import instances from `Unsafe.Coerce'
To import instances alone, use: import Unsafe.Coerce()
|
24 | import Unsafe.Coerce (unsafeCoerce)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[10 of 25] Compiling Data.Attoparsec.Time ( attoparsec-iso8601\Data\Attoparsec\Time.hs, C:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build\Data\Attoparsec\Time.o ) [Data.Attoparsec.Text changed]
[11 of 25] Compiling Data.Aeson.Parser.Time ( Data\Aeson\Parser\Time.hs, C:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build\Data\Aeson\Parser\Time.o ) [Data.Attoparsec.Text changed]
[12 of 25] Compiling Data.Aeson.Types.FromJSON ( Data\Aeson\Types\FromJSON.hs, C:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build\Data\Aeson\Types\FromJSON.o ) [Data.Primitive.PrimArray changed]
[13 of 25] Compiling Data.Aeson.Internal ( Data\Aeson\Internal.hs, C:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build\Data\Aeson\Internal.o ) [Data.Aeson.Types.FromJSON changed]
[14 of 25] Compiling Data.Aeson.Internal.Time ( Data\Aeson\Internal\Time.hs, C:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build\Data\Aeson\Internal\Time.o ) [Data.Attoparsec.Time.Internal changed]
[15 of 25] Compiling Data.Aeson.Encoding.Builder ( Data\Aeson\Encoding\Builder.hs, C:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build\Data\Aeson\Encoding\Builder.o ) [Data.Vector changed]
[16 of 25] Compiling Data.Aeson.Encoding.Internal ( Data\Aeson\Encoding\Internal.hs, C:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build\Data\Aeson\Encoding\Internal.o ) [Data.Scientific changed]
[17 of 25] Compiling Data.Aeson.Encoding ( Data\Aeson\Encoding.hs, C:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build\Data\Aeson\Encoding.o ) [Data.Aeson.Encoding.Internal changed]
[18 of 25] Compiling Data.Aeson.Types.ToJSON ( Data\Aeson\Types\ToJSON.hs, C:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build\Data\Aeson\Types\ToJSON.o ) [Data.Primitive.PrimArray changed]
Access violation in generated code when executing data at 0x103fec440
Attempting to reconstruct a stack trace...
Frame Code address
* 0x845d9c0 0x103fec440
* 0x845da20 0x400c0f8 E:\ghc_dwarf\inplace\bin\ghc-stage2.exe+0x3c0c0f8
* 0x845da80 0x3fec9a1 E:\ghc_dwarf\inplace\bin\ghc-stage2.exe+0x3bec9a1
* 0x845dab0 0x3feca31 E:\ghc_dwarf\inplace\bin\ghc-stage2.exe+0x3beca31
* 0x845dab8 0x34c8934 E:\ghc_dwarf\inplace\bin\ghc-stage2.exe+0x30c8934
* 0x845dac0 0xfa340
* 0x845dac8 0x2a940b78
* 0x845dad0 0x2a98cd69
* 0x845dad8 0x2a98d7d0
CallStack (from HasCallStack):
die', called at .\\Distribution\\Client\\ProjectOrchestration.hs:977:55 in main:Distribution.Client.ProjectOrchestration
cabal.exe: Failed to build aeson-1.4.1.0-inplace. The build process terminated
with exit code 11I could only reproduce it with master on Windows so far. It always triggers but under very specific circumstances:
- GHC built with the flags above, adding dwarf info to the ghc executable or removing dwarf info eliminates the issue.
- Only on a complete rebuild of aeson. Restarting the crashed build finishes without an error.