Skip to content

GitLab

GHC
GHC
Closed
Opened by Andreas Klebinger@AndreasKDeveloper

Loading libraries with FFI exports may cause segfaults in the compiler if they are loaded far from the rts in memory.

Original report below.

In this case we compile aeson which uses TH triggering dynamic loading of a number of libraries.

Some libraries (eg base) have FFI exports which require us to place a relative jump to the RTS in order to register a stable name. Now an issue arises if base is placed more than 2G from the RTS as we can't have relative jumps are limited to a 2GB range.

In the particular case this caused the jump target to underflow, resulting in a jump to unallocated memory and a segfault.

In more detail the PE linker (PEi386.c:ocResolve_PEi386) fails to detect, or properly deal with the bounds violation.

There seems to be some code in place to deal with an overflow already but fails to detect it.

I haven't had any luck with reproducing it outside of building the aeson package with cabal yet. So for now just documenting the fact.

build.mk used

GhcLibHcOpts += -g3
GhcRtsHcOpts += -g3

STRIP_CMD = :

BUILD_PROF_LIBS    = NO
SplitObjs          = NO
SplitSections      = NO
HADDOCK_DOCS       = NO
BUILD_SPHINX_HTML  = NO
BUILD_SPHINX_PDF   = NO
BUILD_MAN          = NO

Error log:

"E:/ghc_dwarf/inplace/bin/ghc-stage2.exe" "--make" "-fbuilding-cabal-package" "-O" "-outputdir" "C:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build" "-odir" "C:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build" "-hidir" "C:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build" "-stubdir" "C:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build""-i" "-iC:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build" "-i." "-iattoparsec-iso8601/" "-ipure" "-iC:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build\autogen" "-iC:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build\global-autogen" "-IC:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build\autogen" "-IC:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build\global-autogen" "-IC:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build" "-Iinclude" "-IC:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build\include" "-optP-include" "-optPC:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build\autogen\cabal_macros.h" "-this-unit-id" "aeson-1.4.1.0-inplace" "-hide-all-packages" "-Wmissing-home-modules" "-no-user-package-db" "-package-db" "C:\Users\Andi\AppData\Roaming\cabal\store\ghc-8.7.20181025\package.db" "-package-db" "C:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\packagedb\ghc-8.7.20181025" "-package-db" "C:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\package.conf.inplace" "-package-id" "attoparsec-0.13.2.2-8913a968506e9e83757f6fe696d6fe61e0d4b4a8" "-package-id" "base-4.12.0.0" "-package-id" "base-compat-0.10.5-34e11ceb2d98e0262d1d958bca2afc3184e70c60" "-package-id" "bytestring-0.10.8.2" "-package-id" "containers-0.6.0.1" "-package-id" "deepseq-1.4.4.0" "-package-id" "dlist-0.8.0.5-681a0f929505417757ba9f9981a50ab1d7c8a0e0" "-package-id" "ghc-prim-0.5.3" "-package-id" "hashable-1.2.7.0-50f89c5dee92df34fc2d6540cfde1983f26d8e31" "-package-id" "primitive-0.6.4.0-c08c185073660c1604acdddfe5c369afae583ba2" "-package-id" "scientific-0.3.6.2-4bea197b4523e02da61c34a1eed01432d9fefff6" "-package-id" "tagged-0.8.6-d3cce1acba663b646f565adb64d80579664d8caa" "-package-id" "template-haskell-2.14.0.0" "-package-id" "text-1.2.3.1" "-package-id" "th-abstraction-0.2.8.0-e197ba78a6de8bf8fc5d00ecb5a358a8b27bcc92" "-package-id" "time-1.8.0.2" "-package-id" "time-locale-c_-0.1.1.5-7549537073e62ce01921c89c30cc6cafeed99b5b" "-package-id" "unordered-con_-0.2.9.0-f5cd33176070f516c88b1aac3ef61959b09fcfa6" "-package-id" "uuid-types-1.0.3-f68643250767dce83d2c227104d15a0aa9c3c77f" "-package-id" "vector-0.12.0.1-3a9a26f81a463f0efefa41528af3e27d3a88cc7d" "-XHaskell2010" "Data.Aeson" "Data.Aeson.Encoding" "Data.Aeson.Parser" "Data.Aeson.Text" "Data.Aeson.Types" "Data.Aeson.TH" "Data.Aeson.QQ.Simple" "Data.Aeson.Encoding.Internal" "Data.Aeson.Internal" "Data.Aeson.Internal.Time" "Data.Aeson.Parser.Internal" "Data.Aeson.Encode" "Data.Aeson.Compat" "Data.Aeson.Encoding.Builder" "Data.Aeson.Internal.Functions" "Data.Aeson.Parser.Unescape" "Data.Aeson.Parser.Time" "Data.Aeson.Types.FromJSON" "Data.Aeson.Types.Generic" "Data.Aeson.Types.ToJSON" "Data.Aeson.Types.Class" "Data.Aeson.Types.Internal" "Data.Attoparsec.Time" "Data.Attoparsec.Time.Internal" "Data.Aeson.Parser.UnescapePure" "-Wall" "-O2" "-hide-all-packages" "-g13"
[ 2 of 25] Compiling Data.Aeson.Internal.Functions ( Data\Aeson\Internal\Functions.hs, C:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build\Data\Aeson\Internal\Functions.o ) [Data.HashMap.Strict changed]
[ 5 of 25] Compiling Data.Aeson.Types.Generic ( Data\Aeson\Types\Generic.hs, C:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build\Data\Aeson\Types\Generic.o ) [Prelude.Compat changed]
[ 6 of 25] Compiling Data.Aeson.Types.Internal ( Data\Aeson\Types\Internal.hs, C:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build\Data\Aeson\Types\Internal.o ) [Data.Vector changed]
[ 7 of 25] Compiling Data.Aeson.Parser.Internal ( Data\Aeson\Parser\Internal.hs, C:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build\Data\Aeson\Parser\Internal.o ) [Data.Scientific changed]
[ 8 of 25] Compiling Data.Aeson.Parser ( Data\Aeson\Parser.hs, C:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build\Data\Aeson\Parser.o ) [Data.Aeson.Parser.Internal changed]
[ 9 of 25] Compiling Data.Attoparsec.Time.Internal ( attoparsec-iso8601\Data\Attoparsec\Time\Internal.hs, C:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build\Data\Attoparsec\Time\Internal.o ) [Prelude.Compat changed]

attoparsec-iso8601\Data\Attoparsec\Time\Internal.hs:24:1: warning: [-Wunused-imports]
    The import of `Unsafe.Coerce' is redundant
      except perhaps to import instances from `Unsafe.Coerce'
    To import instances alone, use: import Unsafe.Coerce()
   |
24 | import Unsafe.Coerce (unsafeCoerce)
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[10 of 25] Compiling Data.Attoparsec.Time ( attoparsec-iso8601\Data\Attoparsec\Time.hs, C:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build\Data\Attoparsec\Time.o ) [Data.Attoparsec.Text changed]
[11 of 25] Compiling Data.Aeson.Parser.Time ( Data\Aeson\Parser\Time.hs, C:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build\Data\Aeson\Parser\Time.o ) [Data.Attoparsec.Text changed]
[12 of 25] Compiling Data.Aeson.Types.FromJSON ( Data\Aeson\Types\FromJSON.hs, C:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build\Data\Aeson\Types\FromJSON.o ) [Data.Primitive.PrimArray changed]
[13 of 25] Compiling Data.Aeson.Internal ( Data\Aeson\Internal.hs, C:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build\Data\Aeson\Internal.o ) [Data.Aeson.Types.FromJSON changed]
[14 of 25] Compiling Data.Aeson.Internal.Time ( Data\Aeson\Internal\Time.hs, C:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build\Data\Aeson\Internal\Time.o ) [Data.Attoparsec.Time.Internal changed]
[15 of 25] Compiling Data.Aeson.Encoding.Builder ( Data\Aeson\Encoding\Builder.hs, C:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build\Data\Aeson\Encoding\Builder.o ) [Data.Vector changed]
[16 of 25] Compiling Data.Aeson.Encoding.Internal ( Data\Aeson\Encoding\Internal.hs, C:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build\Data\Aeson\Encoding\Internal.o ) [Data.Scientific changed]
[17 of 25] Compiling Data.Aeson.Encoding ( Data\Aeson\Encoding.hs, C:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build\Data\Aeson\Encoding.o ) [Data.Aeson.Encoding.Internal changed]
[18 of 25] Compiling Data.Aeson.Types.ToJSON ( Data\Aeson\Types\ToJSON.hs, C:\ghc\msys64\home\Andi\aeson_repro\dist-newstyle\build\x86_64-windows\ghc-8.7.20181025\aeson-1.4.1.0\build\Data\Aeson\Types\ToJSON.o ) [Data.Primitive.PrimArray changed]

Access violation in generated code when executing data at 0x103fec440

 Attempting to reconstruct a stack trace...

   Frame        Code address
 * 0x845d9c0    0x103fec440
 * 0x845da20    0x400c0f8 E:\ghc_dwarf\inplace\bin\ghc-stage2.exe+0x3c0c0f8
 * 0x845da80    0x3fec9a1 E:\ghc_dwarf\inplace\bin\ghc-stage2.exe+0x3bec9a1
 * 0x845dab0    0x3feca31 E:\ghc_dwarf\inplace\bin\ghc-stage2.exe+0x3beca31
 * 0x845dab8    0x34c8934 E:\ghc_dwarf\inplace\bin\ghc-stage2.exe+0x30c8934
 * 0x845dac0    0xfa340
 * 0x845dac8    0x2a940b78
 * 0x845dad0    0x2a98cd69
 * 0x845dad8    0x2a98d7d0

CallStack (from HasCallStack):
  die', called at .\\Distribution\\Client\\ProjectOrchestration.hs:977:55 in main:Distribution.Client.ProjectOrchestration
cabal.exe: Failed to build aeson-1.4.1.0-inplace. The build process terminated
with exit code 11

I could only reproduce it with master on Windows so far. It always triggers but under very specific circumstances:

  • GHC built with the flags above, adding dwarf info to the ghc executable or removing dwarf info eliminates the issue.
  • Only on a complete rebuild of aeson. Restarting the crashed build finishes without an error.
Edited by Andreas Klebinger