GitLab

joelteon encountered this issue in a rather complicated program, and we worked out the cause over IRC.

Suppose that ghci needs to sequentially load three modules A, B and C, where B refers to symbols from A and C refers to symbols from both A and B. (For example modules B, C and another module D all contain Template Haskell that refers to the previous module(s).) The object code linker currently works like this:

• Link the module A.o into a dylib ghc_1.dylib
• Link the module B.o against ghc_1.dylib into a new dylib ghc_2.dylib
• Link the module C.o against ghc_2.dylib into a new dylib ghc_3.dylib

As a result, ghc_2.dylib ends up with a NEEDED (or whatever it's called in Mach-O) entry for ghc_1.dylib, and ghc_3.dylib ends up with a NEEDED entry for ghc_2.dylib.

However, this apparently does not satisfy the OS X dlopen implementation, which complains about a missing symbol _A_foo referenced by ghc_3.dylib and which is defined in ghc_1.dylib. Apparently the dynamic loader only checks the direct dependencies when trying to resolve undefined symbols.

(The Linux dynamic loader seems to be perfectly happy to use an indirect dependency to resolve an undefined symbol. But I found out that the linker gold has the same sort of behavior as the OS X dynamic loader. I don't know whether there is any standard here, but it seems that we cannot rely on the Linux dynamic loader's behavior.)

Presumably the fix is to keep track of all the previous temporary dylibs (rather than just one in last_temp_so) and link against all of them when building a new temporary dylib. I'm slightly worried about quadratic behavior here, but I think it's unlikely to be an issue in practice.

I have a reproducer at http://lpaste.net/808723564239781888 (which I'll add to the test suite next) and oherrala ran the test on an OS X system with the following results:

=====> Last(ghci) 1167 of 4480 [0, 0, 0] 
cd ./ghci/scripts && HC="/Users/oherrala/gits/ghc/inplace/bin/ghc-stage2" HC_OPTS="-dcore-lint -dcmm-lint -dno-debug-output -no-user-package-db -rtsopts -fno-warn-tabs -fno-ghci-history " "/Users/oherrala/gits/ghc/inplace/bin/ghc-stage2" -dcore-lint -dcmm-lint -dno-debug-output -no-user-package-db -rtsopts -fno-warn-tabs -fno-ghci-history  --interactive -v0 -ignore-dot-ghci +RTS -I0.1 -RTS    <Last.script > Last.run.stdout 2> Last.run.stderr
Actual stderr output differs from expected:
--- /dev/null	2015-04-18 17:23:26.000000000 +0300
+++ ./ghci/scripts/Last.run.stderr	2015-04-18 17:23:34.000000000 +0300
@@ -0,0 +1,9 @@
+ghc-stage2: panic! (the 'impossible' happened)
+  (GHC version 7.11.20150418 for x86_64-apple-darwin):
+	Loading temp shared object failed: dlopen(/var/folders/64/90jfy8lj65bcm1k02syxz_l80000gn/T/ghc18812_0/libghc18812_12.dylib, 5): Symbol not found: _LastA_a_closure
+  Referenced from: /var/folders/64/90jfy8lj65bcm1k02syxz_l80000gn/T/ghc18812_0/libghc18812_12.dylib
+  Expected in: flat namespace
+ in /var/folders/64/90jfy8lj65bcm1k02syxz_l80000gn/T/ghc18812_0/libghc18812_12.dylib
+
+Please report this as a GHC bug:  http://www.haskell.org/ghc/reportabug
+
Actual stdout output differs from expected:
--- ./ghci/scripts/Last.stdout	2015-04-18 16:26:55.000000000 +0300
+++ ./ghci/scripts/Last.run.stdout	2015-04-18 17:23:34.000000000 +0300
@@ -1,3 +1,2 @@
 3
 4
-7
*** unexpected failure for Last(ghci)

Trac metadata

Trac field	Value
Version	7.10.1
Type	Bug
TypeOfFailure	OtherFailure
Priority	high
Resolution	Unresolved
Component	Compiler
Test case
Differential revisions
BlockedBy
Related
Blocking
CC
Operating system
Architecture