|
|
# The Dynamic Linking Debate
|
|
|
|
|
|
|
|
|
In GHC 7.8 we made GHCi dynamically-linked. That is, it links against compiled Haskell packages (it already used dynamically-linked C libraries, that is not the issue here).
|
|
|
|
|
|
|
|
|
Why did we do this? The main reason is that we need a complete linker implementation in the RTS, and that's hard (\~7000 LoC currently, supporting three object file formats). This is a lot of code to get right, and a lot to maintain. Supporting all the dark corners of the linking semantics is difficult: e.g. weak symbols were not supported (but have since been implemented). The linker semantics on OS X in particular change from time to time, and we have to keep up.
|
|
|
|
|
|
|
|
|
The alternative is to use the system dynamic linker. The following section outlines the implications of the switch.
|
|
|
|
|
|
## Implications of switching to dynamic linking
|
|
|
|
|
|
|
|
|
Dynamic linking implies
|
|
|
|
|
|
- Generating position-independent code (the `-fPIC` flag), so that the code can be loaded at any address in memory
|
|
|
- Generating dynamic references (the `-dynamic` flag) to entities outside of the current shared library
|
|
|
|
|
|
|
|
|
GHC's output is not deterministic, which means that if we compile both a static and a dynamic library, they need distinct sets of interface files. The mechanism for doing this in GHC is "ways"; so dynamic linking is a "way" just like profiling. When you compile with `-dynamic`, GHC looks for the dynamic version of all the packages.
|
|
|
|
|
|
|
|
|
We decided to keep static linking as the default for ordinary standalone executables, because dynamic linking implies a performance penalty, and there are concerns about backwards compatibility for build systems and suchlike.
|
|
|
|
|
|
|
|
|
To ensure that all the libraries are available both to use in GHCi and to use when static linking, cabal has to build everything twice. To mitigate the overhead of building everything twice, we implemented the `-dynamic-too` flag, which generates both static and dynamic object files from a single compilation, sharing most of the compilation pipeline and only performing the code-generation steps twice.
|
|
|
|
|
|
|
|
|
GHCi can load object files if some source files are precompiled. To make this work with dynamic linking, we have to link the object files together into a temporary shared library in `/tmp`, and then load that. To make it possible for new object files to override old object files, we had to load shared libraries into separate namespaces, which lead to breakage in the old static linker that we still haven't found a good fix for: [\#8395](https://gitlab.haskell.org//ghc/ghc/issues/8395).
|
|
|
|
|
|
|
|
|
When using TemplateHaskell, we have to be able to load object code into GHCi, which means we need to have dynamic versions of all the object files. Therefore, we automatically enable `-dynamic-too` when `TemplateHaskell` is on. This slows down compilation, and is an annoying special case.
|
|
|
|
|
|
## Pros of dynamic linking
|
|
|
|
|
|
- GHCi starts up faster (when the dynamic libs are cached)
|
|
|
- We can get GHCi support on some platforms where we couldn't before, using the LLVM backend + dynamic linking
|
|
|
- Some weird workarounds could go away: the GlobalStore in the RTS and related nonsense in the base package, which is necessary to handle the fact that with static linking we had two copies of the base package loaded. There is similar stuff in the GHC package for the same reason (see [\#8276](https://gitlab.haskell.org//ghc/ghc/issues/8276))
|
|
|
|
|
|
## Cons of dynamic linking
|
|
|
|
|
|
- `-dynamic-too` is buggy, slow, and has an ugly implementation
|
|
|
- linking shared libs on the fly in GHCi is annoying
|
|
|
- It still doesn't work on Windows, because we can't get the GHC package to fit into a DLL.
|
|
|
- In order to support GHCi loading dynamic object files, we had to disable a nice optimisation in `-dynamic` that made all intra-package calls fast static calls instead of indirect dynamic calls.
|
|
|
|
|
|
## Do we need the static linker anyway?
|
|
|
|
|
|
|
|
|
What happens if we build a static executable and link in the GHC package? Can we use the interpreter? Currently the answer is yes, and we fall back to using the RTS linker in this case. (The converse also applies if static linking is the default).
|
|
|
|
|
|
|
|
|
The dynamic linker doesn't support unloading objects currently, so if you want that functionality then the static linker is the only option. The Haxl project at Facebook is using the static linker for this reason, and also due to the performance overhead of dynamic linking. It's not known whether the dynamic linker can support unloading.
|
|
|
|
|
|
## Bugs in the static linker
|
|
|
|
|
|
|
|
|
Copied from [DynamicGhcPrograms](dynamic-ghc-programs):
|
|
|
|
|
|
<table><tr><th>Ticket</th>
|
|
|
<th>Affects OS X x86_64?</th>
|
|
|
<th>Affects OS X x86?</th>
|
|
|
<th>Affects Linux x86_64?</th>
|
|
|
<th>Affects Linux x86?</th>
|
|
|
<th>Affects Windows x86_64?</th>
|
|
|
<th>Affects Windows x86?</th>
|
|
|
<th>Affects other platforms?
|
|
|
</th></tr>
|
|
|
<tr><th>[\#781 GHCi on x86_64, cannot link to static data in shared libs](https://gitlab.haskell.org//ghc/ghc/issues/781)</th>
|
|
|
<th>no</th>
|
|
|
<th>no</th>
|
|
|
<th>**YES**</th>
|
|
|
<th>no</th>
|
|
|
<th>no</th>
|
|
|
<th>no</th>
|
|
|
<th>no
|
|
|
</th></tr>
|
|
|
<tr><th>[\#1883 GHC can't find library using "short" name](https://gitlab.haskell.org//ghc/ghc/issues/1883)</th>
|
|
|
<th>no</th>
|
|
|
<th>no</th>
|
|
|
<th>no</th>
|
|
|
<th>no</th>
|
|
|
<th>**probably**</th>
|
|
|
<th>**YES**</th>
|
|
|
<th>no
|
|
|
</th></tr>
|
|
|
<tr><th>[\#2283 WIndows: loading objects that refer to DLL symbols](https://gitlab.haskell.org//ghc/ghc/issues/2283)</th>
|
|
|
<th>no</th>
|
|
|
<th>no</th>
|
|
|
<th>no</th>
|
|
|
<th>no</th>
|
|
|
<th>**probably**</th>
|
|
|
<th>**YES**</th>
|
|
|
<th>no
|
|
|
</th></tr>
|
|
|
<tr><th>[\#3242 ghci: can't load .so/.DLL for: m (addDLL: could not load DLL)](https://gitlab.haskell.org//ghc/ghc/issues/3242)</th>
|
|
|
<th>no</th>
|
|
|
<th>no</th>
|
|
|
<th>no</th>
|
|
|
<th>no</th>
|
|
|
<th>**probably**</th>
|
|
|
<th>**YES**</th>
|
|
|
<th>no
|
|
|
</th></tr>
|
|
|
<tr><th>[\#3654 Mach-O GHCi linker lacks support for a range of relocation entries](https://gitlab.haskell.org//ghc/ghc/issues/3654)</th>
|
|
|
<th>**YES**</th>
|
|
|
<th>**YES**</th>
|
|
|
<th>no</th>
|
|
|
<th>no</th>
|
|
|
<th>no</th>
|
|
|
<th>no</th>
|
|
|
<th>no
|
|
|
</th></tr>
|
|
|
<tr><th>[\#4244 Use system linker in GHCi to support alpha, ia64, ppc64](https://gitlab.haskell.org//ghc/ghc/issues/4244)</th>
|
|
|
<th>no</th>
|
|
|
<th>no</th>
|
|
|
<th>no</th>
|
|
|
<th>no</th>
|
|
|
<th>no</th>
|
|
|
<th>no</th>
|
|
|
<th>**YES**</th></tr>
|
|
|
<tr><th>[\#5062 Patch: Debug output for OS X linker and coding standard upgrades](https://gitlab.haskell.org//ghc/ghc/issues/5062)</th>
|
|
|
<th>**YES**</th>
|
|
|
<th>**YES**</th>
|
|
|
<th>no</th>
|
|
|
<th>no</th>
|
|
|
<th>no</th>
|
|
|
<th>no</th>
|
|
|
<th>no
|
|
|
</th></tr>
|
|
|
<tr><th>[\#5197 Support static linker semantics for archives and weak symbols](https://gitlab.haskell.org//ghc/ghc/issues/5197)</th>
|
|
|
<th>**YES**</th>
|
|
|
<th>**YES**</th>
|
|
|
<th>**YES**</th>
|
|
|
<th>**YES**</th>
|
|
|
<th>**YES**</th>
|
|
|
<th>**YES**</th>
|
|
|
<th>**YES**</th></tr>
|
|
|
<tr><th>[\#5435 GHCi linker should run constructors for linked libraries](https://gitlab.haskell.org//ghc/ghc/issues/5435)</th>
|
|
|
<th>**YES**</th>
|
|
|
<th>**YES**</th>
|
|
|
<th>**YES**</th>
|
|
|
<th>**YES**</th>
|
|
|
<th>**YES**</th>
|
|
|
<th>**YES**</th>
|
|
|
<th>**YES**</th></tr>
|
|
|
<tr><th>[\#6107 GHCi runtime linker cannot link with duplicate common symbols](https://gitlab.haskell.org//ghc/ghc/issues/6107)</th>
|
|
|
<th>**YES**</th>
|
|
|
<th>**YES**</th>
|
|
|
<th>**YES**</th>
|
|
|
<th>**YES**</th>
|
|
|
<th>**YES**</th>
|
|
|
<th>**YES**</th>
|
|
|
<th>**YES**</th></tr>
|
|
|
<tr><th>[\#7043 32-bit GHC ceiling of negative float SEGFAULT: 11](https://gitlab.haskell.org//ghc/ghc/issues/7043)</th>
|
|
|
<th>no</th>
|
|
|
<th>**YES**</th>
|
|
|
<th>no</th>
|
|
|
<th>no</th>
|
|
|
<th>no</th>
|
|
|
<th>no</th>
|
|
|
<th>no
|
|
|
</th></tr>
|
|
|
<tr><th>[\#7056 GHCi loadArchive "libiconv.a":failed Unknown PEi386 section name \`.drectve'](https://gitlab.haskell.org//ghc/ghc/issues/7056)</th>
|
|
|
<th>no</th>
|
|
|
<th>no</th>
|
|
|
<th>no</th>
|
|
|
<th>no</th>
|
|
|
<th>**probably**</th>
|
|
|
<th>**YES**</th>
|
|
|
<th>no
|
|
|
</th></tr>
|
|
|
<tr><th>[\#7072 GHC interpreter does not find stat64 symbol on Linux](https://gitlab.haskell.org//ghc/ghc/issues/7072)</th>
|
|
|
<th>no</th>
|
|
|
<th>no</th>
|
|
|
<th>**YES**</th>
|
|
|
<th>no</th>
|
|
|
<th>no</th>
|
|
|
<th>no</th>
|
|
|
<th>no
|
|
|
</th></tr>
|
|
|
<tr><th>[\#7097 linker fails to load package with binding to foreign library](https://gitlab.haskell.org//ghc/ghc/issues/7097)</th>
|
|
|
<th>no</th>
|
|
|
<th>no</th>
|
|
|
<th>no</th>
|
|
|
<th>no</th>
|
|
|
<th>**probably**</th>
|
|
|
<th>**YES**</th>
|
|
|
<th>no
|
|
|
</th></tr>
|
|
|
<tr><th>[\#7103 Compiler panic, when loading wxc in GHCi](https://gitlab.haskell.org//ghc/ghc/issues/7103)</th>
|
|
|
<th>no</th>
|
|
|
<th>no</th>
|
|
|
<th>no</th>
|
|
|
<th>no</th>
|
|
|
<th>**probably**</th>
|
|
|
<th>**YES**</th>
|
|
|
<th>no
|
|
|
</th></tr>
|
|
|
<tr><th>[\#7134 ghc-7.6.0.20120810-x86_64-windows.exe -\> internal error R_X86_64_PC32](https://gitlab.haskell.org//ghc/ghc/issues/7134)</th>
|
|
|
<th>no</th>
|
|
|
<th>no</th>
|
|
|
<th>no</th>
|
|
|
<th>no</th>
|
|
|
<th>**YES**</th>
|
|
|
<th>no</th>
|
|
|
<th>no
|
|
|
</th></tr>
|
|
|
<tr><th>[\#7207 linker fails to load package with binding to foreign library (win64)](https://gitlab.haskell.org//ghc/ghc/issues/7207)</th>
|
|
|
<th>no</th>
|
|
|
<th>no</th>
|
|
|
<th>no</th>
|
|
|
<th>no</th>
|
|
|
<th>**YES**</th>
|
|
|
<th>no</th>
|
|
|
<th>no
|
|
|
</th></tr>
|
|
|
<tr><th>[\#7299 threadDelay broken in ghci, Mac OS X](https://gitlab.haskell.org//ghc/ghc/issues/7299)</th>
|
|
|
<th>**YES**</th>
|
|
|
<th>**YES**</th>
|
|
|
<th>no</th>
|
|
|
<th>no</th>
|
|
|
<th>no</th>
|
|
|
<th>no</th>
|
|
|
<th>no
|
|
|
</th></tr>
|
|
|
<tr><th>[\#7357 GHC.exe gives an internal error while linking vector's Monadic.hs](https://gitlab.haskell.org//ghc/ghc/issues/7357)</th>
|
|
|
<th>no</th>
|
|
|
<th>no</th>
|
|
|
<th>no</th>
|
|
|
<th>no</th>
|
|
|
<th>**YES**</th>
|
|
|
<th>no</th>
|
|
|
<th>no
|
|
|
</th></tr></table>
|
|
|
|
|
|
## Bugs in dynamic linking
|
|
|
|
|
|
[ticket:5620](https://gitlab.haskell.org//ghc/ghc/issues/5620)[ticket:8228](https://gitlab.haskell.org//ghc/ghc/issues/8228) |
|
|
\ No newline at end of file |