Load Haskell libraries more lazily in TemplateHaskell/ghci/etc
Motivation
Currently GHC loads shared libraries from -package
or -package-id
directives eagerly. That is, for modes like -XTemplateHaskell
or --interactive
, GHC looks for all packages specified via -package{-id}
and loads their shared libraries at startup, whether or not they're actually needed.
We have found a few downsides of this approach:
-
If a file uses TemplateHaskell then GHC loads every transitive dependency, even when only a small subset of the imported modules are needed. For example, pandoc has many transitive dependencies but only uses TH to generate a few JSON instances.
-
When you use ghci, the shared libraries aren't needed if all you want to do is typecheck or
:browse
code. However, ghci loads those shared libraries unconditionally. -
We found it harder to get shared libraries for some of our C/C++ dependencies. In reality, nothing from our TH splices used those libraries. Unfortunately, GHC's eager nature made it try to load them regardless of whether they were needed.
As it turns out, GHC is capable of loading shared libraries more lazily. In fact, if you only pass -package-db
and not -package
, the package will be loaded lazily; it can even be directly imported, as long as it was registered with exposed: True
. However, most build systems nowadays use -hide-all-packages
which has better behavior; for example, it allows multiple transitive deps of the same package name but different versions.
Internally, our team uses the following simple (2-line) patch: e4ffc94b
The downside of that patch is that it breaks the following use-case:
ghci -package foo -lbar
When running code in ghci, GHC allows the C library bar
to use symbols defined in the Haskell library foo
. This note documents it in more detail:
https://gitlab.haskell.org/ghc/ghc/blob/master/compiler/GHC/Runtime/Linker.hs#L390
With the proposed change, foo
would instead of use symbols from bar
.
Proposal
I propose adding a new flag to GHC to make it load shared libraries from -package
/-package-id
lazily (as in the above patch). We'd probably want to turn that flag on by default in tools such as Cabal and Stack. As far as I can tell they don't rely on the edge case I called out above.
Actually, to be honest, I'm not really aware of any code that relies on that behavior anymore. If so, maybe we should just enable the flag by default? (Note that a couple of GHC tests do correctly fail under the above patch).
If this is better-suited as a GHC proposal, let me know and I can file it there. I wanted to double-check first since this could end up as a fairly minor change.