Automatically unload stale linkables, introduce -funload-strategy

Zubinrequested to merge
wip/unload-strategy into master

Depends on !16582

The main payload is "linker: Automatically reload stale linkables"

Automatically reload stale linkables

Linked BCOs contain direct references to closures, so replacing only a changed module can leave stale references in other modules. Take

-- A.hs
f = 1
-- B.hs
import A
g = f + 1

After loading A and B, change A so that f = 2 and recompile it. B does not need to be recompiled, but its loaded BCO still refers to the old closure for f. The next splice should see f = 2 and g = 3. To get the correct result, we have to reload B, even though we didn't need to recompile it.

The GHC driver calls unload before each upsweep, which unloads everything, so this does not arise during an ordinary GHCi reload. API clients such as HLS may instead want to keep the loader state while recompiling individual modules. Forcing them to unload everything or carefully keep track of what to unload is not ideal.

Now we automatically reload stale linkables, so that if we are asked to link a module whose code has changed, the old code, and everything that refers to it, is dropped and reloaded.

Introduce -funload-strategy

When unloading object code, we have a choice to make. Do we call purgeObj or unloadObj?

purgeObj clears the symbol tables associated with an object, so that future objects can't link against it, but the object stays in memory

unloadObj does the above, but it also marks the object as needing to be unloaded, so at some point in a future GC, the RTS may notice that it is no longer used, and if so, unload it entirely, freeing up the memory.

Ideally we would always unload, but a number of bugs with the implementation of unloadObj mean that it is fragile on many platforms. This is documented in Note [Unloading vs purging objects]. So on these platforms we purge instead.

We introduce the -funload-strategy flag, so that users can opt into purging/unloading on platforms where me make the other choice by default.

The distinction is moot when we are using the dynamic RTS, we don't do either then.

Fixes #27606 Fixes #27740

Merge request reports