While looking at #20493 I noticed that code loaded by the does not have its `fini` destructors executed. For foreign libraries loaded in GHCi this could result in resource leaks and potential mis-behavior.