Add deduplication table for `IfaceType`
The type `IfaceType` is a highly redundant, tree-like data structure. While benchmarking, we realised that the high redundancy of `IfaceType` causes high memory consumption in GHCi sessions when byte code is embedded into the `.hi` file via `-fwrite-if-simplified-core` or `-fbyte-code-and-object-code`. Loading such `.hi` files from disk introduces many duplicates of memory expensive values in `IfaceType`, such as `IfaceTyCon`, `IfaceTyConApp`, `IA_Arg` and many more. We improve the memory behaviour of GHCi by adding an additional deduplication table for `IfaceType` to the serialisation of `ModIface`, similar to how we deduplicate `Name`s and `FastString`s. When reading the interface file back, the table allows us to automatically share identical values of `IfaceType`. To provide some numbers, we evaluated this patch on the agda code base. We loaded the full library from the `.hi` files, which contained the embedded core expressions (`-fwrite-if-simplified-core`). Before this patch: * Load time: 11.7 s, 2.5 GB maximum residency. After this patch: * Load time: 7.3 s, 1.7 GB maximum residency. This deduplication has the beneficial side effect to additionally reduce the size of the on-disk interface files tremendously. For example, on agda, we reduce the size of `.hi` files (with `-fwrite-if-simplified-core`): * Before: 101 MB on disk * Now: 24 MB on disk This has even a beneficial side effect on the cabal store. We reduce the size of the store on disk: * Before: 341 MB on disk * Now: 310 MB on disk Note, none of the dependencies have been compiled with `-fwrite-if-simplified-core`, but `IfaceType` occurs in multiple locations in a `ModIface`. We also add IfaceType deduplication table to .hie serialisation and refactor .hie file serialisation to use the same infrastrucutre as `putWithTables`. Bump haddock submodule to accomodate for changes to the deduplication table layout and binary interface.
Showing
- compiler/GHC/Iface/Binary.hs 113 additions, 27 deletionscompiler/GHC/Iface/Binary.hs
- compiler/GHC/Iface/Ext/Binary.hs 76 additions, 90 deletionscompiler/GHC/Iface/Ext/Binary.hs
- compiler/GHC/Iface/Recomp/Binary.hs 7 additions, 2 deletionscompiler/GHC/Iface/Recomp/Binary.hs
- compiler/GHC/Iface/Type.hs 80 additions, 32 deletionscompiler/GHC/Iface/Type.hs
- compiler/GHC/StgToJS/Object.hs 1 addition, 1 deletioncompiler/GHC/StgToJS/Object.hs
- compiler/GHC/Utils/Binary.hs 155 additions, 17 deletionscompiler/GHC/Utils/Binary.hs
- utils/haddock 1 addition, 1 deletionutils/haddock
Loading
Please register or sign in to comment