Improve reachability queries on ModuleGraph
Introduces `ReachabilityIndex`, an index constructed from a `GHC.Data.Graph.Directed` `Graph` that supports fast reachability queries (in $O(1)$). This abstract data structure is exposed from `GHC.Data.Graph.Directed.Reachability`. This index is constructed from the module graph nodes and cached in `ModuleGraph`, enabling efficient reachability queries on the module graph. Previously, we'd construct a Map of Set of ModuleGraph nodes which used a lot of memory (`O(n^2)` in the number of nodes) and cache that in the `ModuleGraph`. By using the reachability index we get rid of this space leak in the module graph -- even though the index is still quadratic in the number of modules, it is much, much more space efficient due to its representation using an IntMap of IntSet as opposed to the transitive closure we previously cached. In a memory profile of MultiLayerModules with 100x100 modules, memory usage improved from 6GB residency to 2.8GB, out of which roughly 1.8GB are caused by a second space leak related to ModuleGraph. On the same program, it brings compile time from 7.5s to 5.5s. Note how we simplify `checkHomeUnitsClosed` in terms of `isReachableMany` and by avoiding constructing a second graph with the full transitive closure -- it suffices to answer the reachability query on the full graph without collapsing the transitive closure completely into nodes. Unfortunately, solving this leak means we have to do a little bit more work since we can no longer cache the result of turning vertex indices into nodes. This results in a slight regression in MultiLayerModulesTH_Make, but results in large performance and memory wins when compiling large amounts of modules. ------------------------- Metric Decrease: mhu-perf Metric Increase: MultiLayerModulesTH_Make -------------------------
parent
7bd407a6
No related branches found
No related tags found
Pipeline #103512 canceled
Stage: tool-lint
Stage: quick-build
Stage: full-build
Stage: packaging
Stage: testing
Stage: deploy
Showing
- compiler/GHC/Data/Graph/Directed.hs 12 additions, 150 deletionscompiler/GHC/Data/Graph/Directed.hs
- compiler/GHC/Data/Graph/Directed/Internal.hs 79 additions, 0 deletionscompiler/GHC/Data/Graph/Directed/Internal.hs
- compiler/GHC/Data/Graph/Directed/Reachability.hs 168 additions, 0 deletionscompiler/GHC/Data/Graph/Directed/Reachability.hs
- compiler/GHC/Driver/Make.hs 15 additions, 18 deletionscompiler/GHC/Driver/Make.hs
- compiler/GHC/Linker/Deps.hs 8 additions, 9 deletionscompiler/GHC/Linker/Deps.hs
- compiler/GHC/Unit/Module/Graph.hs 31 additions, 18 deletionscompiler/GHC/Unit/Module/Graph.hs
- compiler/ghc.cabal.in 2 additions, 0 deletionscompiler/ghc.cabal.in
- testsuite/tests/count-deps/CountDepsAst.stdout 1 addition, 0 deletionstestsuite/tests/count-deps/CountDepsAst.stdout
- testsuite/tests/count-deps/CountDepsParser.stdout 2 additions, 0 deletionstestsuite/tests/count-deps/CountDepsParser.stdout
Loading