Skip to content

WIP: while chasing dependencies don't check modules which already failed again

David Hewson requested to merge dten/ghc:chasing-speed into master

backstory: I was investigating why "chasing dependencies" seemed so slow (specifically I'm using WSL). We have a project where the dev loop involves loading ~900 modules into GHCi, make and edit, :r and run tests in ghci etc. On my machine it was much slower than other machines, so tried to investigate what was happening during that reloading. -dshow-passes showed that it was spending 40 seconds on "Chasing dependencies" for a :r where nothing had changed. Some investigating showed it was getting module summaries, so i decided it was one of our modules with some crazy case making it take so long. Adding some logging revealed that actually it spent most of its time querying module summaries for things such as "Data.Text" "Data.ByteString", modules that weren't our modules (in fact coming from stack). What seemed worse though it was asking for each of these hundreds of times. Each time deciding that they didn't exist, but it would try again later anyway. The failures are mostly not actual failures, they're just failure to find source ghc/compiler/main/GhcMain.hs:2455 decides that if we found a module but it has no ml_hs_file in the location then pretend we found Nothing.

Change: This change makes it remember that it didn't exist (for a given downsweep call anyway) this cut my chasing dependencies time to 3.5 seconds from 40. So far I've put this down to poor file system performance on WSL's part, but GHC could be more helpful when the file system isn't perfect. I've yet to get the timing difference for someone on a better file system.

Style: This currently is a minimal lines change, but I feel it would be more appropriate for summariseModule to return more detail about what it found, and loop to remember that detail instead of turning everything into Nothing

Merge request reports