Skip to content

Make mode should prioritise modules with many reverse depedencies

Motivation

graph

Suppose we have a module graph that we are compiling with -j2, as in the diagram above. Assume that each module takes the same amount of time to compile. We want to make sure that B and C don't get compiled before A, since then when A gets compiled 1 of 2 cores will be left with no work to do (as D and E can't be started until A finishes).

As far as I can tell, GHC often runs into situations like this. It cannot efficiently use the parallelism it is granted, since it doesn't prioritise work that enables future work.

Proposal

A relatively easy way to improve the situation would be for workers to always compile the available module with the most transitive reverse dependencies.

This is a greedy strategy that tries to maximise the amount of modules available to compile, hoping that this backlog can weather us through bottlenecks in the module graph.

To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information