Skip to content
GitLab

driver: Add timing information to upsweep and some simple analysis scripts

Open Matthew Pickering requested to merge wip/par-stats into master

This commit adds a new flag -ddump-make-stats which shows some statistics about the project build graph after compilation has finished. These can be useful identifying bottlenecks in your projects module structure.

The statistics which are currently outputted are:

  • The modules which took longest to compile.
  • The modules which have the largest "flow". The initial flow is 1, and split evenly between all roots of the dependency graph. The flow is propagated through the graph, accumulated on each node and split evenly on children. The result is that any synchronisation points will have a flow equal to 1, and likewise other important modules will have a high flow value.
  • The length of the longest (critical) path through the project. This provides a lower bound on the projects compilation time.
  • The "parallelism score" which is the sum of compiling all nodes divided by the length of the critical path. This should be a more stable metric then critical path length because it doesn't depend on how fast your computer is.

For example, here is an example of the output from compiling the Cabal library.

===== Maximum Duration (s) =====
000 M: main:Distribution.Simple.Setup (59): time: 1.40 allocs: 1489.93
001 M: main:Distribution.PackageDescription.Check (82): time: 0.68 allocs: 732.85
...
104 M: main:Distribution.Compat.GetShortPathName (9): time: 0.00 allocs: 3.62
105 M: main:Distribution.Compat.FilePath (8): time: 0.00 allocs: 3.46
===== Maximum Flows =====
000 M: main:Distribution.Simple (105): 1.000
001 M: main:Distribution.Simple.Configure (97): 0.346
...
104 M: main:Distribution.Simple.Program.Types (50): 0.002
105 M: main:Distribution.Simple.GHC.ImplInfo (46): 0.000
===== Flows x Time =====
000 M: main:Distribution.Simple (105): 0.175
001 M: main:Distribution.Simple.Configure (97): 0.127
...
104 M: main:Distribution.Backpack.PreExistingComponent (4): 0.000
105 M: main:Distribution.Simple.GHC.ImplInfo (46): 0.000
===== Statistics =====
longest path: 4.291s
parallelism score: 2.247
sequential time: 9.642s

In addition to this, the build graph is also emitted to the eventlog. For each node in the build graph, an event is emitted to the eventlog of the form

node: { "node_id": 0, "node_deps": [0, 1,2,3], "node_desc": "GHC.Driver.Make" }

this allows external tooling to easily reconstruct the actual build graph used by GHC and analyse it using external tools.

Edited by sheaf