GitLab

We want a lightweight way to detect when either (a) the compiler's performance changes, or (b) the performance of compiled code changes. This was triggerred by my noticing that the HEAD nightly builds seem somewhat slower than usual right now.

So the idea is pretty simple. We decide on a fixed set of nofib programs --intiially all of them, but we want the set to remain stable in the future -- and as part of the nightly build we time how long it takes to compile this set with a fixed set of flags (probably -O -fasm, to rule out changes in gcc). Then we time how long it takes to run all those programs, maybe 5 times each. Then we keep track of these two figures for every successful nightly build, and plot the results, giving us a nice way to track GHC's performance over time. We can go back and get some historical figures too. Of course the results are specific to a particular machine, so we'd have to keep track of results per nightly build machine or something.

So basically: some small changes to nofib to allow "just build everything" (I think even NoFibRuns=0 might do this), and to build a fixed set of programs (the set could be built into the nofib build system, no need to make it dynamic). And some way to collect the results and plot graphs.

Trac metadata

Trac field	Value
Version	6.6
Type	Task
TypeOfFailure	OtherFailure
Priority	normal
Resolution	Unresolved
Component	Build System
Test case
Differential revisions
BlockedBy
Related
Blocking
CC
Operating system	Unknown
Architecture	Unknown