More clearly label performance changes in CI output
On a recent run of the CI pipeline (https://gitlab.haskell.org/ghc/ghc/-/jobs/325228), I see text like
Naperian(optasm) compile_time/bytes allocated 59467056.000
(baseline @ HEAD~1) 59375309.333 [unchanged 0.2%]
PmSeriesG(normal) compile_time/bytes allocated 64017488.000
(baseline @ HEAD~1) 73854008.000 [decreased 13.3%]
PmSeriesS(normal) compile_time/bytes allocated 73890464.000
(baseline @ HEAD~1) 77177032.000 [unchanged 4.3%]
The patch fails the PmSeriesG
test, because the change is beyond the acceptable threshold. (In this case: good news, but that's not relevant for this new ticket.) But notice the other two: Naperian
got a tiny bit worse and PmSeriesS
got more than a tiny bit better. Yet the gloss on the right looks the same, both are unchanged
.
I suggest a tiny tweak to this output: when reporting unchanged
, use a negative percentage to indicate decrease and a positive one to indicate increase. That way, it would be easier for me to scan this output and get a general sense whether these small changes add up to good news or bad news.