Allowed metric increases are calculated incorrectly in marge batches
I have no idea how this has ever worked but when a marge batch is created it appears that only the Metric changes indicated in the HEAD commit are allowed to increase, the messages in the other commits are ignored.
How?
-
config.allowed_perf_changes
is set toPerf.get_allowed_perf_changes()
, which gets the allowed changes forHEAD
. - This doesn't take into account the merge base at all or the PERF_BASELINE_COMMIT value.
I suppose the correct thing to do is to gather all the metric changes from the commits between PERF_BASELINE_COMMIT
and HEAD
.