Fix comment typos
The below is only necessary to fix the CI perf fluke that happened in 9897e8c8: ------------------------- Metric Decrease: T5837 T6048 T9020 T12425 T12234 T13035 T12150 Naperian -------------------------
-
mentioned in merge request !2192 (closed)
-
Developer
This is madness
-
Developer
Why do comment changes cause metric changes?
-
Developer
See !2192 (comment 245745). I'm in the process of reading the discussion in !1742 (closed), but it's mostly wandering in the dark wrt. where the regressions come from. Ultimately (!1742 (comment 241777)) we just gave up when we found that this commit "fixes" things again. See the spike in this chart.
-
Maintainer
Amazing. At the very least the commit message should've mentioned in BIG LETTERS in the very first line that this fixes perf issues.
I was never satisfied with it completely but I think this is one data point that shows updating perf test results via commit messages was not a good idea. We don't do it for correctness tests, why do it for perf tests? I never understood the motivation (but I was also not involved in the initial design so maybe I'm missing something).
-
Author Developer
This commit doesn't fix anything, it simply works around a flaw in the perf testing infrastructure that tends to mismeasure certain commits. Nobody knows why that happens one but not the other.
-
Developer
I'll just repeat what I wrote on twitter:
I'm not opposed to updating perf tests in commit messages. Sometimes there are increases caused by a bug fix that crucially needs to do more because of correctness. And performance should never be a reason to reject correctness.
There should be a summary listing the increases since the last release, though, so that after ~2000 intermediate commits we can have an informed decision whether a particular increase was problematic.
I mean, the ultimate issue here is one where we distrust our CI infrastructure enough to just accept a metric increase, simply because we have no clue what regressed and it's not possible to reproduce locally. I've been there multiple times (many times unjustified, because in fact there was a bug in my code) and find that's a terrible situation! Clearly, we have to find a way to trust CI again.
-
mentioned in issue #17686 (closed)