Skip to content

testsuite: Don't consider broken tests to stat measurements

Ben Gamari requested to merge wip/T17987 into master

Previously we would add statistics from tests marked as broken to the stats output. This broke in #17987 since the test was considered to be "broken" solely on the basis of its allocations. In later testsuite runs the "broken" allocations metric was then considered to be the baseline and the test started unexpectedly passing.

We now ignore metrics that arise from tests marked as broken. Of course, this required that we distinguish between "broken" and merely "expected to fail". I took this opportunity to do a bit of refactoring in our representation of test outcomes.

Merge request reports