Perf metrics: Hadrian unnecessarily quotes the names of test_envs
While trying to figure out in !3954 (closed) why putting Metric Decrease (test_env='x86_64-linux-deb9-unreg-hadrian')
in my commit message wasn't accepting the metric decrease on a Hadrian-based CI configuration, @monoidal pointed out that there was actually a distinction in how CI was printing out the test environments on Hadrian-based versus non-Hadrian-based configurations. Compare the header from a Hadrian-based build (example):
Performance Metrics (test environment: "x86_64-linux-deb9-unreg-hadrian"):
Against the header of a non-Hadrian-based build (example):
Performance Metrics (test environment: x86_64-darwin):
Notice how the former has double quotes around the name of the test environment, but the latter does not! Indeed, this discrepancy explains why Hadrian wasn't responding properly to Metric Decrease (test_env='x86_64-linux-deb9-unreg-hadrian')
, since it was looking for the name "x86_64-linux-deb9-unreg-hadrian"
, not x86_64-linux-deb9-unreg-hadrian
.
This patch is enough to fix the issue:
--- a/hadrian/src/Settings/Builders/RunTest.hs
+++ b/hadrian/src/Settings/Builders/RunTest.hs
@@ -142,7 +142,7 @@ runTestBuilderArgs = builder RunTest ? do
, case perfBaseline of
Just commit | not (null commit) -> arg ("--perf-baseline=" ++ show commit)
_ -> mempty
- , emitWhenSet testEnv $ \env -> arg ("--test-env=" ++ show env)
+ , emitWhenSet testEnv $ \env -> arg ("--test-env=" ++ env)
, emitWhenSet testMetricsFile $ \file -> arg ("--metrics-file=" ++ file)
, getTestArgs -- User-provided arguments from command line.
]
MR incoming.