Testrunner fails to generate compile time stats when a test is set to compile_and_run
If we want to collect both compile time and runtime metrics from a single test, the test runner will currently fail.
Conversions test in
test('Conversions', [collect_stats('bytes allocated',3), collect_compiler_stats('bytes allocated',3), only_ways(['normal']) ], compile_and_run, ['-O -msse2'])
then run the test:
$ ./hadrian/build.sh test --only="Conversions"
The test should pass and collect both compile time and runtime metrics. Once !1268 (closed) is merged, you can check this by looking for both "runtime/" and "compile_time/" in your git notes after running the test (make sure to commit all changes to a new commit so you have a clean working tree else the metrics will not we saved):
$ git notes --ref perf show HEAD
The testrunner crashes complaining about a missing stats file:
/run/user/1000/ghctest-bc3dfj9q/test spaces/testsuite/tests/perf/should_run/Conversions.run Conversions [[Errno 2] No such file or directory: '/run/user/1000/ghctest-bc3dfj9q/test spaces/testsuite/tests/perf/should_run/Conversions.run/Conversions.stats'] (normal)
In the .T file we specify
compile_and_run. I'm guessing we have some offending logic that means only the runtime stats file will be generated in this case. When using
compile instead of
compile_and_run, there is still a crash but I assume now we are only generating compile time stats and not runtime stats. In short, the fix will likely support generating compile and runtime stats when using the