perf_notes.py usage output unhelpful
The perf_notes.py
script mentions the following options in its --help
output:
Filtering:
Select which subset of performance metrics to dump
--test-env TEST_ENV The given test environment to be compared. Use 'local'
for locally run results. If using --ci, see .gitlab-ci
file for TEST_ENV settings.
--test-name TEST_NAME
Filters for tests matching the given regular
expression.
--metric METRIC Test metric (one of ['bytes allocated',
'peak_megabytes_allocated', 'max_bytes_used']).
--way WAY Test way (one of ['bytes allocated',
'peak_megabytes_allocated', 'max_bytes_used']).
It took me a bit of time to figure out that the correct use of --metric
is currently something like --metric 'compile_time/bytes allocated'
, while --way
is typically --way normal
or --way optasm
. It would be helpful if the --help
output had correct examples and these options were documented (they seem to be missing from https://gitlab.haskell.org/ghc/ghc/-/wikis/building/running-tests/performance-tests, I'm not sure if they are anywhere else?). Ideally the script would provide options to list all the available metrics/ways and report a helpful error on bad input rather than generating empty results.