perf_notes.py usage output unhelpful
perf_notes.py script mentions the following options in its
Filtering: Select which subset of performance metrics to dump --test-env TEST_ENV The given test environment to be compared. Use 'local' for locally run results. If using --ci, see .gitlab-ci file for TEST_ENV settings. --test-name TEST_NAME Filters for tests matching the given regular expression. --metric METRIC Test metric (one of ['bytes allocated', 'peak_megabytes_allocated', 'max_bytes_used']). --way WAY Test way (one of ['bytes allocated', 'peak_megabytes_allocated', 'max_bytes_used']).
It took me a bit of time to figure out that the correct use of
--metric is currently something like
--metric 'compile_time/bytes allocated', while
--way is typically
--way normal or
--way optasm. It would be helpful if the
--help output had correct examples and these options were documented (they seem to be missing from https://gitlab.haskell.org/ghc/ghc/-/wikis/building/running-tests/performance-tests, I'm not sure if they are anywhere else?). Ideally the script would provide options to list all the available metrics/ways and report a helpful error on bad input rather than generating empty results.