Extend nofib with benchmarks focused on compiler performance
GHC seems to be missing a set of benchmarks focused on the compiler performance that would contain some realistic examples and make it easy to test even small changes in the compiler (by having a simple "before and after" comparison). The two main use cases would be:
- Anyone hacking on GHC could quickly check even for small performance differences (running the benchmark with and without their change).
- Tracking performance of GHC over time (potentially as part of perf.haskell.org)
We already have two collections of performance tests/benchmarks:
tests/perf/compilerwhich contains actual tests (part of
validate) and seems to be focused on larger regressions (if the bounds are too tight, it can generate a fair amount of noise).
nofibwhich is mostly focused on benchmarking the code produced by GHC (the vast majority of the modules compile within 400ms).
The current idea is to extend
nofib with some modules that are only checking the performance of GHC (e.g., might be a library module that doesn't contain
Please see: https://mail.haskell.org/pipermail/ghc-devs/2016-December/013323.html for an initial discussion about this.
- Ticket focused on improvements to
- Proposal of having a similar tests but hosted in a separate repo on github: https://github.com/ghc-proposals/ghc-proposals/pull/26
|Component||NoFib benchmark suite|