Extend nofib with benchmarks focused on compiler performance

GHC seems to be missing a set of benchmarks focused on the compiler performance that would contain some realistic examples and make it easy to test even small changes in the compiler (by having a simple "before and after" comparison). The two main use cases would be:

• Anyone hacking on GHC could quickly check even for small performance differences (running the benchmark with and without their change).
• Tracking performance of GHC over time (potentially as part of perf.haskell.org)

We already have two collections of performance tests/benchmarks:

• tests/perf/compiler which contains actual tests (part of validate) and seems to be focused on larger regressions (if the bounds are too tight, it can generate a fair amount of noise).
• nofib which is mostly focused on benchmarking the code produced by GHC (the vast majority of the modules compile within 400ms).

The current idea is to extend nofib with some modules that are only checking the performance of GHC (e.g., might be a library module that doesn't contain main).

Please see: https://mail.haskell.org/pipermail/ghc-devs/2016-December/013323.html for an initial discussion about this.

Also related:

• Ticket focused on improvements to tests/perf/compiler #12758 (closed)
• Proposal of having a similar tests but hosted in a separate repo on github: https://github.com/ghc-proposals/ghc-proposals/pull/26

Trac metadata

Trac field	Value
Version
Type	Task
TypeOfFailure	OtherFailure
Priority	normal
Resolution	Unresolved
Component	NoFib benchmark suite
Test case
Differential revisions
BlockedBy
Related
Blocking
CC
Operating system
Architecture