Sort out devel2 and Hadrian

I like to use devel2 to develop GHC, as it offers these two very nice benefits:

1. Quick rebuilds (especially when I skip dependency generation)
2. Assertions enabled

With make, running the testsuite on devel2 leads to <= 1 spurious failures in my setup (NixOS). The last time I tested Hadrian (about 10 days ago), I got 7 spurious failures. These extra failures stop me from using Hadrian, and are why I still use make -- though I'd be happy to switch.

In an effort to convince me away from devel2, @mpickering wrote to ghc-devs

In semi-private I have been making very bold claims about the unsuitability of the devel2 as a build flavour of choice for the aspiring developer.

In order to bolster my claim I set out to find out some statistics about

1. devel2 flavour
2. default+no_profiled_libs+omit_pragmas+assertions flavour

The difference between the two is essentially that flavour (2) builds each module with optimisation but disables cross-module optimisation (in particular inlining)

In short the difference between the two flavours is negligible from my testing apart from one larger difference when building larger packages.

Full build + test: Near identical Testsuite run: Near identical Recompile: Near identical - (2) is slower Build Cabal: 167s vs 259s

So if you want to build packages use flavour 2 but otherwise it seems to make little difference.

I found this very helpful, but I was not fully satisfied, writing this in response:

These results surprise me, in a few ways:

• My understanding is that devel2 does not optimise (-O0) the stage-2 compiler. So I'm surprised that the testsuite runs in the same amount of time for both: I would expect (2) to be measurably faster.
• And if the testsuite is the same for both, then I would expect "Build Cabal" to be the same for both: both measurements are looking at the performance of the stage-2 compiler. Yet the difference in time in "Build Cabal" is drastic. Why is this not echoed in the testsuite? (Maybe the testsuite overhead dwarfs the efficiency of the compiler itself?)
• As we have discovered, devel2 behaves differently in the testsuite than your flavour (2). But your text suggests that the only difference is that devel2 is completely unoptimised... and yet the testsuite-behavior differences are not (I think) about performance. So there's some other difference.

To my knowledge, these questions remain unanswered, and cast doubt on @mpickering's claims above.

In any case, I'm posting this ticket as a way to forge forward toward some solution. In my mind, a "solution" is a build flavour (preferably a named one, instead of the agglomeration of 4 other settings, but I don't need to be picky) that

1. rebuilds quickly;
2. has assertions enabled; and
3. is tested as part of our CI infrastructure

One solution would be testing devel2 in CI. Another solution would be to convince me that the combination (@mpickering's flavour (2), above) indeed satisfies goals (1) and (2) -- and then get rid of devel2 entirely (or, better, change its meaning).