CI/Testing: ./validate vs --flavour=validate vs `.validate-linux:` on CI
I found out today that some of our platforms are compiled with -O2 and some with -O. Which again potentially differs from local testing.
./validate
Local testing: validate runs
hadrian -j$THREADS
, where by defaultTHREADS
is the number of cpus your computer has +1
Which based on https://gitlab.haskell.org/ghc/ghc/-/blob/master/hadrian/doc/flavours.md means we compile stage1+ with -O2.
That's fair and sane. If someone just builds GHC they should get an optimized compiler. We release with -O2.
hadrian: --flavour=validate
Naturally I assumed that --flavour=validate would build an compiler just as ./validate would. Just without the validating step. This is not true. Based on the current flavour we instead build libraries and ghc get built with -O.
CI:
In CI we, for the most part, call hadrian explicitly (not relying on ./validate) and use the validate flavour. This results in testing a compiler built with -O1.
However CI builds based on .validate-linux:
do not. There we use the perf
flavour and as a consequence we test the compiler as built with -O2.
This opens up a few questions.
- Should ./validate build the same compiler --flavour=validate does?
- Should that compiler be built using -O2?
Side note: Maybe it was naive of me to make that assumption but I always had assumed that validate would test the compiler with release settings (-O2). This seems was not the case with make, but is now the case for hadrian based ./validate. But is not the case for the --flavour=validate setting.
I suggest with change the --flavour=validate setting to build a compiler that is built using -O2, and use this setting both in ./validate
and CI consistently.
Then ./validate, --flavour=validate and CI would test the compiler using the same settings.
The obvious downside is longer compile times as we know use -O2 for full builds, where some of them currently only build with -O1.