GitLab

They are a particular type of perf tests. This patch introduces a
'stats_files_dir' configuration field in the testsuite driver where all
haddock timing files (and possibly others in the future) are assumed to live.
We also change both the Make and Hadrian build systems to pass respectively
$(TOP)/testsuite/tests/perf/haddock/ and
<build root>/stage1/haddock-timing-files/ as the value of that new
configuration field, and to generate the timing files in those directories
in the first place while generating documentation with haddock.

This new test type can be seen as one dedicated to examining stats files that
are generated while building a GHC distribution. This also lets us get rid of
the 'extra_files' directives in the all.T entries for haddock.base,
haddock.Cabal and haddock.compiler.

We can specify several of those, by using the flag multiple times or
just once but combining the directories with ':'.

Along the way, this patch also fixes the testsuite-related --only flag,
so that we can use it many times instead of being force to specify a
space-separated list of test in a single --only flag.

4 haddock tests assume that .haddock files have been produced, by using the
'req_haddock' modifier. The testsuite driver assumes that this condition is
satisfied if 'config.haddock' is non-empty, but before this patch Hadrian was
always passing the path to where the haddock executable should be, regardless
of whether it is actually there or not.

Instead, we now pass an empty config.haddock when we can't find all of
<build root>/docs/html/libraries/<pkg>/<pkg>.haddock>, where <pkg> ranges over
array, base, ghc-prim, process and template-haskell, and pass the path
to haddock when all those file exists. This has the (desired) effect of skipping
the 4 tests (marked as 'missing library') when the docs haven't been built,
and running the haddock tests when they have.

We used to pass YES/NO, while that particular field is set to True/False. This
happens to fix an unexpected pass, T9208.

When -a or --test-accept is passed, and if one runs the 'test' target, then
any test failing because of mismatching output and which is not expected to
fail will have its expected output adjusted by the test driver, effectively
considering the new output correct from now on.

When this flag is passed, hadrian's 'test' target becomes sensitive to the
PLATFORM and OS environment variable, just like the Make build system:
- when the PLATFORM env var is set to "YES", when accepting a result, accept it
  for the current platform;
- when the OS env var is set to "YES", when accepting a result, accept it
  for all wordsizes of the current operating system.

This can all be combined with `--only="..."` and `TEST="..." to only accept
the new output of a subset of tests.

This moves all URL references to Trac tickets to their corresponding
GitLab counterparts.

- introduce a -k/--keep-test-files flag to prevent cleanup
- add -dstg-lint to the options that are always passed to tests
- infer library ways from the compiler to be tested instead of getting them
  from the flavour (like make)
- likewise for figuring out whether the compiler to be tested is "debugged"
- specify config.exeext
- correctly specify config.in_tree_compiler, instead of always passing True
- fix formatting of how we pass a few test options
- add (potential) extensions to check-* program names
- build check-* programs with the compiler to be tested
- set TEST_HC_OPTS_INTERACTIVE and PYTHON env vars when running tests

This reverts commit 442be690.

This is due to the failures documented in #16087. The condition here could be
improved as it matches on `BUILD_FLAVOUR` instead of looking at the compiler
flags. However, it's better than nothing and I hope we will be able to fix these
issues before long.

This was supposed to be working already but didn't work when we
specified several tests with --only. This patch not only fixes this
but also makes it possible to specify a subset of tests to run with the
TEST environment variable, like the make build system. Here are some
examples:

    hadrian/build.sh test --only=plugins01
    hadrian/build.sh test --only="plugins01 plugins02"
    TEST="plugins01 plugins02" hadrian/build.sh test
    TEST=plugins03 hadrian/build.sh test --only="plugins01 plugins02"

When both the TEST environment variable and the --only flag are used,
we simply concatenate the list of tests from both sources and ask the
testsuite driver to run them all.

This patch addresses #16026.

Test Plan: hadrian/build.sh test --only="plugins01 plugins02"

Reviewers: bgamari, snowleopard

Reviewed By: bgamari, snowleopard

Subscribers: rwbarton, carter

GHC Trac Issues: #16026

Differential Revision: https://phabricator.haskell.org/D5431