Skip to content

Using Make to run tests and test T14999

Test T14999 is skipped:

./hadrian/build -j --flavour=devel2 --freeze1 test --only=T14999

It is marked as unless(... have_gdb() and have_readelf(), skip). But I do have gdb and readelf. So what's going on?

The only place where we detect gdb and readelf is in test.mk

ifeq "$(HAVE_READELF)" "YES"
RUNTEST_OPTS += -e config.have_readelf=True
else
RUNTEST_OPTS += -e config.have_readelf=False
endif

Looks like this was not ported to Hadrian. Using the old method make test TEST=T14999 works (although it tries to use my local ghc instead of the one in _build).

The good news is that T14999 is the only test using either have_gdb() or have_readelf(). The bad news is that there is plenty of other logic in test.mk and I'm not sure what else is missing.

Proposal:

  1. Port all logic initializing RUNTEST_OPTS to Hadrian.

  2. Remove the test target from Makefile. The Python testdriver can call Make (makefile_test) but Make shouldn't be calling Python. I see that we still use Make in .gitlab-ci.yml: make -Ctestsuite list_broken TEST_HC=$GHC, this would have to be changed. Is Make used anywhere else?

  3. The test runner initializes configuration to dummy values config.have_readelf=False https://gitlab.haskell.org/ghc/ghc/-/blob/257f1567395be441ebf7ada996e4edf36abbe7e9/testsuite/driver/testglobals.py#L172. IMO, we should avoid this as much as possible - if a parameter wasn't passed, the test runner should crash. Otherwise tests can get silently skipped like in this case.

To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information