Using Make to run tests and test T14999
Test T14999 is skipped:
./hadrian/build -j --flavour=devel2 --freeze1 test --only=T14999
It is marked as unless(... have_gdb() and have_readelf(), skip)
. But I do have gdb and readelf. So what's going on?
The only place where we detect gdb and readelf is in test.mk
ifeq "$(HAVE_READELF)" "YES"
RUNTEST_OPTS += -e config.have_readelf=True
else
RUNTEST_OPTS += -e config.have_readelf=False
endif
Looks like this was not ported to Hadrian. Using the old method make test TEST=T14999
works (although it tries to use my local ghc instead of the one in _build
).
The good news is that T14999 is the only test using either have_gdb()
or have_readelf()
. The bad news is that there is plenty of other logic in test.mk
and I'm not sure what else is missing.
Proposal:
-
Port all logic initializing
RUNTEST_OPTS
to Hadrian. -
Remove the
test
target from Makefile. The Python testdriver can call Make (makefile_test
) but Make shouldn't be calling Python. I see that we still use Make in.gitlab-ci.yml
:make -Ctestsuite list_broken TEST_HC=$GHC
, this would have to be changed. Is Make used anywhere else? -
The test runner initializes configuration to dummy values
config.have_readelf=False
https://gitlab.haskell.org/ghc/ghc/-/blob/257f1567395be441ebf7ada996e4edf36abbe7e9/testsuite/driver/testglobals.py#L172. IMO, we should avoid this as much as possible - if a parameter wasn't passed, the test runner should crash. Otherwise tests can get silently skipped like in this case.