Test suite should show reason for skipping a test

I want to add a GHCi test, but it's a bit tricky to run the test so I use a make rule and use run_command, like this:

test('UnsafeReenterGhci',
     [only_ways(['ghci']), exit_code(1), extra_files(['UnsafeReenter.hs', 'UnsafeReenterC.c'])],
     run_command,
     ['$MAKE -s --no-print-directory UnsafeReenterGhci'])

The important part is run_command and only_ways(['ghci']). The problem is that it's impossible to run this test, because by default a run_command test can only be run in normal way (see test_common_work() in testlib.py), but there's no way to know this without reading the source code.

Having better documentation might help, but I think the test driver should print why a test is skipped (maybe only in verbose mode -- if such a mode exists). In our case it should print something like "Test is only run in 'normal' way" (because run_command tests only run in 'normal' way).

Correct version of this test is:

test('UnsafeReenterGhci',
     [extra_ways(['ghci']), only_ways(['ghci']), exit_code(1), extra_files(['UnsafeReenter.hs', 'UnsafeReenterC.c'])],
     run_command,
     ['$MAKE -s --no-print-directory UnsafeReenterGhci'])

The extra_ways makes the test driver consider running the test in ghci way, only_ways is to avoid running it in normal way (the default and only way for run_command).

Trac metadata

Trac field	Value
Version	8.5
Type	Task
TypeOfFailure	OtherFailure
Priority	normal
Resolution	Unresolved
Component	Test Suite
Test case
Differential revisions
BlockedBy
Related
Blocking
CC
Operating system
Architecture