Testsuite: run each test in its own /tmp directory, after copying required files
Plan
For each TEST:
- create a directory
<testdir>
inside/tmp
- link/copy all source files that the test needs into
<testdir>
- run the test inside
<testdir>
- delete
<testdir>
Benefits
- running tests in parallel always works, even when two tests write to the same file (for example .hi/.o files, from sharing a .hs file). This should reduce the number of unsuspected and annoying validate failures. There is no longer any need to specify
-outputdir
explicitly. - there is no longer any need to add entries to the perpetually out-of-date
testsuite/.gitignore
. - there is no longer any need to specify
extra_clean
orclean_cmd
setup functions for tests. Sometimes you //will// have to specify which extra files a test needs, using the (new)extra_files
setup function, but it will be immediately clear when to do so (i.e. the test won't run if you don't). By default only files with the name<testname>*
are copied to<testdir>
. - it could become possible to run //ways// for a single test in parallel (i.e.
make TEST=<testname> slow
would become faster) -
clean
ing the testsuite won't be necessary (except for a few files intestsuite/mk
perhaps)
Currently the only way to clean the testsuite is via make CLEANUP=1 CLEAN_ONLY=1
, which requires a call to the testsuite, which requires building mk/ghc-config first. This will all become much simpler.