Hadrian: various improvements around the 'test' rule

• introduce a -k/--keep-test-files flag to prevent cleanup
• add -dstg-lint to the options that are always passed to tests
• infer library ways from the compiler to be tested instead of getting them from the flavour (like make)
• likewise for figuring out whether the compiler to be tested is "debugged"
• specify config.exeext
• correctly specify config.in_tree_compiler, instead of always passing True
• fix formatting of how we pass a few test options
• add (potential) extensions to check-* program names
• build check-* programs with the compiler to be tested
• set TEST_HC_OPTS_INTERACTIVE and PYTHON env vars when running tests

---

For the record, while this branch allows more tests to run, we get the exact same number of unexpected failures as on master, and only one additional unexpected pass. I ran rm -rf _build; ./boot; ./configure; hadrian/build.sh -j4 test on both branches, from a clean state.

On master:

Unexpected results from:
TEST="T1288_ghci T15633a T15633b T15858 T16104 T16260 T2014 T2276_ghci T2386 T3807 T7022 T7040_ghci T7445 T9208 annth_compunits haddockHoogleTest haddockHtmlTest haddockHypsrcTest haddockLatexTest path_with_commas"

SUMMARY for test run started at Tue Mar  5 15:31:21 2019 CET
 0:23:43 spent to go through
    6863 total tests, which gave rise to
   26454 test cases, of which
   19613 were skipped

      29 had missing libraries
    6701 expected passes
      91 expected failures

       0 caused framework failures
       0 caused framework warnings
       1 unexpected passes
      19 unexpected failures
       0 unexpected stat failures

Unexpected passes:
   testsuite/tests/stranal/should_compile/T9208.run  T9208 [unexpected] (optasm)

Unexpected failures:
   testsuite/tests/annotations/should_compile/th/annth_compunits.run  annth_compunits [bad exit code] (normal)
   testsuite/tests/dynlibs/T3807.run                                  T3807 [bad exit code] (normal)
   testsuite/tests/ffi/should_run/T1288_ghci.run                      T1288_ghci [bad exit code] (ghci)
   testsuite/tests/ffi/should_run/T2276_ghci.run                      T2276_ghci [bad exit code] (ghci)
   testsuite/tests/ghci/should_run/T15633a.run                        T15633a [bad exit code] (ghci)
   testsuite/tests/ghci/should_run/T15633b.run                        T15633b [bad stderr] (ghci)
   testsuite/tests/haddock/haddock_testsuite/haddockHtmlTest.run      haddockHtmlTest [bad exit code] (normal)
   testsuite/tests/haddock/haddock_testsuite/haddockLatexTest.run     haddockLatexTest [bad exit code] (normal)
   testsuite/tests/haddock/haddock_testsuite/haddockHoogleTest.run    haddockHoogleTest [bad exit code] (normal)
   testsuite/tests/haddock/haddock_testsuite/haddockHypsrcTest.run    haddockHypsrcTest [bad exit code] (normal)
   testsuite/tests/plugins/T15858.run                                 T15858 [bad stderr] (ghci)
   testsuite/tests/plugins/T16104.run                                 T16104 [bad exit code] (normal)
   testsuite/tests/plugins/T16260.run                                 T16260 [bad exit code] (normal)
   testsuite/tests/polykinds/T7022.run                                T7022 [bad exit code] (normal)
   testsuite/tests/rts/T7040_ghci.run                                 T7040_ghci [bad exit code] (ghci)
   testsuite/tests/th/T2386.run                                       T2386 [bad exit code] (normal)
   testsuite/tests/th/T7445.run                                       T7445 [bad exit code] (normal)
   testsuite/tests/th/T2014/T2014.run                                 T2014 [bad exit code] (normal)
   testsuite/tests/th/TH_linker/path_with_commas.run                  path_with_commas [bad exit code] (normal)

On this branch:

Unexpected results from:
TEST="T1288_ghci T15633a T15633b T15858 T16104 T16260 T2014 T2276_ghci T2386 T3807 T7022 T7040_ghci T7445 T9208 annth_compunits haddockHoogleTest haddockHtmlTest haddockHypsrcTest haddockLatexTest path_with_commas recomp007"

SUMMARY for test run started at Tue Mar  5 15:29:11 2019 CET
 0:25:28 spent to go through
    6863 total tests, which gave rise to
   26594 test cases, of which
   19307 were skipped

      42 had missing libraries
    7132 expected passes
      92 expected failures

       0 caused framework failures
       0 caused framework warnings
       2 unexpected passes
      19 unexpected failures
       0 unexpected stat failures

Unexpected passes:
   /run/user/1001/ghctest-ypo1pf6n/test   spaces/testsuite/tests/driver/recomp007/recomp007.run    recomp007 [unexpected] (normal)
   /run/user/1001/ghctest-ypo1pf6n/test   spaces/testsuite/tests/stranal/should_compile/T9208.run  T9208 [unexpected] (optasm)

Unexpected failures:
   /run/user/1001/ghctest-ypo1pf6n/test   spaces/testsuite/tests/annotations/should_compile/th/annth_compunits.run  annth_compunits [bad exit code] (normal)
   /run/user/1001/ghctest-ypo1pf6n/test   spaces/testsuite/tests/dynlibs/T3807.run                                  T3807 [bad exit code] (normal)
   /run/user/1001/ghctest-ypo1pf6n/test   spaces/testsuite/tests/ffi/should_run/T1288_ghci.run                      T1288_ghci [bad exit code] (ghci)
   /run/user/1001/ghctest-ypo1pf6n/test   spaces/testsuite/tests/ffi/should_run/T2276_ghci.run                      T2276_ghci [bad exit code] (ghci)
   /run/user/1001/ghctest-ypo1pf6n/test   spaces/testsuite/tests/ghci/should_run/T15633a.run                        T15633a [bad exit code] (ghci)
   /run/user/1001/ghctest-ypo1pf6n/test   spaces/testsuite/tests/ghci/should_run/T15633b.run                        T15633b [bad stderr] (ghci)
   /run/user/1001/ghctest-ypo1pf6n/test   spaces/testsuite/tests/haddock/haddock_testsuite/haddockHtmlTest.run      haddockHtmlTest [bad exit code] (normal)
   /run/user/1001/ghctest-ypo1pf6n/test   spaces/testsuite/tests/haddock/haddock_testsuite/haddockLatexTest.run     haddockLatexTest [bad exit code] (normal)
   /run/user/1001/ghctest-ypo1pf6n/test   spaces/testsuite/tests/haddock/haddock_testsuite/haddockHoogleTest.run    haddockHoogleTest [bad exit code] (normal)
   /run/user/1001/ghctest-ypo1pf6n/test   spaces/testsuite/tests/haddock/haddock_testsuite/haddockHypsrcTest.run    haddockHypsrcTest [bad exit code] (normal)
   /run/user/1001/ghctest-ypo1pf6n/test   spaces/testsuite/tests/plugins/T15858.run                                 T15858 [bad stderr] (ghci)
   /run/user/1001/ghctest-ypo1pf6n/test   spaces/testsuite/tests/plugins/T16104.run                                 T16104 [bad exit code] (normal)
   /run/user/1001/ghctest-ypo1pf6n/test   spaces/testsuite/tests/plugins/T16260.run                                 T16260 [bad exit code] (normal)
   /run/user/1001/ghctest-ypo1pf6n/test   spaces/testsuite/tests/polykinds/T7022.run                                T7022 [bad exit code] (normal)
   /run/user/1001/ghctest-ypo1pf6n/test   spaces/testsuite/tests/rts/T7040_ghci.run                                 T7040_ghci [bad exit code] (ghci)
   /run/user/1001/ghctest-ypo1pf6n/test   spaces/testsuite/tests/th/T2386.run                                       T2386 [bad exit code] (normal)
   /run/user/1001/ghctest-ypo1pf6n/test   spaces/testsuite/tests/th/T7445.run                                       T7445 [bad exit code] (normal)
   /run/user/1001/ghctest-ypo1pf6n/test   spaces/testsuite/tests/th/T2014/T2014.run                                 T2014 [bad exit code] (normal)
   /run/user/1001/ghctest-ypo1pf6n/test   spaces/testsuite/tests/th/TH_linker/path_with_commas.run                  path_with_commas [bad exit code] (normal)