GitLab

Summary:
This is a fixup of https://phabricator.haskell.org/D233

The only difference is in findTFiles (first commit), which
previously broke Windows runner; now I translated literally
instead attempting to improve it, and checked it works.

Test Plan:
I used validate under 2,3 on Linux and under 2 on msys2.
On Windows I've seen a large number of failures, but they don't
seem to be connected with the patch.

Reviewers: hvr, simonmar, thomie, austin

Reviewed By: austin

Subscribers: thomie, carter, ezyang, simonmar

Differential Revision: https://phabricator.haskell.org/D310

GHC Trac Issues: #9184

This reverts commit 084d241b.

This is a possible culprit of Windows breakage reported at ghc-devs.

Summary:
Most of the changes is adaptation of old Python 2 only code.
My priority was not breaking Python 2, and so I avoided bigger
changes to the driver. In particular, under Python 3 the output
is a str and buffering cannot be disabled.

To test, define PYTHON=python3 in testsuite/mk/boilerplate.mk.

Thanks to aspidites <emarshall85@gmail.com> who provided the initial patch.

Test Plan: validate under 2 and 3

Reviewers: hvr, simonmar, thomie, austin

Reviewed By: thomie, austin

Subscribers: aspidites, thomie, simonmar, ezyang, carter

Differential Revision: https://phabricator.haskell.org/D233

GHC Trac Issues: #9184

Signed-off-by: Sergei Trofimovich <slyfox@gentoo.org>

Summary:
New function signal_exit_code hides differences between target platforms
handling of fatal error signals and the applications' exit codes. E.g.
on Linux the application exit code which receives fatal error signal
is encoded as 128 + signal value. On the other hand on Solaris
the application exit code is signal value alone.

Test Plan: validated on Linux and tested on Solaris

Reviewers: austin, simonmar

Reviewed By: simonmar

Subscribers: phaskell, simonmar, relrod, ezyang, carter

Differential Revision: https://phabricator.haskell.org/D122

With THREADS=n, for n > 1, it becomes impossible to match the
performance numbers to the test case name. Hence include it in the
output. This also makes grepping through a bunch of logs for a specific
test case much easier, and outweighs the extra verbosity.

this makes it easier to spot a “just over the mark” change (e.g. +5.1%),
compared to a more radical jump (e.g. +15%).

which makes it print performance numbers even when the test succeeds
(good for historic analysis)

Signed-off-by: Austin Seipp <austin@well-typed.com>

check_stdout(f) allows you to override the test framework's
diff based output checking with another mechanism.  f is
a function which takes two arguments: the first is the
filename containing the observed stdout, the second is the
normaliser that would have been applied (in case you want
to read, normalise, and then do something.)
Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

by ensuring this in the test runner.

More often than not the output of the performance tests is in the way,
rather than helping. This allows the use of `make SKIP_PERF_TESTS=YES`
to skip these tests. Fixes #8413

Select verbosity with "make VERBOSE=n". Options so far:
 n=0: No per-test output
 n=1: Only failing test results
 n=2: As above, plus progress information (names of all tests)
 n=3: As aobve, plus commands called.
Default currently is n=3, although n=2 might be a nicer default.

This makes the test run alone, to increase the chance of a
multi-CPU race happening.

The driver now also supports nested lists of setup functions

Predicates now match the order they are documented on the wiki

The name 'os' clashes with a python library

Gives a list of tickets that the testsuite thinks are broken, and
what bug it thinks is the reason. This can then be pasted into trac
and 'previewed', which will show any closed tickets with strikeout.

In particular, the 3 values you most need to care about (framework
failures, unexpected passes, unexpected failures) are now together
in a block.