Cheng Shao authored 9 months ago and Marge Bot committed 9 months ago

This patch adjusts test cases to handle the stdcall removal:

- Some stdcall usages are replaced with ccall since stdcall doesn't
  make sense anymore.
- We also preserve some stdcall usages, and check in the expected
  warning messages to ensure GHC always warn about stdcall usages
  (-Wunsupported-calling-conventions) as expected.
- Error code testsuite coverage is slightly improved,
  -Wunsupported-calling-conventions is now tested.
- Obsolete code paths related to i386 windows are also removed.

The list_broken make target will transitively depend on the
calibrate.out target, which used STAGE1_GHC instead of TEST_HC. It
really should be TEST_HC since that's what get passed in the gitlab CI
config.

Andreas Klebinger authored 4 years ago and Marge Bot committed 4 years ago

alignment clearly should be a power of two. This patch makes it
so. We do so by using the #alignment directive instead of using
the size of the type.

Andreas Klebinger authored 4 years ago and Marge Bot committed 4 years ago

We now seem to use -Werror there. Which caused some long standing
warnings to become errors.

I applied changes to remove the warnings allowing the testsuite to
run on windows as well.

Azel authored 6 years ago and Tamar Christina committed 6 years ago

Summary:
Uses Win32's System.Win32.Console.CtrlHandler.withConsoleCtrlHandler to add
to Windows's version of the timeout executable a CTRL-C/CTRL-BREAK
handler which does the close IO port/kill job cleanup, just in case.

Signed-off-by: ARJANEN Loïc Jean David <arjanen.loic@gmail.com>

Reviewers: Phyx, bgamari

Reviewed By: Phyx

Subscribers: dfeuer, thomie, carter

GHC Trac Issues: #12721

Differential Revision: https://phabricator.haskell.org/D4631

Ryan Scott authored 7 years ago and David Feuer committed 7 years ago

GHC 8.4.1 is out, so now GHC's support window only extends
back to GHC 8.2. This means we can delete gobs of code that were
only used for GHC 8.0 support. Hooray!

Test Plan: ./validate

Reviewers: bgamari, erikd, dfeuer

Reviewed By: bgamari, dfeuer

Subscribers: alexbiehl, dfeuer, rwbarton, thomie, carter

Differential Revision: https://phabricator.haskell.org/D4492

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

This reverts commit 138434fb.

Summary:
I don't really know why this stopped working for me, but it
did on a recent clean.  I don't know if this is right but
it solved the problem for me.

Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Test Plan: validate

Reviewers: bgamari, austin

Subscribers: rwbarton, thomie

Differential Revision: https://phabricator.haskell.org/D3318

Rufflewind authored 8 years ago and Ben Gamari committed 8 years ago

1. DoParamM requires the FlexibleContexts pragma now.

2. topHandler02 and topHandler03 were broken as timeout.py failed to
   translate signals to exit codes.

3. topHandler03 does not produce a consistent stderr, as it depends on
   what the user has /bin/sh set to.  dash writes "Terminated" whereas
   bash and zsh produce nothing in non-interactive mode.

4. The remaining tests are broken due to changes in the error message
   formatting.

Test Plan: validate

Reviewers: thomie, dfeuer, austin, hvr, bgamari

Reviewed By: bgamari

Subscribers: Phyx, dfeuer

Differential Revision: https://phabricator.haskell.org/D2807

Bump the version of `Win32` to `2.5.0.0` which is a major update and includes
fixes for wrong alignments and wrong 64-bit types. Strangely enough this also
seems to resolve #12713, where `T10858` was failing due to too-low allocations.

The underlying type aliases have changed, so there is a potential
for user programs not to compile anymore, but the types were incorrect.

This also requires a bump in the `directory`, `Cabal`, and `process`
submodules.

Original author: Tamar Christina <tamar@zhox.com>

Test Plan: ./validate

Reviewers: bgamari, RyanGlScott, austin

Subscribers: hvr, RyanGlScott, thomie, #ghc_windows_task_force

Differential Revision: https://phabricator.haskell.org/D2938

Summary:
Previously we would make direct calls to `diff` using `os.system`.
On Windows `os.system` is implemented using the standard
idiom `CreateProcess .. WaitForSingleObject ..`.

This again runs afoul with the `_exec` behaviour on Windows. So we ran
into some trouble where sometimes `diff` would return before it's done.

On tests which run multiple ways, such as `8086` what happens is that
we think the diff is done and continue. The next way tries to set things
up again by removing any previous directory. This would then fail with
and error saying the directory can't be removed. Which is true, because
the previous diff code/child is still running.

We shouldn't make any external calls to anything using `os.system`.
Instead just use `runCmd` which uses `timeout`. This also ensures that if
we hit the cygwin bug where diff or any other utility hangs, we kill it and
continue and not hang the entire test and leave hanging processes.

Further more we also:
Ignore error lines from `removeFile` from tools in the testsuite. This is a rather large
hammer to work around the fact that `hsc2hs` often tries to remove it's own file too early.
When this is patched the workaround can be removed. See Trac #9775

We mark `prog003` as skip. Since this test randomly fails and passes. For stability it's disabled
but it is a genuine bug which we should find. It's something with interface files being
overwritten. See Trac #11317

when `rmtree` hits a readonly file, the `onerror` handler is raised afterwards but not
during the tree walk. It doesn't allow you to recover and continue as we thought.
Instead you have to explicitly start again. This is why sometimes even though we
call `cleanup` before `os.mkdirs`, it would sometimes fail with an error that the
folder already exists. So we now do a second walk.

A new verbosity level (4) will strip the silent flags from `MAKE` invocations so you can actually
see what's going on.

Test Plan: ./validate on build bots.

Reviewers: bgamari, austin

Reviewed By: bgamari

Subscribers: mpickering, thomie, #ghc_windows_task_force

Differential Revision: https://phabricator.haskell.org/D2894

GHC Trac Issues: #12661, #11317, #9775

as suggested by @gracjan at
https://github.com/ghc/ghc/commit/efc4a1661f0fc1004a4b7b0914f3d3a08c2e791a#commitcomment-20284337

Summary:
we spawn the child processes with handle inheritance on. So they inherit the std handles.
The problem is that the job handle gets inherited too.
So the `JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE` doesn't get used since there are
open handles to the job in the children.

We then terminate the top level process which is `sh` but leaves the children around.

This explicitly disallows the inheritance of the job and events handle.

Test Plan: ./validate

Reviewers: austin, bgamari

Reviewed By: bgamari

Subscribers: thomie, #ghc_windows_task_force

Differential Revision: https://phabricator.haskell.org/D2895

GHC Trac Issues: #13004

Summary:
Timeout has been broken by my previous patch.
The timeout event was not being processed correctly,
as such hanging processes would not be killed as they should
have been.

This corrects it.

Test Plan:
./validate

~/ghc/testsuite/timeout/install-inplace/bin/timeout.exe 10 "sleep 10000s"

Reviewers: austin, RyanGlScott, bgamari

Reviewed By: bgamari

Subscribers: thomie, #ghc_windows_task_force

Differential Revision: https://phabricator.haskell.org/D2880

GHC Trac Issues: #13004

Summary:
Fix issues preventing x86 GHC to build on Windows and
fix segfault in the testsuite.

Test Plan: ./validate

Reviewers: austin, erikd, simonmar, bgamari

Reviewed By: bgamari

Subscribers: #ghc_windows_task_force, thomie

Differential Revision: https://phabricator.haskell.org/D2789

Tamar Christina authored 8 years ago and Ben Gamari committed 8 years ago

In a land far far away, a project called Cygwin was born.
Cygwin used newlib as it's standard C library implementation.

But Cygwin wanted to emulate POSIX systems as closely as possible.
So it implemented `execv` using the Windows function `spawnve`.

Specifically

```
spawnve (_P_OVERLAY, path, argv, cur_environ ())
```

`_P_OVERLAY` is crucial, as it makes the function behave *sort of*
like execv on linux. the child process replaces the original process.

With one major difference because of the difference in process models
on Windows: the original process signals the caller that it's done.

this is why the file is still locked. because it's still running,
control was returned because the parent process was destroyed,
but the child is still running.

I think it's just pure dumb luck, that the older runtimes are slow
enough to give the process time to terminate before we tried deleting
the file.  Which explains why you do have sporadic failures even on
older runtimes like 2.5.0, of a test or two (like T7307).

So this patch fixes a couple of things. I leverage the existing
`timeout.exe` to implement a workaround for this issue.

a) The old timeout used to start the process then assign it to the job.
   This is slightly faulty since child processes are only assigned to a
   job is their parent were assigned at the time they started. So this
   was a race condition. I now create the process suspended, assign it
   to the job and then resume it. Which means all child processes are
   not running under the same job.

b) First things, Is to prevent dangling child processes. I mark the job
   with `JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE` so when the last process in
   the job is done, it insures all processes under the job are killed.

c) Secondly, I change the way we wait for results. Instead of waiting
   for the parent process to terminate, I wait for the job itself to
   terminate.

   There's a slight subtlety there, we can't wait on the job itself.
   Instead we have to create an I/O Completion port and wait for signals
   on it.  See
   https://blogs.msdn.microsoft.com/oldnewthing/20130405-00/?p=4743

This fixes the issues on all runtimes for me and makes T7307 pass
consistenly.

The threading was also simplified by hiding all the locking in a single
semaphore and a completion class. Futhermore some additional error
reporting was added.

For encoding the testsuite now no longer passes a file handle to the
subprocess since on windows, sh.exe seems to acquire a lock on the file
that is not released in a timely fashion.

I suspect this because cygwin seems to emulate console handles by
creating file handles and using those for std handles. So when we give
it an existing file handle it just locks the file. I what's happening is
that it's not releasing the handle until all shared cygwin processes are
dead. Which explains why it worked in single threaded mode.

So now instead we pass a pipe and do not interpret the resulting data.

Any bytes written to stdin or read out of stdout/stderr are done so in
binary mode and we do not interpret the data. The reason for this is
that we have encoding tests in GHC which pass invalid utf-8. If we try
to handle the data as text then python will throw an exception instead
of a test comparison failing.

Also I have fixed the ability to override `PYTHON` when calling `make
tests`. This now works the same as with `.\validate`.

Finally, after cleaning up the locks I was able to make the abort
behavior work correctly as I believe it was intended: when you press
Ctrl+C and send an interrupt signal, the testsuite finishes the active
tests and then gracefully exits showing you a report of the progress it
did make. So using Ctrl+C will not just *die* as it did before.

These changes lift the restriction on which python version you use
(msys/mingw) or which runtime or python 3 or python 2.  All combinations
should now be supported.

Test Plan:
PATH=/usr/local/bin:/mingw64/bin:$APPDATA/cabal/bin:$PATH &&
PYTHON=/usr/bin/python THREADS=9 make test
THREADS=9 make test
PATH=/usr/local/bin:/mingw64/bin:$APPDATA/cabal/bin:$PATH &&
PYTHON=/usr/bin/python ./validate --quiet --testsuite-only

Reviewers: erikd, RyanGlScott, bgamari, austin

Subscribers: jrtc27, mpickering, thomie, #ghc_windows_task_force

Differential Revision: https://phabricator.haskell.org/D2684

GHC Trac Issues: #12725, #12554, #12661, #12004

Ben Gamari authored 8 years ago and Ben Gamari committed 8 years ago

Previously if the test is interrupted (e.g. with Ctrl-C) any processes
which it spawned may not be properly terminated. Here we catch any
exception and ensure that we job is terminated.

Test Plan: Validate on Windows

Reviewers: Phyx, austin

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D2599

This is a followup to e1293bbf, but then
for Windows timeout.

Differential Revision: https://phabricator.haskell.org/D1207

[skip ci]

Summary:
This is a fixup of https://phabricator.haskell.org/D233

The only difference is in findTFiles (first commit), which
previously broke Windows runner; now I translated literally
instead attempting to improve it, and checked it works.

Test Plan:
I used validate under 2,3 on Linux and under 2 on msys2.
On Windows I've seen a large number of failures, but they don't
seem to be connected with the patch.

Reviewers: hvr, simonmar, thomie, austin

Reviewed By: austin

Subscribers: thomie, carter, ezyang, simonmar

Differential Revision: https://phabricator.haskell.org/D310

GHC Trac Issues: #9184

This reverts commit 084d241b.

This is a possible culprit of Windows breakage reported at ghc-devs.

Summary:
Most of the changes is adaptation of old Python 2 only code.
My priority was not breaking Python 2, and so I avoided bigger
changes to the driver. In particular, under Python 3 the output
is a str and buffering cannot be disabled.

To test, define PYTHON=python3 in testsuite/mk/boilerplate.mk.

Thanks to aspidites <emarshall85@gmail.com> who provided the initial patch.

Test Plan: validate under 2 and 3

Reviewers: hvr, simonmar, thomie, austin

Reviewed By: thomie, austin

Subscribers: aspidites, thomie, simonmar, ezyang, carter

Differential Revision: https://phabricator.haskell.org/D233

GHC Trac Issues: #9184

Use a python timeout program, so that we don't have to worry about
whether or not the compiler we're testing has built the timeout
program correctly

The python timeout program doesn't work on mingw, so we still use the
Haskell program on Windows

Contributed by: Krister Walfridsson <krister.walfridsson@gmail.com>