1. 03 May, 2018 1 commit
    • niteria's avatar
      Correctly add unwinding info in manifestSp and makeFixupBlocks · 6132d7c5
      niteria authored
      In `manifestSp` the unwind info was before the relevant instruction, not
      after.  I added some notes to establish semantics.  Also removes
      redundant annotation in stg_catch_frame.
      
      For `makeFixupBlocks` it looks like we were off by `wORD_SIZE dflags`.
      I'm not sure why, but it lines up with `manifestSp`.  In fact it lines
      up so well so that I can consolidate the Sp unwind logic in
      `maybeAddUnwind`.  I detected the problems with `makeFixupBlocks` by
      running T14779b after patching D4559.
      
      Test Plan: added a new test
      
      Reviewers: bgamari, scpmw, simonmar, erikd
      
      Reviewed By: bgamari
      
      Subscribers: thomie, carter
      
      GHC Trac Issues: #14999
      
      Differential Revision: https://phabricator.haskell.org/D4606
      6132d7c5
  2. 31 Mar, 2018 1 commit
  3. 02 Jan, 2018 1 commit
    • Tamar Christina's avatar
      Windows: fix all failing tests. · 27b7b4db
      Tamar Christina authored
      This makes the testsuite pass clean on Windows again.
      It also fixes the `libstdc++-6.dll` error harbormaster
      was showing.
      
      I'm marking some tests as isolated tests to reduce their
      flakiness (mostly concurrency tests) when the test system
      is under heavy load.
      
      Updates process submodule.
      
      Test Plan: ./validate
      
      Reviewers: hvr, bgamari, erikd, simonmar
      
      Reviewed By: bgamari
      
      Subscribers: rwbarton, thomie, carter
      
      Differential Revision: https://phabricator.haskell.org/D4277
      27b7b4db
  4. 12 Sep, 2017 1 commit
  5. 28 Jul, 2017 1 commit
  6. 20 Jul, 2017 1 commit
    • Ben Gamari's avatar
      testsuite: Pipe stdin directly to process · c9c762dc
      Ben Gamari authored
      Previously the driver would read the stdin content from the source file
      and then write it to the subprocess' stdin. We now simply open the stdin
      file and provide that handle to the subprocess as its stdin
      
      Test Plan: Validate
      
      Reviewers: austin
      
      Subscribers: rwbarton, thomie, goldfire
      
      Differential Revision: https://phabricator.haskell.org/D3735
      c9c762dc
  7. 11 Jul, 2017 1 commit
  8. 12 May, 2017 1 commit
  9. 24 Apr, 2017 1 commit
  10. 03 Apr, 2017 2 commits
  11. 03 Mar, 2017 1 commit
    • rwbarton's avatar
      testsuite: Move echoing commands in make invocations to VERBOSE=5 · 6421c6f2
      rwbarton authored
      D2894 added a new verbosity level VERBOSE=4 to strip -s/--silent
      flags from make invocations in test commands. This will probably
      cause the test to fail of course, but is useful for seeing what
      a test that's already failing is doing.
      
      However there was already an undocumented meaning of VERBOSE=4,
      added in commit cfeededf, that causes the results of performance
      tests to be printed unconditionally (even when they are within the
      expected range). nomeata's ghc builder uses these figures to
      collect historical data on performance test figures. The new
      meaning of VERBOSE=4 added in D2894 means that any test that uses
      make now fails on the builder.
      
      This commit moves the new behavior of D2894 to the level VERBOSE=5
      so that nomeata's ghc builder again produces useful results on
      failing tests. It also adds documentation for both settings.
      
      Test Plan: did some manual testing
      
      Reviewers: austin, bgamari, Phyx, nomeata
      
      Reviewed By: bgamari, Phyx
      
      Subscribers: nomeata, thomie, Phyx
      
      Differential Revision: https://phabricator.haskell.org/D3141
      6421c6f2
  12. 26 Feb, 2017 1 commit
    • rwbarton's avatar
      tests: remove extra_files.py (#12223) · 3415bcaa
      rwbarton authored
      The script I used is included as testsuite/driver/kill_extra_files.py,
      though at this point it is for mostly historical interest.
      
      Some of the tests in libraries/hpc relied on extra_files.py, so this
      commit includes an update to that submodule.
      
      One test in libraries/process also relies on extra_files.py, but we
      cannot update that submodule so easily, so for now we special-case it
      in the test driver.
      3415bcaa
  13. 28 Jan, 2017 1 commit
    • Tamar Christina's avatar
      Add delete retry loop. [ci skip] · 1f366b8d
      Tamar Christina authored
      Summary:
      On Windows we have to retry the delete a couple of times.
      The reason for this is that a `FileDelete` command just marks a
      file for deletion. The file is really only removed when the last
      handle to the file is closed. Unfortunately there are a lot of
      system services that can have a file temporarily opened using a shared
      readonly lock, such as the built in AV and search indexer.
      
      We can't really guarantee that these are all off, so what we can do is
      whenever after a `rmtree` the folder still exists to try again and wait a bit.
      
      Based on what I've seen from the tests on CI server, is that this is relatively rare.
      So overall we won't be retrying a lot. If after a reasonable amount of time the folder is
      still locked then abort the current test by throwing an exception, this so it won't fail
      with an even more cryptic error.
      
      The issue is that these services often open a file using `FILE_SHARE_DELETE` permissions.
      So they can seemingly be removed, and for most intended purposes they are, but recreating
      the file with the same name will fail as the FS will prevent data loss.
      
      The MSDN docs for `DeleteFile` says:
      
      ```
      The DeleteFile function marks a file for deletion on close.
      Therefore, the file deletion does not occur until the last handle
      to the file is closed. Subsequent calls to CreateFile to open the
      file fail with ERROR_ACCESS_DENIED.
      ```
      
      Retrying seems to be a common pattern, SQLite has it in their driver
      http://www.sqlite.org/src/info/89f1848d7f
      
      The only way to avoid this is to run each way of a test in it's own folder.
      This would also have the added bonus of increased parallelism.
      
      Reviewers: austin, bgamari
      
      Reviewed By: bgamari
      
      Subscribers: thomie, #ghc_windows_task_force
      
      Differential Revision: https://phabricator.haskell.org/D2936
      
      GHC Trac Issues: #12661, #13162
      1f366b8d
  14. 10 Jan, 2017 1 commit
    • rwbarton's avatar
      testsuite driver: don't append to existing output files · 35a5b603
      rwbarton authored
      If you happen to have a T1234.run.stdout file lying aroud (probably
      from before the move to running tests in temporary subdirectories)
      it gets symlinked into the T1234.run directory since its name starts
      with T1234; and then program output gets appended to the existing
      file (through the symlink). We should open the file for writing
      instead, to replace the symlink with a new file.
      
      Test Plan: tested locally, + harbormaster
      
      Reviewers: austin, Phyx, bgamari
      
      Reviewed By: Phyx, bgamari
      
      Subscribers: thomie
      
      Differential Revision: https://phabricator.haskell.org/D2946
      35a5b603
  15. 28 Dec, 2016 1 commit
    • Tamar Christina's avatar
      Fix various issues with testsuite code on Windows · a3704409
      Tamar Christina authored
      Summary:
      Previously we would make direct calls to `diff` using `os.system`.
      On Windows `os.system` is implemented using the standard
      idiom `CreateProcess .. WaitForSingleObject ..`.
      
      This again runs afoul with the `_exec` behaviour on Windows. So we ran
      into some trouble where sometimes `diff` would return before it's done.
      
      On tests which run multiple ways, such as `8086` what happens is that
      we think the diff is done and continue. The next way tries to set things
      up again by removing any previous directory. This would then fail with
      and error saying the directory can't be removed. Which is true, because
      the previous diff code/child is still running.
      
      We shouldn't make any external calls to anything using `os.system`.
      Instead just use `runCmd` which uses `timeout`. This also ensures that if
      we hit the cygwin bug where diff or any other utility hangs, we kill it and
      continue and not hang the entire test and leave hanging processes.
      
      Further more we also:
      Ignore error lines from `removeFile` from tools in the testsuite. This is a rather large
      hammer to work around the fact that `hsc2hs` often tries to remove it's own file too early.
      When this is patched the workaround can be removed. See Trac #9775
      
      We mark `prog003` as skip. Since this test randomly fails and passes. For stability it's disabled
      but it is a genuine bug which we should find. It's something with interface files being
      overwritten. See Trac #11317
      
      when `rmtree` hits a readonly file, the `onerror` handler is raised afterwards but not
      during the tree walk. It doesn't allow you to recover and continue as we thought.
      Instead you have to explicitly start again. This is why sometimes even though we
      call `cleanup` before `os.mkdirs`, it would sometimes fail with an error that the
      folder already exists. So we now do a second walk.
      
      A new verbosity level (4) will strip the silent flags from `MAKE` invocations so you can actually
      see what's going on.
      
      Test Plan: ./validate on build bots.
      
      Reviewers: bgamari, austin
      
      Reviewed By: bgamari
      
      Subscribers: mpickering, thomie, #ghc_windows_task_force
      
      Differential Revision: https://phabricator.haskell.org/D2894
      
      GHC Trac Issues: #12661, #11317, #9775
      a3704409
  16. 21 Dec, 2016 1 commit
  17. 09 Dec, 2016 1 commit
    • Rufflewind's avatar
      testsuite: make tests respond to SIGINT properly · ca593c7d
      Rufflewind authored
      The `std*_buffer` need to be bytes to avoid breaking Python 3.
      
      Also, using a blanket `except` in Python without specifying the
      exception types will catch special exceptions such as
      `KeyboardInterrupt`, which can prevent the program from being
      interrupted properly.
      
      Test Plan: validate
      
      Reviewers: thomie, austin, bgamari
      
      Reviewed By: bgamari
      
      Differential Revision: https://phabricator.haskell.org/D2805
      ca593c7d
  18. 01 Dec, 2016 1 commit
  19. 30 Nov, 2016 1 commit
    • Tamar Christina's avatar
      Fix testsuite threading, timeout, encoding and performance issues on Windows · 0ce59be3
      Tamar Christina authored
      In a land far far away, a project called Cygwin was born.
      Cygwin used newlib as it's standard C library implementation.
      
      But Cygwin wanted to emulate POSIX systems as closely as possible.
      So it implemented `execv` using the Windows function `spawnve`.
      
      Specifically
      
      ```
      spawnve (_P_OVERLAY, path, argv, cur_environ ())
      ```
      
      `_P_OVERLAY` is crucial, as it makes the function behave *sort of*
      like execv on linux. the child process replaces the original process.
      
      With one major difference because of the difference in process models
      on Windows: the original process signals the caller that it's done.
      
      this is why the file is still locked. because it's still running,
      control was returned because the parent process was destroyed,
      but the child is still running.
      
      I think it's just pure dumb luck, that the older runtimes are slow
      enough to give the process time to terminate before we tried deleting
      the file.  Which explains why you do have sporadic failures even on
      older runtimes like 2.5.0, of a test or two (like T7307).
      
      So this patch fixes a couple of things. I leverage the existing
      `timeout.exe` to implement a workaround for this issue.
      
      a) The old timeout used to start the process then assign it to the job.
         This is slightly faulty since child processes are only assigned to a
         job is their parent were assigned at the time they started. So this
         was a race condition. I now create the process suspended, assign it
         to the job and then resume it. Which means all child processes are
         not running under the same job.
      
      b) First things, Is to prevent dangling child processes. I mark the job
         with `JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE` so when the last process in
         the job is done, it insures all processes under the job are killed.
      
      c) Secondly, I change the way we wait for results. Instead of waiting
         for the parent process to terminate, I wait for the job itself to
         terminate.
      
         There's a slight subtlety there, we can't wait on the job itself.
         Instead we have to create an I/O Completion port and wait for signals
         on it.  See
         https://blogs.msdn.microsoft.com/oldnewthing/20130405-00/?p=4743
      
      This fixes the issues on all runtimes for me and makes T7307 pass
      consistenly.
      
      The threading was also simplified by hiding all the locking in a single
      semaphore and a completion class. Futhermore some additional error
      reporting was added.
      
      For encoding the testsuite now no longer passes a file handle to the
      subprocess since on windows, sh.exe seems to acquire a lock on the file
      that is not released in a timely fashion.
      
      I suspect this because cygwin seems to emulate console handles by
      creating file handles and using those for std handles. So when we give
      it an existing file handle it just locks the file. I what's happening is
      that it's not releasing the handle until all shared cygwin processes are
      dead. Which explains why it worked in single threaded mode.
      
      So now instead we pass a pipe and do not interpret the resulting data.
      
      Any bytes written to stdin or read out of stdout/stderr are done so in
      binary mode and we do not interpret the data. The reason for this is
      that we have encoding tests in GHC which pass invalid utf-8. If we try
      to handle the data as text then python will throw an exception instead
      of a test comparison failing.
      
      Also I have fixed the ability to override `PYTHON` when calling `make
      tests`. This now works the same as with `.\validate`.
      
      Finally, after cleaning up the locks I was able to make the abort
      behavior work correctly as I believe it was intended: when you press
      Ctrl+C and send an interrupt signal, the testsuite finishes the active
      tests and then gracefully exits showing you a report of the progress it
      did make. So using Ctrl+C will not just *die* as it did before.
      
      These changes lift the restriction on which python version you use
      (msys/mingw) or which runtime or python 3 or python 2.  All combinations
      should now be supported.
      
      Test Plan:
      PATH=/usr/local/bin:/mingw64/bin:$APPDATA/cabal/bin:$PATH &&
      PYTHON=/usr/bin/python THREADS=9 make test
      THREADS=9 make test
      PATH=/usr/local/bin:/mingw64/bin:$APPDATA/cabal/bin:$PATH &&
      PYTHON=/usr/bin/python ./validate --quiet --testsuite-only
      
      Reviewers: erikd, RyanGlScott, bgamari, austin
      
      Subscribers: jrtc27, mpickering, thomie, #ghc_windows_task_force
      
      Differential Revision: https://phabricator.haskell.org/D2684
      
      GHC Trac Issues: #12725, #12554, #12661, #12004
      0ce59be3
  20. 25 Nov, 2016 1 commit
  21. 17 Nov, 2016 1 commit
    • Ben Gamari's avatar
      testsuite: Rip out hack for #12554 · 4d4f3533
      Ben Gamari authored
      @Phyx is working on correctly fixing (pun intended) the underlying issue
      that prompted this hack. It turns out that `timeout` it the culprit.
      Moreover, this hack breaks on msys python builds, which don't export
      `WindowsError`.
      
      Test Plan: Validate on Windows with `msys` python.
      
      Reviewers: Phyx, austin
      
      Subscribers: thomie, Phyx
      
      Differential Revision: https://phabricator.haskell.org/D2724
      
      GHC Trac Issues: #12554
      4d4f3533
  22. 17 Oct, 2016 2 commits
    • Ben Gamari's avatar
      testsuite/driver: More Unicode awareness · 7d2df320
      Ben Gamari authored
      Explicitly specify utf8 encoding in a few spots which were failing on
      Windows with Python 3.
      
      Test Plan: Validate
      
      Reviewers: austin, thomie
      
      Differential Revision: https://phabricator.haskell.org/D2602
      
      GHC Trac Issues: #9184
      7d2df320
    • Ben Gamari's avatar
      testsuite: Work around #12554 · 9cb44598
      Ben Gamari authored
      It seems that Python 2.7.11 and "recent" msys2 releases are broken,
      holding open file locks unexpected. This causes rmtree to intermittently
      fail. Even worse, it would fail silently (since we pass
      ignore_errors=True), causing makedirs to fail later.
      
      We now explicitly check for the existence of the test directory before
      attempting to delete it and disable ignore_errors. Moreover, on Windows
      we now try multiple times to rmtree the testdir, working around the
      apparently msys bug.
      
      This is all just terrible, but Phyx and I spent several hours trying to
      track down the issue to no available. The workaround is better than
      nothing.
      9cb44598
  23. 08 Oct, 2016 1 commit
  24. 30 Jun, 2016 1 commit
    • thomie's avatar
      Testsuite: do not depend on sys.stdout.encoding · e8d62711
      thomie authored
      The cause of #12213 is in dump_stdout and dump_stderr:
      
            print(read_no_crs(<filename>))
      
      Commit 6f6f5154 changed read_no_crs to
      return a unicode string. Printing a unicode strings works fine as long
      as sys.stdout.encoding is 'UTF-8'.
      
      There are two reasons why sys.stdout.encoding might not be 'UTF-8'.
      
      * When output is going to a file, sys.stdout and sys.stdout do not respect
        the locale:
      
        $ LC_ALL=en_US.utf8 python -c 'import sys; print(sys.stderr.encoding)'
        UTF-8
        $ LC_ALL=en_US.utf8 python -c 'import sys; print(sys.stderr.encoding)' 2>/dev/null
        None
      
      * When output is going to the terminal, explicitly reopening sys.stdout has
        the side-effect of changing sys.stdout.encoding from 'UTF-8' to 'None'.
      
            sys.stdout = os.fdopen(sys.__stdout__.fileno(), "w", 0)
      
        We currently do this to set a buffersize of 0 (the actual
        buffersize used is irrelevant for the sys.stdout.encoding problem).
      
      Solution: fix dump_stdout and dump_stderr to not use read_no_crs.
      e8d62711
  25. 29 Jun, 2016 2 commits
  26. 28 Jun, 2016 4 commits
    • thomie's avatar
      Testsuite: framework failure improvements (#11165) · 782cacf5
      thomie authored
      * add framework failures to unexpected results list
      * report errors in .T files as framework failures (show in summary)
      * don't report missing tests when framework failures in .T files
      782cacf5
    • thomie's avatar
      Testsuite: cleanup printing of summary · d8e9b876
      thomie authored
      Just use a simple list of tuples, instead of a nested map.
      
      -90 lines of code.
      d8e9b876
    • thomie's avatar
      Testsuite: open/close stdin/stdout/stderr explicitly · 58f0086b
      thomie authored
      This allows run_command's to contain `|`, and `no_stdin` isn't necessary
      anymore.
      
      Unfortunately it doesn't fix T7037 on Windows which I had hoped it would
      (testsuite driver tries to read a file that it just created itself, but
      the OS says it doesn't exist).
      
      The only drawback of this commit is that the command that the testsuite
      prints to the terminal (for debugging purposes) doesn't mention the
      files that stdout and stderr are redirected to anymore. This is probably
      ok.
      
      Update submodule unix.
      
      Differential Revision: https://phabricator.haskell.org/D1234
      58f0086b
    • thomie's avatar
      Testsuite: simplify extra_file handling · 206b4a1d
      thomie authored
      Before, `extra_files(['.hpc/Main.mix'])` meant copy `Main.mix` to
      `<testdir>/.hpc/Main.mix`. This feature wasn't really necessary, so now
      it just means copy `Main.mix` to `<testdir>/Main.mix`. This simplifies
      the implementation.
      
      Some small other cleanups as well. -40 lines of code.
      206b4a1d
  27. 27 Jun, 2016 3 commits
  28. 24 Jun, 2016 1 commit
  29. 20 Jun, 2016 4 commits