1. 09 Dec, 2016 4 commits
    • Rufflewind's avatar
      testsuite: make tests respond to SIGINT properly · ca593c7d
      Rufflewind authored
      The `std*_buffer` need to be bytes to avoid breaking Python 3.
      
      Also, using a blanket `except` in Python without specifying the
      exception types will catch special exceptions such as
      `KeyboardInterrupt`, which can prevent the program from being
      interrupted properly.
      
      Test Plan: validate
      
      Reviewers: thomie, austin, bgamari
      
      Reviewed By: bgamari
      
      Differential Revision: https://phabricator.haskell.org/D2805
      ca593c7d
    • Ben Gamari's avatar
      Mark T12903 as broken on OS X · 62418b87
      Ben Gamari authored
      Something has recently broken it. See #12956.
      62418b87
    • Sylvain Henry's avatar
      Scrutinee Constant Folding · d3b546b1
      Sylvain Henry authored
      This patch introduces new rules to perform constant folding through
      case-expressions.
      
      E.g.,
      ```
      case t -# 10# of _ {  ===> case t of _ {
               5#      -> e1              15#     -> e1
               8#      -> e2              18#     -> e2
               DEFAULT -> e               DEFAULT -> e
      ```
      
      The initial motivation is that it allows "Merge Nested Cases"
      optimization to kick in and to further simplify the code
      (see Trac #12877).
      
      Currently we recognize the following operations for Word# and Int#: Add,
      Sub, Xor, Not and Negate (for Int# only).
      
      Test Plan: validate
      
      Reviewers: simonpj, austin, bgamari
      
      Reviewed By: simonpj, bgamari
      
      Subscribers: thomie
      
      Differential Revision: https://phabricator.haskell.org/D2762
      
      GHC Trac Issues: #12877
      d3b546b1
    • Ryan Scott's avatar
      Disambiguate reified closed type family kinds in TH · f65ff2c4
      Ryan Scott authored
      Summary:
      A continuation of #8953. This fixes an oversight in which the
      left-hand sides of closed type families, when reified in Template Haskell,
      would not be given kind annotations, even when they are necessary for
      disambiguation purposes in the presence of `PolyKinds`.
      
      Fixes #8953 and #12646.
      
      Test Plan: ./validate
      
      Reviewers: hvr, bgamari, austin, goldfire
      
      Reviewed By: goldfire
      
      Subscribers: thomie
      
      Differential Revision: https://phabricator.haskell.org/D2795
      
      GHC Trac Issues: #8953, #12646
      f65ff2c4
  2. 08 Dec, 2016 4 commits
  3. 07 Dec, 2016 6 commits
    • Alan Zimmerman's avatar
      Remove stray commented out line in all.T · 9bcc4e33
      Alan Zimmerman authored
      9bcc4e33
    • Alan Zimmerman's avatar
      Fix pretty printer test to nog generate stdout · 58d78dc0
      Alan Zimmerman authored
      It was doing a dump to stdout, which is not repeatable across platforms.
      58d78dc0
    • Alan Zimmerman's avatar
      Add HsSyn prettyprinter tests · 499e4382
      Alan Zimmerman authored
      Summary:
      Add prettyprinter tests, which take a file, parse it, pretty print it,
      re-parse the pretty printed version and then compare the original and
      new ASTs (ignoring locations)
      
      Updates haddock submodule to match the AST changes.
      
      There are three issues outstanding
      
      1. Extra parens around a context are not reproduced. This will require an
         AST change and will be done in a separate patch.
      
      2. Currently if an `HsTickPragma` is found, this is not pretty-printed,
         to prevent noise in the output.
      
         I am not sure what the desired behaviour in this case is, so have left
         it as before. Test Ppr047 is marked as expected fail for this.
      
      3. Apart from in a context, the ParsedSource AST keeps all the parens from
         the original source.  Something is happening in the renamer to remove the
         parens around visible type application, causing T12530 to fail, as the
         dumped splice decl is after the renamer.
      
         This needs to be fixed by keeping the parens, but I do not know where ...
      499e4382
    • Simon Marlow's avatar
      Overhaul of Compact Regions (#12455) · 7036fde9
      Simon Marlow authored
      Summary:
      This commit makes various improvements and addresses some issues with
      Compact Regions (aka Compact Normal Forms).
      
      This was the most important thing I wanted to fix.  Compaction
      previously prevented GC from running until it was complete, which
      would be a problem in a multicore setting.  Now, we compact using a
      hand-written Cmm routine that can be interrupted at any point.  When a
      GC is triggered during a sharing-enabled compaction, the GC has to
      traverse and update the hash table, so this hash table is now stored
      in the StgCompactNFData object.
      
      Previously, compaction consisted of a deepseq using the NFData class,
      followed by a traversal in C code to copy the data.  This is now done
      in a single pass with hand-written Cmm (see rts/Compact.cmm). We no
      longer use the NFData instances, instead the Cmm routine evaluates
      components directly as it compacts.
      
      The new compaction is about 50% faster than the old one with no
      sharing, and a little faster on average with sharing (the cost of the
      hash table dominates when we're doing sharing).
      
      Static objects that don't (transitively) refer to any CAFs don't need
      to be copied into the compact region.  In particular this means we
      often avoid copying Char values and small Int values, because these
      are static closures in the runtime.
      
      Each Compact# object can support a single compactAdd# operation at any
      given time, so the Data.Compact library now enforces mutual exclusion
      using an MVar stored in the Compact object.
      
      We now get exceptions rather than killing everything with a barf()
      when we encounter an object that cannot be compacted (a function, or a
      mutable object).  We now also detect pinned objects, which can't be
      compacted either.
      
      The Data.Compact API has been refactored and cleaned up.  A new
      compactSize operation returns the size (in bytes) of the compact
      object.
      
      Most of the documentation is in the Haddock docs for the compact
      library, which I've expanded and improved here.
      
      Various comments in the code have been improved, especially the main
      Note [Compact Normal Forms] in rts/sm/CNF.c.
      
      I've added a few tests, and expanded a few of the tests that were
      there.  We now also run the tests with GHCi, and in a new test way
      that enables sanity checking (+RTS -DS).
      
      There's a benchmark in libraries/compact/tests/compact_bench.hs for
      measuring compaction speed and comparing sharing vs. no sharing.
      
      The field totalDataW in StgCompactNFData was unnecessary.
      
      Test Plan:
      * new unit tests
      * validate
      * tested manually that we can compact Data.Aeson data
      
      Reviewers: gcampax, bgamari, ezyang, austin, niteria, hvr, erikd
      
      Subscribers: thomie, simonpj
      
      Differential Revision: https://phabricator.haskell.org/D2751
      
      GHC Trac Issues: #12455
      7036fde9
    • Ben Gamari's avatar
      Really mark T9577 as broken · 4dd6b37f
      Ben Gamari authored
      I failed at the last attempt.
      4dd6b37f
    • Ben Gamari's avatar
      Mark T9577 as broken on Darwin due to #12937 · 39143a47
      Ben Gamari authored
      39143a47
  4. 06 Dec, 2016 5 commits
  5. 05 Dec, 2016 5 commits
    • Tamar Christina's avatar
      Color output is wreaking havoc on test results · 847d2293
      Tamar Christina authored
      Summary:
      D2716 introduced colors into the output of GHC.
      These color ourputs are done using escape characters output
      to the terminal.
      
      These however are wreaking havoc on the testsuite output as now
      no stderr with a warning or error will match anymore.
      
      Instead of accepting the new codes as expected values instead I
      turn them off. So the testsuite is consistent on platforms/terminals we
      don't support colors on.
      
      Test Plan:
      any test that outputs colors. e.g.
      
      make test TEST=T9576
      
      Reviewers: austin, Rufflewind, bgamari
      
      Subscribers: thomie, #ghc_windows_task_force
      
      Differential Revision: https://phabricator.haskell.org/D2787
      
      GHC Trac Issues: #8809
      847d2293
    • Simon Peyton Jones's avatar
      Test Trac #12925 · 3e3f7c21
      Simon Peyton Jones authored
      3e3f7c21
    • Simon Peyton Jones's avatar
      Use isFamFreeTyCon now we have it · e9123102
      Simon Peyton Jones authored
      Refactoring only
      e9123102
    • Simon Peyton Jones's avatar
      Fix used-variable calculation (Trac #12548) · 6305674f
      Simon Peyton Jones authored
      The used-variable calculation for pattern synonyms is a little
      tricky, for reasons described in RnBinds
      Note [Pattern synonym builders don't yield dependencies]
      
      It was right semantically, but the "unused-variable warning" was
      wrong, which led to Trac #12548.
      6305674f
    • Simon Peyton Jones's avatar
      Fix an asymptotic bug in the occurrence analyser · 517d03e4
      Simon Peyton Jones authored
      Trac #12425 and #12234 showed up a major and long-standing
      bug in the occurrence analyser, whereby it could generate
      explonentially large program!
      
      There's a lot of commentary on #12425; and it's all described
      in Note [Loop breakers, node scoring, and stability]
      
      I did quite a lot of refactoring to make the code comprehensibe
      again (its structure had bit-rotted rather), so the patch
      looks bigger than it really is.
      
      Hurrah!
      
      I did a nofib run to check that I hadn't inadertently ruined
      anything:
      
      --------------------------------------------------------------------------------
              Program           Size    Allocs   Runtime   Elapsed  TotalMem
      --------------------------------------------------------------------------------
                fluid          -0.3%     -1.5%      0.01      0.01     +0.0%
               parser          -0.9%     +0.6%      0.04      0.04     +0.0%
               prolog          -0.1%     +1.2%      0.00      0.00     +0.0%
      
      --------------------------------------------------------------------------------
                  Min          -0.9%     -1.5%     -8.6%     -8.7%     +0.0%
                  Max          +0.1%     +1.2%     +7.7%     +7.8%     +2.4%
       Geometric Mean          -0.2%     -0.0%     -0.2%     -0.3%     +0.0%
      
      I checked what happened in 'prolog'.  It seems that we have a
      recursive data structure something like this
      
         f :: [blah]
         f x = build (\cn.  ...g...  )
      
         g :: [blah2]
         g y = ....(foldr k z (f y))....
      
      If we inline 'f' into 'g' we get better fusion than the other
      way round, but we don't have any way to spot that at the moment.
      (I wonder if we could do worker/wrapper for functions returning
      a 'build'?)  It was happening before by a fluke.
      
      Anyway I decided to accept this; it's relatively rare I think.
      517d03e4
  6. 02 Dec, 2016 2 commits
  7. 01 Dec, 2016 5 commits
  8. 30 Nov, 2016 4 commits
    • Ben Gamari's avatar
      Revert "Make globals use sharedCAF" · e2330b6d
      Ben Gamari authored
      This reverts commit 6f7ed1e5 due to breakage of
      the build on Windows.
      e2330b6d
    • Ben Gamari's avatar
      testsuite: Use python3 by default · 605bb9b4
      Ben Gamari authored
      Summary:
      It turns out that Phyx's fix for #12554 (D2684) still fails with mingw-w64
      python 2.7. However, Python 3 (both msys2 and mingw-w64) work fine. Given that
      supporting Python 2 has already become rather tiresome (as @thomie warned it
      would), let's just move to python3 by default.
      
      Test Plan: Validate
      
      Reviewers: austin, Phyx
      
      Reviewed By: Phyx
      
      Subscribers: Phyx, thomie
      
      Differential Revision: https://phabricator.haskell.org/D2766
      
      GHC Trac Issues: #12554
      605bb9b4
    • Tamar Christina's avatar
      Update test output for Windows · dd9ba503
      Tamar Christina authored
      Following D2684 these two tests need to be updated:
      
       * T7037: timeout.exe now waits until all processes are finished.
                this makes T7037 reliable. So enabled.
      
       * T876: Unknown reason, allocations are much lower than before.
      
      Test Plan: ./validate
      
      Reviewers: austin, simonmar, bgamari
      
      Reviewed By: bgamari
      
      Subscribers: thomie, #ghc_windows_task_force
      
      Differential Revision: https://phabricator.haskell.org/D2759
      
      GHC Trac Issues: #12725, #12004
      dd9ba503
    • Tamar Christina's avatar
      Fix testsuite threading, timeout, encoding and performance issues on Windows · 0ce59be3
      Tamar Christina authored
      In a land far far away, a project called Cygwin was born.
      Cygwin used newlib as it's standard C library implementation.
      
      But Cygwin wanted to emulate POSIX systems as closely as possible.
      So it implemented `execv` using the Windows function `spawnve`.
      
      Specifically
      
      ```
      spawnve (_P_OVERLAY, path, argv, cur_environ ())
      ```
      
      `_P_OVERLAY` is crucial, as it makes the function behave *sort of*
      like execv on linux. the child process replaces the original process.
      
      With one major difference because of the difference in process models
      on Windows: the original process signals the caller that it's done.
      
      this is why the file is still locked. because it's still running,
      control was returned because the parent process was destroyed,
      but the child is still running.
      
      I think it's just pure dumb luck, that the older runtimes are slow
      enough to give the process time to terminate before we tried deleting
      the file.  Which explains why you do have sporadic failures even on
      older runtimes like 2.5.0, of a test or two (like T7307).
      
      So this patch fixes a couple of things. I leverage the existing
      `timeout.exe` to implement a workaround for this issue.
      
      a) The old timeout used to start the process then assign it to the job.
         This is slightly faulty since child processes are only assigned to a
         job is their parent were assigned at the time they started. So this
         was a race condition. I now create the process suspended, assign it
         to the job and then resume it. Which means all child processes are
         not running under the same job.
      
      b) First things, Is to prevent dangling child processes. I mark the job
         with `JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE` so when the last process in
         the job is done, it insures all processes under the job are killed.
      
      c) Secondly, I change the way we wait for results. Instead of waiting
         for the parent process to terminate, I wait for the job itself to
         terminate.
      
         There's a slight subtlety there, we can't wait on the job itself.
         Instead we have to create an I/O Completion port and wait for signals
         on it.  See
         https://blogs.msdn.microsoft.com/oldnewthing/20130405-00/?p=4743
      
      This fixes the issues on all runtimes for me and makes T7307 pass
      consistenly.
      
      The threading was also simplified by hiding all the locking in a single
      semaphore and a completion class. Futhermore some additional error
      reporting was added.
      
      For encoding the testsuite now no longer passes a file handle to the
      subprocess since on windows, sh.exe seems to acquire a lock on the file
      that is not released in a timely fashion.
      
      I suspect this because cygwin seems to emulate console handles by
      creating file handles and using those for std handles. So when we give
      it an existing file handle it just locks the file. I what's happening is
      that it's not releasing the handle until all shared cygwin processes are
      dead. Which explains why it worked in single threaded mode.
      
      So now instead we pass a pipe and do not interpret the resulting data.
      
      Any bytes written to stdin or read out of stdout/stderr are done so in
      binary mode and we do not interpret the data. The reason for this is
      that we have encoding tests in GHC which pass invalid utf-8. If we try
      to handle the data as text then python will throw an exception instead
      of a test comparison failing.
      
      Also I have fixed the ability to override `PYTHON` when calling `make
      tests`. This now works the same as with `.\validate`.
      
      Finally, after cleaning up the locks I was able to make the abort
      behavior work correctly as I believe it was intended: when you press
      Ctrl+C and send an interrupt signal, the testsuite finishes the active
      tests and then gracefully exits showing you a report of the progress it
      did make. So using Ctrl+C will not just *die* as it did before.
      
      These changes lift the restriction on which python version you use
      (msys/mingw) or which runtime or python 3 or python 2.  All combinations
      should now be supported.
      
      Test Plan:
      PATH=/usr/local/bin:/mingw64/bin:$APPDATA/cabal/bin:$PATH &&
      PYTHON=/usr/bin/python THREADS=9 make test
      THREADS=9 make test
      PATH=/usr/local/bin:/mingw64/bin:$APPDATA/cabal/bin:$PATH &&
      PYTHON=/usr/bin/python ./validate --quiet --testsuite-only
      
      Reviewers: erikd, RyanGlScott, bgamari, austin
      
      Subscribers: jrtc27, mpickering, thomie, #ghc_windows_task_force
      
      Differential Revision: https://phabricator.haskell.org/D2684
      
      GHC Trac Issues: #12725, #12554, #12661, #12004
      0ce59be3
  9. 29 Nov, 2016 4 commits
  10. 28 Nov, 2016 1 commit
    • Simon Peyton Jones's avatar
      Test Trac #12885 · 27a6bdf0
      Simon Peyton Jones authored
      ...which is fixed by
      
        commit 0476a64e
        Author: Simon Peyton Jones <simonpj@microsoft.com>
        Date:   Tue Oct 25 15:22:17 2016 +0100
      
          Fix a bug in mk_superclasses_of
      27a6bdf0