1. 12 Jun, 2019 3 commits
  2. 04 Jun, 2019 1 commit
  3. 29 May, 2019 1 commit
    • Alp Mestanogullari's avatar
      testsuite: introduce 'static_stats' tests · 7a75a094
      Alp Mestanogullari authored and  Marge Bot's avatar Marge Bot committed
      They are a particular type of perf tests. This patch introduces a
      'stats_files_dir' configuration field in the testsuite driver where all
      haddock timing files (and possibly others in the future) are assumed to live.
      We also change both the Make and Hadrian build systems to pass respectively
      $(TOP)/testsuite/tests/perf/haddock/ and
      <build root>/stage1/haddock-timing-files/ as the value of that new
      configuration field, and to generate the timing files in those directories
      in the first place while generating documentation with haddock.
      
      This new test type can be seen as one dedicated to examining stats files that
      are generated while building a GHC distribution. This also lets us get rid of
      the 'extra_files' directives in the all.T entries for haddock.base,
      haddock.Cabal and haddock.compiler.
      7a75a094
  4. 14 May, 2019 1 commit
  5. 09 Apr, 2019 1 commit
    • Artem Pyanykh's avatar
      codegen: fix memset unroll for small bytearrays, add 64-bit sets · af4cea7f
      Artem Pyanykh authored and  Marge Bot's avatar Marge Bot committed
      Fixes #16052
      
      When the offset in `setByteArray#` is statically known, we can provide
      better alignment guarantees then just 1 byte.
      
      Also, memset can now do 64-bit wide sets.
      
      The current memset intrinsic is not optimal however and can be
      improved for the case when we know that we deal with
      
      (baseAddress at known alignment) + offset
      
      For instance, on 64-bit
      
      `setByteArray# s 1# 23# 0#`
      
      given that bytearray is 8 bytes aligned could be unrolled into
      `movb, movw, movl, movq, movq`; but currently it is
      `movb x23` since alignment of 1 is all we can embed into MO_Memset op.
      af4cea7f
  6. 08 Apr, 2019 1 commit
  7. 21 Mar, 2019 1 commit
  8. 20 Mar, 2019 1 commit
  9. 15 Mar, 2019 1 commit
  10. 07 Mar, 2019 1 commit
    • Ben Gamari's avatar
      testsuite: Introduce fragile modifier · 4ca271d1
      Ben Gamari authored and  Marge Bot's avatar Marge Bot committed
      Now since we have been a bit more stringent in testsuite cleanliness we
      have been marking a lot of tests as fragile using the `skip` modifier.
      However, this unfortunately means that we lose the association with the
      ticket number documenting the fragility.
      
      Here we introduce `fragile` and `fragile_for` to retain this
      information.
      4ca271d1
  11. 24 Feb, 2019 2 commits
  12. 21 Feb, 2019 1 commit
  13. 18 Feb, 2019 1 commit
  14. 16 Feb, 2019 1 commit
  15. 10 Feb, 2019 1 commit
    • Ben Gamari's avatar
      testsuite: Report stdout and stderr in JUnit output · 224fec69
      Ben Gamari authored and  Marge Bot's avatar Marge Bot committed
      This patch makes the JUnit output more useful as now we also report the
      stdout/stderr in the message which can be used to quickly identify why a
      test is failing without downloading the log.
      
      This also introduces TestResult,
      previously we were simply passing around tuples, making things the
      implementation rather difficult to follow and harder to extend.
      224fec69
  16. 04 Feb, 2019 1 commit
  17. 01 Feb, 2019 1 commit
  18. 30 Jan, 2019 5 commits
  19. 27 Jan, 2019 2 commits
  20. 16 Jan, 2019 1 commit
    • Alec Theriault's avatar
      Fix tests for `integer-simple` · d2eb344a
      Alec Theriault authored and Ben Gamari's avatar Ben Gamari committed
      A bunch of tests for `integer-simple` were now broken for a foolish reason:
      unlike the `integer-gmp` case, there is no CorePrep optimization for turning
      small integers directly into applications of `S#`.
      
      Rather than port this optimization to `integer-simple` (which would involve
      moving a bunch of `integer-simple` names into `PrelNames`), I switched
      as many tests as possible to use `Int`.
      
      The printing of `Integer` is already tested in `print037`.
      d2eb344a
  21. 30 Dec, 2018 1 commit
  22. 25 Dec, 2018 1 commit
    • Ben Gamari's avatar
      testsuite: Fix a variety of issues when building with integer-simple · 99378207
      Ben Gamari authored
       * Mark arith011 as broken with integer-simple
      
         As noted in #16091, arith011 fails when run against integer-simple with a
         "divide by zero" exception. This suggests that integer-gmp and integer-simple
         are handling division by zero differently.
      
       * This also fixes broken_without_gmp; the lack of types made the previous
         failure silent, sadly. Improves situation of #16043.
      
       * Mark several tests implicitly depending upon integer-gmp as broken
         with integer-simple. These expect to see Core coming from integer-gmp,
         which breaks with integer-simple.
      
       * Increase runtime timeout multiplier of T11627a with integer-simple
      
         I previously saw that T11627a timed out in all profiling ways when run against
         integer-simple. I suspect this is due to integer-simple's rather verbose heap
         representation. Let's see whether increasing the runtime timeout helps.
      
         Fixes test for #11627.
      
      This is all in service of fixing #16043.
      99378207
  23. 23 Dec, 2018 2 commits
  24. 14 Dec, 2018 1 commit
  25. 13 Dec, 2018 2 commits
    • Ben Gamari's avatar
      testsuite: Normalise away spurious differences in out-of-scope instances · 9d9f4c9a
      Ben Gamari authored
      This fixes a variety of testsuite failures with integer-simple of the form
      
      ```
      --- typecheck/should_fail/tcfail072.run/tcfail072.stderr.normalised
      +++ typecheck/should_fail/tcfail072.run/tcfail072.comp.stderr.normalised
      @@ -12,7 +12,7 @@
                 -- Defined in ‘integer-<IMPL>-<VERSION>:GHC.Integer.Type’
               instance Ord () -- Defined in ‘GHC.Classes’
               ...plus 21 others
      -        ...plus three instances involving out-of-scope types
      +        ...plus two instances involving out-of-scope types
               (use -fprint-potential-instances to see them all)
            In the expression: g A
             In an equation for ‘g’: g (B _ _) = g A
      ```
      
      In service of fixing #16043.
      9d9f4c9a
    • Ben Gamari's avatar
      testsuite: Fix a number of GHCi-related failures due to integer-simple · 14500dab
      Ben Gamari authored
      Towards fixing #16043.
      14500dab
  26. 22 Nov, 2018 2 commits
    • Tamar Christina's avatar
      testuite: update more windows tests outputs · 67277e7c
      Tamar Christina authored and Ben Gamari's avatar Ben Gamari committed
      Test Plan: ./validate
      
      Reviewers: bgamari, simonmar
      
      Subscribers: rwbarton, carter
      
      Differential Revision: https://phabricator.haskell.org/D5362
      67277e7c
    • Krzysztof Gogolewski's avatar
      Fix GhostScript detection (Trac #15856) · 011e39d7
      Krzysztof Gogolewski authored and Ben Gamari's avatar Ben Gamari committed
      The option `confdir` (used in GhostScript test) was set correctly
      via `--config` in `test.mk` and incorrectly via `config/ghc`.
      AFAICT, some time ago this was working because the
      incorrect assignment was done first, and later it broke.
      
      Hardian doesn't pass `confdir`. I removed `confdir` and use
      `config.top` to determine the directory of the
      `good.ps` and `bad.ps` files. This is simpler.
      
      I also removed some redundant assignments in `config/ghc`.
      
      Test Plan: manually set config.have_profiling and make test
      
      Reviewers: bgamari
      
      Subscribers: rwbarton, carter
      
      GHC Trac Issues: #15856
      
      Differential Revision: https://phabricator.haskell.org/D5298
      011e39d7
  27. 07 Nov, 2018 1 commit
    • davide's avatar
      testsuite: Save performance metrics in git notes. · 932cd41d
      davide authored and Ben Gamari's avatar Ben Gamari committed
      This patch makes the following improvement:
        - Automatically records test metrics (per test environment) so that
          the programmer need not supply nor update expected values in *.T
          files.
          - On expected metric changes, the programmer need only indicate the
            direction of change in the git commit message.
        - Provides a simple python tool "perf_notes.py" to compare metrics
          over time.
      
      Issues:
        - Using just the previous commit allows performance to drift with each
          commit.
          - Currently we allow drift as we have a preference for minimizing
            false positives.
          - Some possible alternatives include:
            - Use metrics from a fixed commit per test: the last commit that
              allowed a change in performance (else the oldest metric)
            - Or use some sort of aggregate since the last commit that allowed
              a change in performance (else all available metrics)
            - These alternatives may result in a performance issue (with the
              test driver) having to heavily search git commits/notes.
        - Run locally, performance tests will trivially pass unless the tests
          were run locally on the previous commit. This is often not the case
          e.g.  after pulling recent changes.
      
      Previously, *.T files contain statements such as:
      ```
      stats_num_field('peak_megabytes_allocated', (2, 1))
      compiler_stats_num_field('bytes allocated',
                               [(wordsize(64), 165890392, 10)])
      ```
      This required the programmer to give the expected values and a tolerance
      deviation (percentage). With this patch, the above statements are
      replaced with:
      ```
      collect_stats('peak_megabytes_allocated', 5)
      collect_compiler_stats('bytes allocated', 10)
      ```
      So that programmer must only enter which metrics to test and a tolerance
      deviation. No expected value is required. CircleCI will then run the
      tests per test environment and record the metrics to a git note for that
      commit and push them to the git.haskell.org ghc repo. Metrics will be
      compared to the previous commit. If they are different by the tolerance
      deviation from the *.T file, then the corresponding test will fail. By
      adding to the git commit message e.g.
      ```
       # Metric (In|De)crease <metric(s)> <options>: <tests>
      Metric Increase ['bytes allocated', 'peak_megabytes_allocated'] \
               (test_env='linux_x86', way='default'):
          Test012, Test345
      Metric Decrease 'bytes allocated':
          Test678
      Metric Increase:
          Test711
      ```
      This will allow the noted changes (letting the test pass). Note that by
      omitting metrics or options, the change will apply to all possible
      metrics/options (i.e. in the above, an increase for all metrics in all
      test environments is allowed for Test711)
      
      phabricator will use the message in the description
      
      Reviewers: bgamari, hvr
      
      Reviewed By: bgamari
      
      Subscribers: rwbarton, carter
      
      GHC Trac Issues: #12758
      
      Differential Revision: https://phabricator.haskell.org/D5059
      932cd41d
  28. 05 Sep, 2018 1 commit
  29. 12 Aug, 2018 1 commit