1. 24 Dec, 2015 1 commit
  2. 23 Dec, 2015 1 commit
  3. 17 Dec, 2015 2 commits
    • Ben Gamari's avatar
      TcTypeable: Don't use bogus fingerprints when suppress-uniques is enabled · 786d528e
      Ben Gamari authored
      Previously the Typeable implementation would intentionally create TyCon
      representations with bogus fingerprints to avoid fingerprints (which may
      change often) from leaking into test output. As pointed out by Richard
      in #10376 this is very bad as simply enabling a debug flag,
      `-dsuppress-uniques`, completely breaks the soundness of `Typeable`!
      
      This patch removes this behavior and replaces it with logic in the
      testsuite driver to filter out spurious changes due to Typeable
      representations.
      
      Test Plan: Validate
      
      Reviewers: austin, simonpj, goldfire
      
      Reviewed By: goldfire
      
      Subscribers: thomie
      
      Differential Revision: https://phabricator.haskell.org/D1629
      
      GHC Trac Issues: #10376
      786d528e
    • Simon Marlow's avatar
      Remote GHCi, -fexternal-interpreter · 4905b83a
      Simon Marlow authored
      Summary:
      (Apologies for the size of this patch, I couldn't make a smaller one
      that was validate-clean and also made sense independently)
      
      (Some of this code is derived from GHCJS.)
      
      This commit adds support for running interpreted code (for GHCi and
      TemplateHaskell) in a separate process.  The functionality is
      experimental, so for now it is off by default and enabled by the flag
      -fexternal-interpreter.
      
      Reaosns we want this:
      
      * compiling Template Haskell code with -prof does not require
        building the code without -prof first
      
      * when GHC itself is profiled, it can interpret unprofiled code, and
        the same applies to dynamic linking.  We would no longer need to
        force -dynamic-too with TemplateHaskell, and we can load ordinary
        objects into a dynamically-linked GHCi (and vice versa).
      
      * An unprofiled GHCi can load and run profiled code, which means it
        can use the stack-trace functionality provided by profiling without
        taking the performance hit on the compiler that profiling would
        entail.
      
      Amongst other things; see
      https://ghc.haskell.org/trac/ghc/wiki/RemoteGHCi for more details.
      
      Notes on the implementation are in Note [Remote GHCi] in the new
      module compiler/ghci/GHCi.hs.  It probably needs more documenting,
      feel free to suggest things I could elaborate on.
      
      Things that are not currently implemented for -fexternal-interpreter:
      
      * The GHCi debugger
      * :set prog, :set args in GHCi
      * `recover` in Template Haskell
      * Redirecting stdin/stdout for the external process
      
      These are all doable, I just wanted to get to a working validate-clean
      patch first.
      
      I also haven't done any benchmarking yet.  I expect there to be slight hit
      to link times for byte code and some penalty due to having to
      serialize/deserialize TH syntax, but I don't expect it to be a serious
      problem.  There's also lots of low-hanging fruit in the byte code
      generator/linker that we could exploit to speed things up.
      
      Test Plan:
      * validate
      * I've run parts of the test suite with
      EXTRA_HC_OPTS=-fexternal-interpreter, notably tests/ghci and tests/th.
      There are a few failures due to the things not currently implemented
      (see above).
      
      Reviewers: simonpj, goldfire, ezyang, austin, alanz, hvr, niteria, bgamari, gibiansky, luite
      
      Subscribers: thomie
      
      Differential Revision: https://phabricator.haskell.org/D1562
      4905b83a
  4. 24 Nov, 2015 1 commit
    • elaforge's avatar
      Rearrange error msgs and add section markers (Trac #11014). · c05fddde
      elaforge authored
      This puts the "Relevant bindings" section at the end.
      
      It uses a TcErrors.Report Monoid to divide messages by importance and
      then mappends them together.  This is not the most efficient way since
      there are various intermediate Reports and list appends, but it probably
      doesn't matter since error messages shouldn't get that large, and are
      usually prepended.  In practice, everything is `important` except
      `relevantBindings`, which is `supplementary`.
      
      ErrMsg's errMsgShortDoc and errMsgExtraInfo were extracted into ErrDoc,
      which has important, context, and suppelementary fields.  Each of those
      three sections is marked with a bullet character, '•' on unicode
      terminals and '*' on ascii terminals.  Since this breaks tons of tests,
      I also modified testlib.normalise_errmsg to strip out '•'s.
      
      --- Additional notes:
      
      To avoid prepending * to an empty doc, I needed to filter empty docs.
      This seemed less error-prone than trying to modify everyone who produces
      SDoc to instead produce Maybe SDoc.  So I added `Outputable.isEmpty`.
      Unfortunately it needs a DynFlags, which is kind of bogus, but otherwise
      I think I'd need another Empty case for SDoc, and then it couldn't be a
      newtype any more.
      
      ErrMsg's errMsgShortString is only used by the Show instance, which is
      in turn only used by Show HscTypes.SourceError, which is in turn only
      needed for the Exception instance.  So it's probably possible to get rid
      of errMsgShortString, but that would a be an unrelated cleanup.
      
      Fixes #11014.
      
      Test Plan: see above
      
      Reviewers: austin, simonpj, thomie, bgamari
      
      Reviewed By: thomie, bgamari
      
      Subscribers: simonpj, nomeata, thomie
      
      Differential Revision: https://phabricator.haskell.org/D1427
      
      GHC Trac Issues: #11014
      c05fddde
  5. 29 Oct, 2015 1 commit
    • thomie's avatar
      Testsuite: report and error out on unfound tests · 032be43b
      thomie authored
      Users are sometimes confused why their test doesn't run. It is usually
      because of a misspelled testname, for example using 'TEST=1234' instead
      of 'TEST=T1234'. After this patch it is hopefully more clear what the
      problem is, showing:
      
          ERROR: tests not found: ['1234']
      
      Instead of:
      
          0 total tests, which gave rise to
          0 test cases, of which
          0 were skipped
      
      Reviewed by: austin, bgamari
      
      Differential Revision: https://phabricator.haskell.org/D1388
      032be43b
  6. 05 Oct, 2015 1 commit
  7. 22 Sep, 2015 1 commit
  8. 20 Sep, 2015 1 commit
    • Edward Z. Yang's avatar
      Always run explicitly requested ways (extra_ways) for fast runs. · b89c4913
      Edward Z. Yang authored
      
      
      To keep validates fast, we only one run one way.  But I think that
      it's important for some tests to run them a few ways, just to
      make sure functionality, e.g. the profiler, is working.  This commit
      changes the logic so that any way specified in extra_ways is always
      run for fast.  The big changes is now profiling tests are run on
      validate.
      
      I also made it so the G1 garbage collector tests only run on slow.
      Signed-off-by: default avatarEdward Z. Yang <ezyang@cs.stanford.edu>
      
      Test Plan: validate
      
      Reviewers: austin, thomie, bgamari
      
      Reviewed By: austin, thomie, bgamari
      
      Subscribers: thomie
      
      Differential Revision: https://phabricator.haskell.org/D1251
      b89c4913
  9. 12 Sep, 2015 1 commit
  10. 03 Sep, 2015 2 commits
  11. 02 Sep, 2015 3 commits
    • thomie's avatar
      Testsuite: delete dead code · bd16e0bc
      thomie authored
      bd16e0bc
    • thomie's avatar
      Testsuite: by default run all tests for a single way · c43c8e2c
      thomie authored
      `make test` now runs all tests for a single way only. Use `make slowtest` to
      get the previous behaviour (i.e. run all tests for all ways).
      
      The intention is to use this new `make test` setting for Phabricator, as
      a reasonable compromise between `make fasttest` (what it previously
      used) and a fullblown `make slowtest` (which runs all tests for all
      ways).
      
      See Note [validate and testsuite speed] in toplevel Makefile.
      
      Differential Revision: https://phabricator.haskell.org/D1178
      c43c8e2c
    • thomie's avatar
      Testsuite: refactoring only · ad26c54b
      thomie authored
      * Rename `platform_wordsize_qualify` to `find_expected_file`, and make
        it return a filename instead of an (absolute) filepath.
      
      * Replace most usages of `qualify` by `in_testdir`. Others usage sites
        will be deleted in a later commit.
      
      These changes will be useful in a later commit, when we'll distinguish
      between files in the source directory and those in a (newly created)
      test directory.
      
      Reviewed by: austin, bgamari
      
      Differential Revision: https://phabricator.haskell.org/D1186
      ad26c54b
  12. 15 Aug, 2015 1 commit
  13. 14 Jul, 2015 2 commits
  14. 13 Jul, 2015 2 commits
  15. 30 Jun, 2015 1 commit
  16. 15 Jun, 2015 1 commit
  17. 12 Jun, 2015 3 commits
    • thomie's avatar
      Testsuite: add function compile_timeout_multiplier (#10345) · 6e542a62
      thomie authored
      And rename timeout_multiplier to run_timeout_multiplier.
      
      timeout_multiplier was added in commit
      a0038979. The name suggested that it
      would affect any test, but it actually only affected tests that had a
      run component, and only that run component (as needed by test T367).
      
      Differential Revision: https://phabricator.haskell.org/D982
      6e542a62
    • thomie's avatar
      Testsuite: diff non-whitespace normalised output (#10152) · 5ddd9041
      thomie authored
      On a test failure, we show a diff between the expected and the actual
      output. The method of how we do this has changed a couple of times:
      
      * In 2007: 9951189c
        "On failure, diff the normalised test outputs"
      
      * In 2011: 3019b1e4
        "When the output files differ, present the diffs between the *actual*
        output, not the normalised output. The latter may have newlines
        removed, making the diff unreadable."
      
      * In 2015 (now): do something in between.
        - Do apply the normalisers again, to make the diff smaller (only
          showing the actual problem).
        - But don't apply normalise_whitespace, as it indeed makes the diff
          unreadable.
      
      Differential Revision: https://phabricator.haskell.org/D984
      5ddd9041
    • thomie's avatar
      Testsuite: fix the little known CHECK_FILES_WRITTEN=1 · c14bd017
      thomie authored
      The testsuite driver has a little known feature to check which files
      each test writes to, whether there are tests that write to same file,
      and whether the tests leave any files behind when CLEANUP=1. It uses
      strace under the hood.
      
      This commit fixes some bitrot, and filters out some more strace lines
      that we're not interested in (and are shown as framework failures
      otherwise).
      
      Differential Revision: https://phabricator.haskell.org/D979
      c14bd017
  18. 11 Jun, 2015 2 commits
    • thomie's avatar
      Testsuite: change some expect_fail tests to expect_broken · 5e66a698
      thomie authored
      Change the following tests from expect_fail to expect_broken: and list
      the ticket number:
      
          * driver/sigof03m/sigof03 (#9252)
          * driver/static001 (#8127)
          * partial-sigs/should_compile/EqualityConstraint (#9478)
          * partial-sigs/should_compile/ExtraNumAMROn (#9478)
          * partial-sigs/should_compile/PatBind2 (#9478)
          * partial-sigs/should_fail/TidyClash2 (#9478)
          * simplCore/should_compile/T8832 (#8832)
      
      The following tests are still marked as expect_fail, but it is not
      clearly documented why so:
      
          * gadt/lazypatok
          * indexed-types/should_fail/SkolemOccursLoop
      
      All other expect_fail tests are only expected to fail on either a
      certain platform/os or for a certain way only.
      
      Differential Revision: https://phabricator.haskell.org/D966
      5e66a698
    • thomie's avatar
      Testsuite Windows: fix T8172 (#8172) · da84fd54
      thomie authored
      Use the new function `normalise_drive_letter` to change D:\ to C:\
      before comparing outputs.
      da84fd54
  19. 09 Jun, 2015 1 commit
    • thomie's avatar
      Revert "The test runner now also works under the msys-native Python." · bb996712
      thomie authored
      To make the test runner work under msys-native Python...
      
      Commit 5258566e broke the msys testsuite
      driver (#10441). It changed the quoting of `config.compiler` from single
      quotes to double quote, which turns out to not be compatible with what
      the function `passThroughCmd` expected.
      
      We could fix `passThroughCmd` to handle the case where `config.compiler`
      is double quoted, and scatter some notes around to make sure the quoting
      done in various places of the testsuite driver stay compatible.
      
      Instead, this commit reverts 101c62e2,
      which introdced the function `passThroughCmd` in the first place
      (#9626). ezyang reports that doing this revert fixes the testsuite
      driver for him using the the following version of msys2:
      
        msys2-keyring r8.3864337-1
        msys2-runtime 2.1.0.16351.cd3184b-1
        msys2-runtime-devel 2.1.0.16351.cd3184b-1
        msys2-w32api-headers 5.0.0.4456.c8b6742-1
        msys2-w32api-runtime 5.0.0.4455.32db221-1
      
      Ideally we'd know what minimum version of msys2 we require, but for now
      this fix is better than nothing.
      
      Only gintas ever reported the original problem, and he actually
      mentioned shortly afterwards: "This may have been fixed by a recent
      release of msys2, but I am not sure."
      
      Differential Revision: https://phabricator.haskell.org/D952
      bb996712
  20. 06 Jun, 2015 1 commit
    • thomie's avatar
      Testsuite: only show output diff when test is expected to pass · ae83a81a
      thomie authored
      Don't let the output of tests that either have missing libraries or are
      expected to be broken obscure real failures.
      
      This makes it easier to analyse the testlogs.
      
      The only consequence is that when a test fails because a certain library
      isn't installed, you have to check the all.T file in which the test is
      defined to actually find out _which_ library that is. Before it would
      print something like
      
          Compile failed (status 256) errors were:
      
          stm052.hs:10:8: error:
           Could not find module ‘System.Random’
           Use -v to see a list of the files searched for.
      
      And now it doesn't. I think this is an acceptable tradeoff.
      
      Differential Revision: https://phabricator.haskell.org/D945
      ae83a81a
  21. 04 Jun, 2015 2 commits
    • thomie's avatar
      Testsuite: add/fix cleanup for certain tests · e340f6ec
      thomie authored
      * extra_clean argument should be a list
        Add an assert to prevent regressions.
      
      * properly clean package conf direcories
        They are directories now, which was causing problems.
      
      * properly clean write_interface_* tests
        We were getting these errors:
      
        [Errno 21] Is a directory: './driver/write_interface_oneshot'
        [Errno 39] Directory not empty: './driver/write_interface_oneshot'
        [Errno 21] Is a directory: './driver/write_interface_make'
        [Errno 39] Directory not empty: './driver/write_interface_make'
      
      * outputdir() is better than -outputdir, as it knows how to (pre)clean
        itself.
      e340f6ec
    • thomie's avatar
      Make validate more quiet · d0063e89
      thomie authored
        * By default use V=0, and call the testsuite with VERBOSE=2, which we
          did before only with validate --quiet. This disables printing the
          test commands it runs.
      
        * When --quiet is used, call the testsuite with VERBOSE=1. This
          disables printing the '====> Scanning' lines, and doesn't print
          which test is being run. So it only prints something when a test
          accidentally prints to stdout or when it fails.
      
          Don't set this option on Travis, as Travis will cancel a build if it
          doesn't see any output for more than 10 minutes.
      
        * When --quiet is used, set the new test option NO_PRINT_SUMMARY,
          which skips printing the test summary. Only the list of unexpected
          failures is printed, if there are any. Note that the full summary
          can still be found in testsuite_summary.txt
      
        * When --quiet is used, don't pass the `-v` flag to `ghc-pkg check`
      
        * When --quiet is used, don't print the Oops! header. It shoud be
          clear from the list of failing tests that something is wrong.
      
      This is all done to get the most out of 30 lines of logfile. These changes can
      be disabled later by simply not passing the --quiet flag to validate.
      
      Differential Revision: https://phabricator.haskell.org/D942
      d0063e89
  22. 28 May, 2015 1 commit
    • Rufflewind's avatar
      Testdriver: don't use os.popen in config/ghc · ef904660
      Rufflewind authored
      Rewrite config/ghc to use getStdout (which use subprocess.Popen) instead
      of os.popen, which is deprecated; this also avoids the use of shell
      
      Also:
      * Move getStdout to driver/testutil.py so both config/ghc and
        driver/runtests.py can use it
      
      * Remove support for Python below 2.4, which doesn't have subprocess
      
      Reviewed By: thomie
      
      Differential Revision: https://phabricator.haskell.org/D908
      ef904660
  23. 23 May, 2015 1 commit
  24. 14 Apr, 2015 1 commit
  25. 03 Apr, 2015 1 commit
  26. 01 Apr, 2015 1 commit
    • thomie's avatar
      Don't `make accept` output of expect_broken tests · d4cf7051
      thomie authored
      When running `make accept` on a directory in the testsuite, don't accept
      the output of tests that are marked as expect_broken. This makes it
      easier to review `git diff` after running `make accept`.
      
      When you change an error message in the compiler that shows up in the
      output of many tests, you can run `make accept` in the testsuite
      directory, and all expected test output will be updated. But since your
      change didn't magically fix all the other bugs in the compiler for which
      we have an expect_broken test, the output for those tests should
      probably not be updated.
      
      Before, the effect of running `make accept` could be that some tests
      would end up in the 'unexpected passes' group.
      
      [skip ci]
      
      Reviewed By: austin
      
      Differential Revision: https://phabricator.haskell.org/D781
      d4cf7051
  27. 31 Mar, 2015 2 commits
  28. 23 Mar, 2015 1 commit
  29. 17 Mar, 2015 1 commit