1. 17 Mar, 2020 3 commits
    • Ömer Sinan Ağacan's avatar
      Update sanity checking for TSOs: · 92327e3a
      Ömer Sinan Ağacan authored
      - Remove an invalid assumption about GC checking what_next field. The GC
        doesn't care about what_next at all, if a TSO is reachable then all
        its pointers are followed (other than global_tso, which is only
        followed by compacting GC).
      
      - Remove checkSTACK in checkTSO: TSO stacks will be visited in
        checkHeapChain, or checkLargeObjects etc.
      
      - Add an assertion in checkTSO to check that the global_link field is
        sane.
      
      - Did some refactor to remove forward decls in checkGlobalTSOList and
        added braces around single-statement if statements.
      92327e3a
    • Sylvain Henry's avatar
      Modules: Core (#13009) · 18a346a4
      Sylvain Henry authored
      Update submodule: haddock
      18a346a4
    • Xia Li-yao's avatar
  2. 15 Mar, 2020 6 commits
    • Ömer Sinan Ağacan's avatar
      Fix global_link of TSOs for threads reachable via dead weaks · cfcc3c9a
      Ömer Sinan Ağacan authored
      Fixes #17785
      
      Here's how the problem occurs:
      
      - In generation 0 we have a TSO that is finished (i.e. it has no more
        work to do or it is killed).
      
      - The TSO only becomes reachable after collectDeadWeakPtrs().
      
      - After collectDeadWeakPtrs() we switch to WeakDone phase where we don't
        move TSOs to different lists anymore (like the next gen's thread list
        or the resurrected_threads list).
      
      - So the TSO will never be moved to a generation's thread list, but it
        will be promoted to generation 1.
      
      - Generation 1 collected via mark-compact, and because the TSO is
        reachable it is marked, and its `global_link` field, which is bogus at
        this point (because the TSO is not in a list), will be threaded.
      
      - Chaos ensues.
      
      In other words, when these conditions hold:
      
      - A TSO is reachable only after collectDeadWeakPtrs()
      - It's finished (what_next is ThreadComplete or ThreadKilled)
      - It's retained by mark-compact collector (moving collector doesn't
        evacuate the global_list field)
      
      We end up doing random mutations on the heap because the TSO's
      global_list field is not valid, but it still looks like a heap pointer
      so we thread it during compacting GC.
      
      The fix is simple: when we traverse old_threads lists to resurrect
      unreachable threads the threads that won't be resurrected currently
      stays on the old_threads lists. Those threads will never be visited
      again by MarkWeak so we now reset the global_list fields. This way
      compacting GC does not thread pointers to nowhere.
      
      Testing
      -------
      
      The reproducer in #17785 is quite large and hard to build, because of
      the dependencies, so I'm not adding a regression test.
      
      In my testing the reproducer would take a less than 5 seconds to run,
      and once in every ~5 runs would fail with a segfault or an assertion
      error. In other cases it also fails with a test failure. Because the
      tests never fail with the bug fix, assuming the code is correct, this
      also means that this bug can sometimes lead to incorrect runtime
      results.
      
      After the fix I was able to run the reproducer repeatedly for about an
      hour, with no runtime crashes or test failures.
      
      To run the reproducer clone the git repo:
      
          $ git clone https://github.com/osa1/streamly --branch ghc-segfault
      
      Then clone primitive and atomic-primops from their git repos and point
      to the clones in cabal.project.local. The project should then be
      buildable using GHC HEAD. Run the executable `properties` with `+RTS -c
      -DZ`.
      
      In addition to the reproducer above I run the test suite using:
      
          $ make slowtest EXTRA_HC_OPTS="-debug -with-rtsopts=-DS \
              -with-rtsopts=-c +RTS -c -RTS" SKIPWAY='nonmoving nonmoving_thr'
      
      This enables compacting GC always in both GHC when building the test
      programs and when running the test programs, and also enables sanity
      checking when running the test programs. These set of flags are not
      compatible for all tests so there are some failures, but I got the same
      set of failures with this patch compared to GHC HEAD.
      cfcc3c9a
    • Judah Jacobson's avatar
      Add a -no-haddock flag. · c35c545d
      Judah Jacobson authored
      This flag undoes the effect of a previous "-haddock" flag.  Having both flags makes it easier
      for build systems to enable Haddock parsing in a set of global flags, but then disable it locally for
      specific targets (e.g., third-party packages whose comments don't pass the validation in the latest GHC).
      
      I added the flag to expected-undocumented-flags.txt since `-haddock` was alreadyin that list.
      c35c545d
    • Sylvain Henry's avatar
      Refactor CmmToAsm (disentangle DynFlags) · 2e82465f
      Sylvain Henry authored
      This patch disentangles a bit more DynFlags from the native code
      generator (CmmToAsm).
      
      In more details:
      
      - add a new NCGConfig datatype in GHC.CmmToAsm.Config which contains the
        configuration of a native code generation session
      - explicitly pass NCGConfig/Platform arguments when necessary
      - as a consequence `sdocWithPlatform` is gone and there are only a few
        `sdocWithDynFlags` left
      - remove the use of `unsafeGlobalDynFlags` from GHC.CmmToAsm.CFG
      - remove `sdocDebugLevel` (now we pass the debug level via NCGConfig)
      
      There are still some places where DynFlags is used, especially because
      of pretty-printing (CLabel), because of Cmm helpers (such as
      `cmmExprType`) and because of `Outputable` instance for the
      instructions. These are left for future refactoring as this patch is
      already big.
      2e82465f
    • vdukhovni's avatar
      Note platform-specific Foreign.C.Types in context · dd6ffe6b
      vdukhovni authored
      Also fix the markup in the general note at the top of the module.  Haddock
      (usability trade-off), does not support multi-line emphasised text.
      dd6ffe6b
    • Brian Foley's avatar
      Remove some dead code · b4774598
      Brian Foley authored
      From the notes.ghc.drop list found using weeder in #17713
      b4774598
    • Krzysztof Gogolewski's avatar
      Document restriction on SCC pragma syntax · d30aeb4b
      Krzysztof Gogolewski authored
      Currently, the names of cost centres must be quoted or
      be lowercase identifiers.
      
      Fixes #17916.
      d30aeb4b
  3. 14 Mar, 2020 18 commits
    • Krzysztof Gogolewski's avatar
      1de3ab4a
    • Ömer Sinan Ağacan's avatar
      49e9d739
    • cgibbard's avatar
      Enable stage1 build of haddock · 3f116d35
      cgibbard authored
      The submodule has already been bumped to contain the fix.
      3f116d35
    • Vladislav Zavialov's avatar
      Remove second tcLookupTcTyCon in tcDataDefn · bee4cdad
      Vladislav Zavialov authored
      Before this patch, tcDataDefn used to call tcLookupTcTyCon twice in a row:
      	1. in bindTyClTyVars itself
      	2. in the continuation passed to it
      
      Now bindTyClTyVars passes the TcTyCon to the continuation, making
      the second lookup unnecessary.
      bee4cdad
    • Ben Gamari's avatar
      base: Make `open` calls interruptible · 93c88c26
      Ben Gamari authored
      As noted in #17912, `open` system calls were `safe` rather than
      `interruptible`. Consequently, the program could not be interrupted with
      SIGINT if stuck in a slow open operation. Fix this by marking
      `c_safe_open` as interruptible.
      93c88c26
    • Simon Peyton Jones's avatar
      Refactoring in TcSMonad · 73133a3b
      Simon Peyton Jones authored
      This patch is just refactoring: no change in
      behaviour.
      
      I removed the rather complicated
          checkConstraintsTcS
          checkTvConstraintsTcS
      
      in favour of simpler functions
          emitImplicationTcS
          emitTvImplicationTcS
          pushLevelNoWorkList
      
      The last of these is a little strange, but overall
      it's much better I think.
      73133a3b
    • Simon Peyton Jones's avatar
      Wrap an implication around class-sig kind errors · e3c374cc
      Simon Peyton Jones authored
      Ticket #17841 showed that we can get a kind error
      in a class signature, but lack an enclosing implication
      that binds its skolems.
      
      This patch
      
      * Adds the wrapping implication: the new call to
        checkTvConstraints in tcClassDecl1
      
      * Simplifies the API to checkTvConstraints, which
        was not otherwise called at all.
      
      * Simplifies TcErrors.report_unsolved by *not*
        initialising the TidyEnv from the typechecker lexical
        envt.  It's enough to do so from the free vars of the
        unsolved constraints; and we get silly renamings if
        we add variables twice: once from the lexical scope
        and once from the implication constraint.
      e3c374cc
    • Ben Gamari's avatar
      8b95ddd3
    • Simon Peyton Jones's avatar
      Improve CSE.combineAlts · 88f7a762
      Simon Peyton Jones authored
      This patch improves the way that CSE combines identical
      alternatives.  See #17901.
      
      I'm still not happy about the duplication between CSE.combineAlts
      and GHC.Core.Utils.combineIdenticalAlts; see the Notes with those
      functions.  But this patch is a step forward.
      
      Metric Decrease:
          T12425
          T5642
      88f7a762
    • Simon Peyton Jones's avatar
      Simple refactor of cheapEqExpr · 2f8c7767
      Simon Peyton Jones authored
      No change in functionality.  Just seems tidier (and signficantly more
      efficient) to deal with ticks directly than to call stripTicksTopE.
      2f8c7767
    • Ben Gamari's avatar
      nonmoving: Remove redundant bitmap clearing · fdfa2d01
      Ben Gamari authored
      nonmovingSweep already clears the bitmap in the sweep loop. There is no
      reason to do so a second time.
      fdfa2d01
    • Ben Gamari's avatar
      nonmoving: Don't traverse filled segment list in pause · 20d4d676
      Ben Gamari authored
      The non-moving collector would previously walk the entire filled segment
      list during the preparatory pause. However, this is far more work than
      is strictly necessary. We can rather get away with merely collecting the
      allocators' filled segment list heads and process the lists themselves
      during the concurrent phase. This can significantly reduce the maximum
      gen1 GC pause time in programs with high rates of long-lived allocations.
      20d4d676
    • Judah Jacobson's avatar
      7c3e39a9
    • Judah Jacobson's avatar
      Allow overriding LD_STAGE0 and AR_STAGE0 in the configure script. · 8f7dd571
      Judah Jacobson authored
      Previously it was possible to override the stage0 C compiler via `CC_STAGE0`,
      but you couldn't override `ld` or `ar` in stage0.  This change allows overriding them
      by setting `LD_STAGE0` or `AR_STAGE0`, respectively.
      
      Our team uses this feature internally to take more control of our GHC build
      and make it run more hermetically.
      8f7dd571
    • Sylvain Henry's avatar
      Use correct option name (-opti) (fix #17314) · 7432b327
      Sylvain Henry authored
      s/pgmo/opti
      7432b327
    • Sylvain Henry's avatar
      Make: refactor GMP rules · 4f117135
      Sylvain Henry authored
      Document and use simpler rules for the ghc-gmp.h header.
      4f117135
    • Sylvain Henry's avatar
      Hadrian: fix absolute buildroot support (#17822) · b989845e
      Sylvain Henry authored
      Shake's "**" wildcard doesn't match absolute root. We must use "//" instead.
      b989845e
    • Simon Peyton Jones's avatar
      Fix Lint · c12a2ec5
      Simon Peyton Jones authored
      Ticket #17590 pointed out a bug in the way the linter dealt with
      type lets, exposed by the new uniqAway story.
      
      The fix is described in Note [Linting type lets]. I ended up
      putting the in-scope Ids in a different env field, le_ids,
      rather than (as before) sneaking them into the TCvSubst.
      
      Surprisingly tiresome, but done.
      
      Metric Decrease:
          hie002
      c12a2ec5
  4. 13 Mar, 2020 5 commits
    • Ben Gamari's avatar
      gitlab-ci: Distinguish integer-simple test envs · 7f25557a
      Ben Gamari authored
      Previously two integer-simple jobs declared the same test environment. One (the nightly job) was built in the perf way, the other in the validate way. Consequently they had appreciably different performance characteristics, causing in the nightly job to spuriously fail with performance changes.
      7f25557a
    • Ben Gamari's avatar
      gitlab-ci: Rework triggering of release builds · f124ff0d
      Ben Gamari authored
      Use a push option instead of tagging.
      f124ff0d
    • Paavo Parkkinen's avatar
      Update documentation for closureSize · 2f292db8
      Paavo Parkkinen authored
      2f292db8
    • Sylvain Henry's avatar
      Rename isDllName · 44fad4a9
      Sylvain Henry authored
      I wanted to fix the dangling comment in `isDllName` ("This is the cause
      of #", #8696 is already mentioned earlier). I took the opportunity to
      change the function name to better reflect what it does.
      44fad4a9
    • Alp Mestanogullari's avatar
      hadrian: improve dependency tracking for the check-* programs · 6a65b8c2
      Alp Mestanogullari authored
      The code in Rules.Register responsible for finding all the build artifacts
      that Cabal installs when registering a library (static/shared libs, .hi files,
      ...) was looking in the wrong place. This patch fixes that logic and makes sure
      we gather all those artifacts in a list to declare that the rule for a given
      `.conf` file, our proxy for "Hadrian, please install this package in the package
      db for this stage", also produces those artifacts under the said package
      database.
      
      We also were completely missing some logic to declare that the check-* programs
      have dependencies besides their source code, at least when testing an in-tree
      compiler.
      
      Finally, this patch also removes redundant packages from 'testsuitePackages',
      since they should already be covered by the stage<N>Packages lists from
      Settings.Default.
      
      With this patch, after a complete build and freezing stage 1, a change to
      `compiler/parser/Parser.y` results in rebuilding the ghc lib, reinstalling it,
      and rebuilding the few programs that depend on it, _including_ `check-ppr` and
      `check-api-annotations` (therefore fixing #17273).
      6a65b8c2
  5. 12 Mar, 2020 7 commits
    • Simon Peyton Jones's avatar
      Expose compulsory unfoldings always · 3a259092
      Simon Peyton Jones authored
      The unsafeCoerce# patch requires that unsafeCoerce# has
      a compulsory unfolding that is always available.  So we have
      to be careful to expose compulsory unfoldings unconditionally
      and consistently.
      
      We didn't get this quite right: #17871.  This patch fixes
      it.  No real surprises here.
      
      See Note [Always expose compulsory unfoldings] in GHC.Iface.Tidy
      3a259092
    • Kirill Elagin's avatar
      pretty-printer: Do not print ApplicativeDo join · 5cb93af7
      Kirill Elagin authored
      * Do not print `join` in ApplictiveStmt, unless ppr-debug
      * Print parens around multiple parallel binds
      
      When ApplicativeDo is enabled, the renamer analyses the statements of a
      `do` block and in certain cases marks them as needing to be rewritten
      using `join`.
      
      For example, if you have:
      
      ```
      foo = do
        a <- e1
        b <- e2
        doSomething a b
      ```
      
      it will be desugared into:
      
      ```
      foo = join (doSomething <$> e1 <*> e2)
      ```
      
      After renaming but before desugaring the expression is stored
      essentially as:
      
      ```
      foo = do
        [will need join] (a <- e1 | b <- e2)
        [no return] doSomething a b
      ```
      
      Before this change, the pretty printer would print a call to `join`,
      even though it is not needed at this stage at all. The expression will be
      actually rewritten into one using join only at desugaring, at which
      point a literal call to join will be inserted.
      5cb93af7
    • Kirill Elagin's avatar
      pretty-printer: Properly parenthesise LastStmt · 1f9db3e7
      Kirill Elagin authored
      After ApplicatveDo strips the last `return` during renaming, the pretty
      printer has to restore it. However, if the return was followed by `$`,
      the dollar was stripped too and not restored.
      
      For example, the last stamement in:
      
      ```
        foo = do
          x <- ...
          ...
          return $ f x
      ```
      
      would be printed as:
      
      ```
          return f x
      ```
      
      This commit preserved the dolar, so it becomes:
      
      ```
          return $ f x
      ```
      1f9db3e7
    • Ryan Scott's avatar
      Make DeriveFunctor-generated code require fewer beta reductions · cb93a1a4
      Ryan Scott authored
      Issue #17880 demonstrates that `DeriveFunctor`-generated code is
      surprisingly fragile when rank-_n_ types are involved. The culprit is
      that `$fmap` (the algorithm used to generate `fmap` implementations)
      was too keen on applying arguments with rank-_n_ types to lambdas,
      which fail to typecheck more often than not.
      
      In this patch, I change `$fmap` (both the specification and the
      implementation) to produce code that avoids creating as many lambdas,
      avoiding problems when rank-_n_ field types arise.
      See the comments titled "Functor instances" in `TcGenFunctor` for a
      more detailed description. Not only does this fix #17880, but it also
      ensures that the code that `DeriveFunctor` generates will continue
      to work after simplified subsumption is implemented (see #17775).
      
      What is truly amazing is that #17880 is actually a regression
      (introduced in GHC 7.6.3) caused by commit
      49ca2a37, the fix #7436. Prior to
      that commit, the version of `$fmap` that was used was almost
      identical to the one used in this patch! Why did that commit change
      `$fmap` then? It was to avoid severe performance issues that would
      arise for recursive `fmap` implementations, such as in the example
      below:
      
      ```hs
      data List a = Nil | Cons a (List a) deriving Functor
      
      -- ===>
      
      instance Functor List where
        fmap f Nil = Nil
        fmap f (Cons x xs) = Cons (f x) (fmap (\y -> f y) xs)
      ```
      
      The fact that `\y -> f y` was eta expanded caused significant
      performance overheads. Commit
      49ca2a37 fixed this performance
      issue, but it went too far. As a result, this patch partially
      reverts 49ca2a37.
      
      To ensure that the performance issues pre-#7436 do not resurface,
      I have taken some precautionary measures:
      
      * I have added a special case to `$fmap` for situations where the
        last type variable in an application of some type occurs directly.
        If this special case fires, we avoid creating a lambda expression.
        This ensures that we generate
        `fmap f (Cons x xs) = Cons (f x) (fmap f xs)` in the derived
        `Functor List` instance above. For more details, see
        `Note [Avoid unnecessary eta expansion in derived fmap implementations]`
        in `TcGenFunctor`.
      * I have added a `T7436b` test case to ensure that the performance
        of this derived `Functor List`-style code does not regress.
      
      When implementing this, I discovered that `$replace`, the algorithm
      which generates implementations of `(<$)`, has a special case that is
      very similar to the `$fmap` special case described above. `$replace`
      marked this special case with a custom `Replacer` data type, which
      was a bit overkill. In order to use the same machinery for both
      `Functor` methods, I ripped out `Replacer` and instead implemented
      a simple way to detect the special case. See the updated commentary
      in `Note [Deriving <$]` for more details.
      cb93a1a4
    • Sylvain Henry's avatar
      Use a Set to represent Ways · a6989971
      Sylvain Henry authored
      Should make `member` queries faster and avoid messing up with missing
      `nubSort`.
      
      Metric Increase:
          hie002
      a6989971
    • Sylvain Henry's avatar
      Refactor interpreterDynamic and interpreterProfiled · bc41e471
      Sylvain Henry authored
      * `interpreterDynamic` and `interpreterProfiled` now take `Interp`
        parameters instead of DynFlags
      
      * slight refactoring of `ExternalInterp` so that we can read the iserv
        configuration (which is pure) without reading an MVar.
      bc41e471
    • Sylvain Henry's avatar
      Refactor GHC.Driver.Session (Ways and Flags) · 8e6febce
      Sylvain Henry authored
      * extract flags and ways into their own modules (with some renaming)
      
      * remove one SOURCE import of GHC.Driver.Session from GHC.Driver.Phases
      
      * when GHC uses dynamic linking (WayDyn), `interpWays` was only
        reporting WayDyn even if the host was profiled (WayProf).  Now it
        returns both as expected (might fix #16803).
      
      * `mkBuildTag :: [Way] -> String` wasn't reporting a canonical tag for
        differently ordered lists. Now we sort and nub the list to fix this.
      8e6febce
  6. 11 Mar, 2020 1 commit
    • Ömer Sinan Ağacan's avatar
      Zero any slop after compaction in compacting GC · 3aa9b35f
      Ömer Sinan Ağacan authored
      In copying GC, with the relevant debug flags enabled, we release the old
      blocks after a GC, and the block allocator zeroes the space before
      releasing a block. This effectively zeros the old heap.
      
      In compacting GC we reuse the blocks and previously we didn't zero the
      unused space in a compacting generation after compaction. With this
      patch we zero the slop between the free pointer and the end of the block
      when we're done with compaction and when switching to a new block
      (because the current block doesn't have enough space for the next object
      we're shifting).
      3aa9b35f