1. 22 Sep, 2019 14 commits
    • Daniel Gröber (dxld)'s avatar
      rts: retainer: Remove outdated invariants on traversePushStack · cb7220b3
      Daniel Gröber (dxld) authored
      These invariants don't seem to make any sense in the current code.  The
      text talks about c_child_r as if it were an StgClosure, for which RSET()
      would make sense, but it's a retainer aka 'CostCentreStack*'.
    • Daniel Gröber (dxld)'s avatar
      rts: retainer: Cleanup comments and strings for traversal extraction · bd78b696
      Daniel Gröber (dxld) authored
      A lot of comments and strings are still talking about old names, fix
    • Daniel Gröber (dxld)'s avatar
      rts: retainer: Reduce DEBUG_RETAINER ifdef noise · 64ec45a7
      Daniel Gröber (dxld) authored
      Keeping track of the maximum stack seems like a good idea in all
      configurations. The associated ASSERTs only materialize in debug mode but
      having the statistic is nice.
      To make the debug code less prone to bitrotting I introduce a function
      'debug()' which doesn't actually print by default and is #define'd away
      only when the standard DEBUG define is off.
    • Daniel Gröber (dxld)'s avatar
      rts: retainer: Rename heap traversal functions for extraction · b7e15d17
      Daniel Gröber (dxld) authored
      This gets all remaining functions in-line with the new 'traverse' prefix
      and module name.
    • Daniel Gröber (dxld)'s avatar
      rts: retainer: Remove obsolete debug code · ec1d76e2
      Daniel Gröber (dxld) authored
      Commit dbef766c ("Profiling cleanup.") made this debug code obsolete by
      removing the 'cost' function without a replacement. As best I can tell the
      retainer profiler used to do some heap census too and this debug code was
      mainly concerned with that.
    • Daniel Gröber (dxld)'s avatar
      rts: RetainerSet: Remove obsolete fist/second-approach choice · f3bb7397
      Daniel Gröber (dxld) authored
      In the old code when DEBUG_RETAINER was set, FIRST_APPROACH is
      implied. However ProfHeap.c now depends on printRetainerSetShort which is
      only available with SECOND_APPROACH. This is because with FIRST_APPROACH
      retainerProfile() will free all retainer sets before returning so by the
      time ProfHeap calls dumpCensus the retainer set pointers are segfaulty.
      Since all of this debugging code obviously hasn't been compiled in ages
      anyways I'm taking the liberty of just removing it.
      Remember guys: Dead code is a liability not an asset :)
    • Daniel Gröber (dxld)'s avatar
      rts: retainer: simplify pop() control flow · 48e816f0
      Daniel Gröber (dxld) authored
      Instead of breaking out of the switch-in-while construct using `return` this
      uses `goto out` which makes it possible to share a lot of the out-variable
      assignment code in all the cases.
      I also replaced the nasty `while(true)` business by the real loop
      condition: `while(*c == NULL)`. All `break` calls inside the switch aready
      have either a check for NULL or an assignment of `c` to NULL so this should
      not change any behaviour.
      Using `goto out` also allowed me to remove another minor wart: In the
      MVAR_*/WEAK cases the popOff() call used to happen before reading the
      stackElement. This looked like a use-after-free hazard to me as the stack
      is allocated in blocks and depletion of a block could mean it getting freed
      and possibly overwritten by zero or garbage, depending on the block
      allocator's behaviour.
    • Daniel Gröber (dxld)'s avatar
      rts: retainer: Pull retainer specific code into a callback · b03db9da
      Daniel Gröber (dxld) authored
      This essentially turns the heap traversal code into a visitor. You add a
      bunch of roots to the work-stack and then the callback you give to
      traverseWorkStack() will be called with every reachable closure at least
    • Daniel Gröber (dxld)'s avatar
      rts: Generalise profiling heap traversal flip bit handling · 2f2f6dd5
      Daniel Gröber (dxld) authored
      This commit starts renaming some flip bit related functions for the
      generalised heap traversal code and adds provitions for sharing the
      per-closure profiling header field currently used exclusively for retainer
      profiling with other heap traversal profiling modes.
    • Daniel Gröber (dxld)'s avatar
    • Daniel Gröber (dxld)'s avatar
      rts: retainer: Generalise per-stackElement data · f79ac2ef
      Daniel Gröber (dxld) authored
      This essentially ammounts to s/retainer/stackData/, s/c_child_r/data/ and
      some temporary casting of c_child_r to stackData until refactoring of this
      module is completed by a subsequent commit. We also introduce a new union
      'stackData' which will contain the actual extra data to be stored on the
      The idea is to make the heap traversal logic of the retainer profiler ready
      for extraction into it's own module. So talking about "retainers" there
      doesn't really make sense anymore.
      Essentially the "retainers" we store in the stack are just data associated
      with the push()ed closures which we return when pop()ing it.
    • Daniel Gröber (dxld)'s avatar
      rts: retainer: Move info.next.parent to stackElement · 94ecdb4f
      Daniel Gröber (dxld) authored
      I don't see a point in having this live in 'info', just seems to make the
      code more complicated.
    • Daniel Gröber (dxld)'s avatar
      rts: retainer: Turn global traversal state into a struct · ead05f80
      Daniel Gröber (dxld) authored
      Global state is ugly and hard to test. Since the profiling code isn't quite
      as performance critical as, say, GC we should prefer better code here.
      I would like to move the 'flip' bit into the struct too but that's
      complicated by the fact that the defines which use it directly are also
      called from ProfHeap where the traversalState is not easily
      available. Maybe in a future commit.
    • Daniel Gröber (dxld)'s avatar
      rts: retainer: Remove cStackSize debug counter · da12da79
      Daniel Gröber (dxld) authored
      This can only ever be one since 5f1d949a ("Remove explicit recursion in
      retainer profiling"), so it's pointless.
  2. 09 Jun, 2019 1 commit
    • Daniel Gröber (dxld)'s avatar
      rts: Fix RetainerProfile early return with TREC_CHUNK · 8e60e3f0
      Daniel Gröber (dxld) authored
      When pop() returns with `*c == NULL` retainerProfile will immediately
      return. All other code paths is pop() continue with the next stackElement
      when this happens so it seems weird to me that TREC_CHUNK we would suddenly
      abort everything even though the stack might still have elements left to
  3. 13 Dec, 2018 1 commit
  4. 05 Dec, 2018 1 commit
    • Alexander Vershilov's avatar
      Remove explicit recursion in retainer profiling (fixes #14758) · 5f1d949a
      Alexander Vershilov authored
      Retainer profiling contained a recursion that under
      certain circumstances could lead to the stack overflow
      in C code.
      The idea of the improvement is to keep an explicit stack for the
      object, more precise to reuse existing stack, but allow new type of
      objects to be stored there.
      There is no reliable reproducer that is not a big program
      but in some cases foldr (+) 0 [1..10000000] can work.
      Reviewers: bgamari, simonmar, erikd, osa1
      Reviewed By: bgamari, osa1
      Subscribers: osa1, rwbarton, carter
      GHC Trac Issues: #14758
      Differential Revision: https://phabricator.haskell.org/D5351
  5. 07 Sep, 2018 1 commit
    • Ömer Sinan Ağacan's avatar
      Various RTS bug fixes: · d9a26c7e
      Ömer Sinan Ağacan authored
      - Retainer profiler: init_srt_thunk() should mark the stack entry as SRT
      - Retainer profiler: Remove an incorrect assertion about FUN_STATIC.
        FUN_STATIC does not have to have an SRT.
      - Fix nptrs of BCO
      Test Plan: validate
      Reviewers: simonmar, bgamari, erikd
      Reviewed By: simonmar
      Subscribers: rwbarton, carter
      Differential Revision: https://phabricator.haskell.org/D5134
  6. 29 Aug, 2018 1 commit
    • David Feuer's avatar
      Finish stable split · f48e276a
      David Feuer authored
      Long ago, the stable name table and stable pointer tables were one.
      Now, they are separate, and have significantly different
      implementations. I believe the time has come to finish the split
      that began in #7674.
      * Divide `rts/Stable` into `rts/StableName` and `rts/StablePtr`.
      * Give each table its own mutex.
      * Add FFI functions `hs_lock_stable_ptr_table` and
      `hs_unlock_stable_ptr_table` and document them.
        These are intended to replace the previously undocumented
      `hs_lock_stable_tables` and `hs_lock_stable_tables`,
        which are now documented as deprecated synonyms.
      * Make `eqStableName#` use pointer equality instead of unnecessarily
      comparing stable name table indices.
      Reviewers: simonmar, bgamari, erikd
      Reviewed By: bgamari
      Subscribers: rwbarton, carter
      GHC Trac Issues: #15555
      Differential Revision: https://phabricator.haskell.org/D5084
  7. 27 Aug, 2018 1 commit
  8. 21 Aug, 2018 1 commit
  9. 07 Jun, 2018 1 commit
  10. 05 Jun, 2018 1 commit
    • Ömer Sinan Ağacan's avatar
      Rename some mutable closure types for consistency · 4075656e
      Ömer Sinan Ağacan authored
      Naming is now consistent with other CLEAR/DIRTY objects (MVAR, MUT_VAR,
      (alternatively we could rename MVAR_DIRTY/MVAR_CLEAN etc. to MVAR0/MVAR)
      Removed a few comments in Scav.c about FROZEN0 being on the mut_list
      because it's now clear from the closure type.
      Reviewers: bgamari, simonmar, erikd
      Reviewed By: simonmar
      Subscribers: rwbarton, thomie, carter
      Differential Revision: https://phabricator.haskell.org/D4784
  11. 19 May, 2018 1 commit
  12. 16 May, 2018 1 commit
    • Simon Marlow's avatar
      Merge FUN_STATIC closure with its SRT · 838b6903
      Simon Marlow authored
      The idea here is to save a little code size and some work in the GC,
      by collapsing FUN_STATIC closures and their SRTs.
      This is (4) in a series; see D4632 for more details.
      There's a tradeoff here: more complexity in the compiler in exchange
      for a modest code size reduction (probably around 0.5%).
      * GHC binary itself (statically linked) is 1% smaller
      * -0.2% binary sizes in nofib (-0.5% module sizes)
      Full nofib results comparing D4634 with this: P177 (ignore runtimes,
      these aren't stable on my laptop)
      Test Plan: validate, nofib
      Reviewers: bgamari, niteria, simonpj, erikd
      Subscribers: thomie, carter
      Differential Revision: https://phabricator.haskell.org/D4637
  13. 10 Apr, 2018 1 commit
    • Ben Gamari's avatar
      rts/RetainerProfile: Handle BLOCKING_QUEUES · d5f6d7a0
      Ben Gamari authored
      push() considers BLOCKING_QUEUES to be an invalid closure type which
      should never be present on the stack. However, retainClosure made no
      accomodation for this and ended up pushing such a closure. This lead
      to #14947.
      Test Plan: Validate
      Reviewers: simonmar, erikd
      Reviewed By: simonmar
      Subscribers: thomie, carter, RyanGlScott
      GHC Trac Issues: #14947
      Differential Revision: https://phabricator.haskell.org/D4538
  14. 05 Apr, 2018 1 commit
  15. 25 Mar, 2018 1 commit
  16. 25 Feb, 2018 1 commit
  17. 01 Feb, 2018 1 commit
    • blaze's avatar
      Make RTS keep less memory (fixes #14702) · 0171e09e
      blaze authored
      Currently runtime keeps hold to 4*used_memory. This includes, in
      particular, nursery, which can be quite large on multiprocessor
      machines: 16 CPUs x 64Mb each is 1GB. Multiplying it by 4 means whatever
      actual memory usage is, runtime will never release memory under 4GB, and
      this is quite excessive for processes which only need a lot of memory
      shortly (think building data structures from large files).
      This diff makes multiplier to apply only to GC-managed memory, leaving
      all "static" allocations alone.
      Test Plan: make test TEST="T14702"
      Reviewers: bgamari, erikd, simonmar
      Reviewed By: simonmar
      Subscribers: rwbarton, thomie, carter
      GHC Trac Issues: #14702
      Differential Revision: https://phabricator.haskell.org/D4338
  18. 19 Sep, 2017 1 commit
    • Ben Gamari's avatar
      rts/RetainerProfile: Adding missing closure types to isRetainer · 6252292d
      Ben Gamari authored
      orzo in `#ghc` reported seeing a crash due to the retainer profiler encountering
      a BLOCKING_QUEUE closure, which isRetainer didn't know about. I performed an
      audit to make sure that all of the valid closure types were listed; they
      weren't. This is my guess of how they should appear.
      Test Plan: Validate
      Reviewers: simonmar, austin, erikd
      Reviewed By: simonmar
      Subscribers: rwbarton, thomie
      GHC Trac Issues: #14235
      Differential Revision: https://phabricator.haskell.org/D3967
  19. 01 Aug, 2017 1 commit
    • Ben Gamari's avatar
      rts: Fix "variable set but not used" warning · 74c70166
      Ben Gamari authored
      gcc complains about this while building with Hadrian,
      rts/RetainerProfile.c: In function ‘computeRetainerSet’:
      rts/RetainerProfile.c:1758:18: error:
           error: variable ‘rtl’ set but not used
               RetainerSet *rtl;
      1758 |     RetainerSet *rtl;
           |                  ^
      Reviewers: austin, erikd, simonmar, Phyx
      Reviewed By: Phyx
      Subscribers: Phyx, rwbarton, thomie
      Differential Revision: https://phabricator.haskell.org/D3801
  20. 29 Jun, 2017 1 commit
  21. 29 Apr, 2017 1 commit
  22. 15 Feb, 2017 1 commit
  23. 29 Nov, 2016 1 commit
  24. 14 Nov, 2016 1 commit
    • Simon Marlow's avatar
      Remove CONSTR_STATIC · 55d535da
      Simon Marlow authored
      We currently have two info tables for a constructor
      * XXX_con_info: the info table for a heap-resident instance of the
        constructor, It has type CONSTR, or one of the specialised types like
      * XXX_static_info: the info table for a static instance of this
        constructor, which has type CONSTR_STATIC or CONSTR_STATIC_NOCAF.
      I'm getting rid of the latter, and using the `con_info` info table for
      both static and dynamic constructors.  For rationale and more details
      see Note [static constructors] in SMRep.hs.
      I also removed these macros: `isSTATIC()`, `ip_STATIC()`,
      `closure_STATIC()`, since they relied on the CONSTR/CONSTR_STATIC
      distinction, and anyway HEAP_ALLOCED() does the same job.
      Test Plan: validate
      Reviewers: bgamari, simonpj, austin, gcampax, hvr, niteria, erikd
      Subscribers: thomie
      Differential Revision: https://phabricator.haskell.org/D2690
      GHC Trac Issues: #12455
  25. 20 Jul, 2016 1 commit
    • gcampax's avatar
      Compact Regions · cf989ffe
      gcampax authored
      This brings in initial support for compact regions, as described in the
      ICFP 2015 paper "Efficient Communication and Collection with Compact
      Normal Forms" (Edward Z. Yang et.al.) and implemented by Giovanni
      Some things may change before the 8.2 release, but I (Simon M.) wanted
      to get the main patch committed so that we can iterate.
      What documentation there is is in the Data.Compact module in the new
      compact package.  We'll need to extend and polish the documentation
      before the release.
      Test Plan:
      (new test cases included)
      Reviewers: ezyang, simonmar, hvr, bgamari, austin
      Subscribers: vikraman, Yuras, RyanGlScott, qnikst, mboes, facundominguez, rrnewton, thomie, erikd
      Differential Revision: https://phabricator.haskell.org/D1264
      GHC Trac Issues: #11493
  26. 17 May, 2016 1 commit
    • Erik de Castro Lopo's avatar
      rts: More const correct-ness fixes · 33c029dd
      Erik de Castro Lopo authored
      In addition to more const-correctness fixes this patch fixes an
      infelicity of the previous const-correctness patch (995cf0f3) which
      left `UNTAG_CLOSURE` taking a `const StgClosure` pointer parameter
      but returning a non-const pointer. Here we restore the original type
      signature of `UNTAG_CLOSURE` and add a new function
      `UNTAG_CONST_CLOSURE` which takes and returns a const `StgClosure`
      pointer and uses that wherever possible.
      Test Plan: Validate on Linux, OS X and Windows
      Reviewers: Phyx, hsyl20, bgamari, austin, simonmar, trofi
      Reviewed By: simonmar, trofi
      Subscribers: thomie
      Differential Revision: https://phabricator.haskell.org/D2231
  27. 12 May, 2016 1 commit