1. 04 May, 2016 3 commits
    • Erik de Castro Lopo's avatar
      rts: Replace `nat` with `uint32_t` · db9de7eb
      Erik de Castro Lopo authored
      The `nat` type was an alias for `unsigned int` with a comment saying
      it was at least 32 bits. We keep the typedef in case client code is
      using it but mark it as deprecated.
      
      Test Plan: Validated on Linux, OS X and Windows
      
      Reviewers: simonmar, austin, thomie, hvr, bgamari, hsyl20
      
      Differential Revision: https://phabricator.haskell.org/D2166
      db9de7eb
    • Simon Marlow's avatar
      Add +RTS -AL<size> · f703fd6b
      Simon Marlow authored
      +RTS -AL<size> controls the total size of large objects that can be
      allocated before a GC is triggered.  Previously this was always just the
      value of -A, and the limit mainly existed to prevent runaway allocation
      in pathalogical programs that allocate a lot of large objects.  However,
      since the limit is shared between all cores, on a large multicore the
      default becomes more restrictive, and can end up triggering GC well
      before it would normally have been.
      
      Arguably a better default would be A*N, but this is probably excessive.
      Adding a flag lets you choose, and I've left the default as it was.
      
      See docs for usage.
      f703fd6b
    • Simon Marlow's avatar
      Allow limiting the number of GC threads (+RTS -qn<n>) · 76ee2607
      Simon Marlow authored
      This allows the GC to use fewer threads than the number of capabilities.
      At each GC, we choose some of the capabilities to be "idle", which means
      that the thread running on that capability (if any) will sleep for the
      duration of the GC, and the other threads will do its work.  We choose
      capabilities that are already idle (if any) to be the idle capabilities.
      
      The idea is that this helps in the following situation:
      
      * We want to use a large -N value so as to make use of hyperthreaded
        cores
      * We use a large heap size, so GC is infrequent
      * But we don't want to use all -N threads in the GC, because that
        thrashes the memory too much.
      
      See docs for usage.
      76ee2607
  2. 21 Nov, 2015 1 commit
  3. 18 Nov, 2015 1 commit
  4. 25 Nov, 2014 1 commit
  5. 24 Nov, 2014 1 commit
    • Ömer Sinan Ağacan's avatar
      accessors to RTS flag values -- #5364 · 1617a10a
      Ömer Sinan Ağacan authored
      Summary: Implementation of #5364. Mostly boilerplate, reading FILE fields is missing.
      
      Test Plan:
      - Get some feedback on missing parts. (FILE fields)
      - Get some feedback on module name.
      - Get some feedback on other things.
      - Get code reviewed.
      - Make sure test suite is passing. (I haven't run it myself)
      
      Reviewers: hvr, austin, ezyang
      
      Reviewed By: ezyang
      
      Subscribers: ekmett, simonmar, ezyang, carter, thomie
      
      Differential Revision: https://phabricator.haskell.org/D306
      
      GHC Trac Issues: #5364
      
      Conflicts:
      	includes/rts/Flags.h
      1617a10a
  6. 12 Nov, 2014 1 commit
  7. 04 May, 2014 1 commit
  8. 02 May, 2014 1 commit
    • Simon Marlow's avatar
      Per-thread allocation counters and limits · b0534f78
      Simon Marlow authored
      This tracks the amount of memory allocation by each thread in a
      counter stored in the TSO.  Optionally, when the counter drops below
      zero (it counts down), the thread can be sent an asynchronous
      exception: AllocationLimitExceeded.  When this happens, given a small
      additional limit so that it can handle the exception.  See
      documentation in GHC.Conc for more details.
      
      Allocation limits are similar to timeouts, but
      
        - timeouts use real time, not CPU time.  Allocation limits do not
          count anything while the thread is blocked or in foreign code.
      
        - timeouts don't re-trigger if the thread catches the exception,
          allocation limits do.
      
        - timeouts can catch non-allocating loops, if you use
          -fno-omit-yields.  This doesn't work for allocation limits.
      
      I couldn't measure any impact on benchmarks with these changes, even
      for nofib/smp.
      b0534f78
  9. 01 Oct, 2013 1 commit
  10. 24 Sep, 2012 1 commit
    • Simon Marlow's avatar
      Another overhaul of the recent_activity / idle GC handling (#5991) · 0b79d5cd
      Simon Marlow authored
      Improvements:
      
       - we now turn off the timer signal in the non-threaded RTS after
         idleGCDelay.  This should make the xmonad users on #5991 happy.
      
       - we now turn off the timer signal after idleGCDelay even if the
         idle GC is disabled with +RTS -I0.
      
       - we now do *not* turn off the timer when profiling.
      
       - more comments to explain the meaning of the various ACTIVITY_*
         values
      0b79d5cd
  11. 27 Feb, 2012 1 commit
  12. 13 Dec, 2011 1 commit
    • Simon Marlow's avatar
      New flag +RTS -qi<n>, avoid waking up idle Capabilities to do parallel GC · a02eb298
      Simon Marlow authored
      This is an experimental tweak to the parallel GC that avoids waking up
      a Capability to do parallel GC if we know that the capability has been
      idle for a (tunable) number of GC cycles.  The idea is that if you're
      only using a few Capabilities, there's no point waking up the ones
      that aren't busy.
      
      e.g. +RTS -qi3
      
      says "A Capability will participate in parallel GC if it was running
      at all since the last 3 GC cycles."
      
      Results are a bit hit and miss, and I don't completely understand why
      yet.  Hence, for now it is turned off by default, and also not
      documented except in the +RTS -? output.
      a02eb298
  13. 25 Nov, 2011 1 commit
    • Simon Marlow's avatar
      Time handling overhaul · 6b109851
      Simon Marlow authored
      Terminology cleanup: the type "Ticks" has been renamed "Time", which
      is an StgWord64 in units of TIME_RESOLUTION (currently nanoseconds).
      The terminology "tick" is now used consistently to mean the interval
      between timer signals.
      
      The ticker now always ticks in realtime (actually CLOCK_MONOTONIC if
      we have it).  Before it used CPU time in the non-threaded RTS and
      realtime in the threaded RTS, but I've discovered that the CPU timer
      has terrible resolution (at least on Linux) and isn't much use for
      profiling.  So now we always use realtime.  This should also fix
      
      The default tick interval is now 10ms, except when profiling where we
      drop it to 1ms.  This gives more accurate profiles without affecting
      runtime too much (<1%).
      
      Lots of cleanups - the resolution of Time is now in one place
      only (Rts.h) rather than having calculations that depend on the
      resolution scattered all over the RTS.  I hope I found them all.
      6b109851
  14. 27 Oct, 2011 1 commit
  15. 18 Jul, 2011 3 commits
    • Duncan Coutts's avatar
      Add new fully-accurate per-spark trace/eventlog events · 084b64f2
      Duncan Coutts authored
      Replaces the existing EVENT_RUN/STEAL_SPARK events with 7 new events
      covering all stages of the spark lifcycle:
        create, dud, overflow, run, steal, fizzle, gc
      
      The sampled spark events are still available. There are now two event
      classes for sparks, the sampled and the fully accurate. They can be
      enabled/disabled independently. By default +RTS -l includes the sampled
      but not full detail spark events. Use +RTS -lf-p to enable the detailed
      'f' and disable the sampled 'p' spark.
      
      Includes work by Mikolaj <mikolaj.konarski@gmail.com>
      084b64f2
    • Duncan Coutts's avatar
      Move GC tracing into a separate trace class · 46b70749
      Duncan Coutts authored
      Previously GC was included in the scheduler trace class. It can be
      enabled specifically with +RTS -vg or -lg, though note that both -v
      and -l on their own now default to a sensible set of trace classes,
      currently: scheduler, gc and sparks.
      46b70749
    • Duncan Coutts's avatar
      add a new trace class for spark events · a5192d48
      Duncan Coutts authored
      a5192d48
  16. 25 May, 2011 1 commit
  17. 12 Apr, 2011 1 commit
    • Simon Marlow's avatar
      Cleanup sweep and fix a bug in RTS flag processing. · 5463b55b
      Simon Marlow authored
      This code has accumulated a great deal of cruft over the years, this
      pass cleans up a lot of the surrounding cruft but leaves the actual
      argument processing alone - so there's still more that could be done.
      
      Bug fixed:
      
        - ghc_rts_opts should not be subject to the --rtsopts setting.  If
          the programmer explicitly declares options with ghc_rts_opts, they
          shouldn't also have to accept command-line RTS options to make them
          work.
      5463b55b
  18. 01 Feb, 2011 1 commit
  19. 15 Dec, 2010 1 commit
    • Simon Marlow's avatar
      Implement stack chunks and separate TSO/STACK objects · f30d5273
      Simon Marlow authored
      This patch makes two changes to the way stacks are managed:
      
      1. The stack is now stored in a separate object from the TSO.
      
      This means that it is easier to replace the stack object for a thread
      when the stack overflows or underflows; we don't have to leave behind
      the old TSO as an indirection any more.  Consequently, we can remove
      ThreadRelocated and deRefTSO(), which were a pain.
      
      This is obviously the right thing, but the last time I tried to do it
      it made performance worse.  This time I seem to have cracked it.
      
      2. Stacks are now represented as a chain of chunks, rather than
         a single monolithic object.
      
      The big advantage here is that individual chunks are marked clean or
      dirty according to whether they contain pointers to the young
      generation, and the GC can avoid traversing clean stack chunks during
      a young-generation collection.  This means that programs with deep
      stacks will see a big saving in GC overhead when using the default GC
      settings.
      
      A secondary advantage is that there is much less copying involved as
      the stack grows.  Programs that quickly grow a deep stack will see big
      improvements.
      
      In some ways the implementation is simpler, as nothing special needs
      to be done to reclaim stack as the stack shrinks (the GC just recovers
      the dead stack chunks).  On the other hand, we have to manage stack
      underflow between chunks, so there's a new stack frame
      (UNDERFLOW_FRAME), and we now have separate TSO and STACK objects.
      The total amount of code is probably about the same as before.
      
      There are new RTS flags:
      
         -ki<size> Sets the initial thread stack size (default 1k)  Egs: -ki4k -ki2m
         -kc<size> Sets the stack chunk size (default 32k)
         -kb<size> Sets the stack chunk buffer size (default 1k)
      
      -ki was previously called just -k, and the old name is still accepted
      for backwards compatibility.  These new options are documented.
      f30d5273
  20. 22 Jun, 2010 1 commit
    • dmp@rice.edu's avatar
      Add support for collecting PAPI native events · d4942f78
      dmp@rice.edu authored
      This patch extends the PAPI support in the RTS to allow collection of native
      events. PAPI can collect data for native events that are exposed by the
      hardware beyond the PAPI present events. The native events supported on your
      hardware can found by using the papi_native_avail tool.
      
      The RTS already allows users to specify PAPI preset events from the command
      line. This patch extends that support to allow users to specify native events.
      The changes needed are:
      
      1) New option (#) for the RTS PAPI flag for native events. For example, to
         collect the native event 0x40000000, use ./a.out +RTS -a#0x40000000 -sstderr
      
      2) Update the PAPI_FLAGS struct to store whether the user specified event is a
         papi preset or a native event
      
      3) Update init_countable_events function to add the native events after parsing
         the event code and decoding the name using PAPI_event_code_to_name
      d4942f78
  21. 30 Nov, 2009 1 commit
    • Simon Marlow's avatar
      Implement a new heap-tuning option: -H · 32395093
      Simon Marlow authored
      -H alone causes the RTS to use a larger nursery, but without exceeding
      the amount of memory that the application is already using.  It trades
      off GC time against locality: the default setting is to use a
      fixed-size 512k nursery, but this is sometimes worse than using a very
      large nursery despite the worse locality.
      
      Not all programs get faster, but some programs that use large heaps do
      much better with -H.  e.g. this helps a lot with #3061 (binary-trees),
      though not as much as specifying -H<large>.  Typically using -H<large>
      is better than plain -H, because the runtime doesn't know ahead of
      time how much memory you want to use.
      
      Should -H be on by default?  I'm not sure, it makes some programs go
      slower, but others go faster.
      32395093
  22. 25 Sep, 2009 1 commit
    • Simon Marlow's avatar
      Add a way to generate tracing events programmatically · 5407ad8e
      Simon Marlow authored
      added:
      
       primop  TraceEventOp "traceEvent#" GenPrimOp
         Addr# -> State# s -> State# s
         { Emits an event via the RTS tracing framework.  The contents
           of the event is the zero-terminated byte string passed as the first
           argument.  The event will be emitted either to the .eventlog file,
           or to stderr, depending on the runtime RTS flags. }
      
      and added the required RTS functionality to support it.  Also a bit of
      refactoring in the RTS tracing code.
      5407ad8e
  23. 15 Sep, 2009 1 commit
    • Simon Marlow's avatar
      Improve the default parallel GC settings, and sanitise the flags (#3340) · 53628e91
      Simon Marlow authored
      Flags (from +RTS -?):
      
        -qg[<n>]  Use parallel GC only for generations >= <n>
                  (default: 0, -qg alone turns off parallel GC)
        -qb[<n>]  Use load-balancing in the parallel GC only for generations >= <n>
                  (default: 1, -qb alone turns off load-balancing)
      
      these are good defaults for most parallel programs.  Single-threaded
      programs that want to make use of parallel GC will probably want +RTS
      -qg1 (this is documented).
      
      I've also updated the docs.
      53628e91
  24. 29 Aug, 2009 1 commit
    • Simon Marlow's avatar
      Unify event logging and debug tracing. · a5288c55
      Simon Marlow authored
        - tracing facilities are now enabled with -DTRACING, and -DDEBUG
          additionally enables debug-tracing.  -DEVENTLOG has been
          removed.
      
        - -debug now implies -eventlog
      
        - events can be printed to stderr instead of being sent to the
          binary .eventlog file by adding +RTS -v (which is implied by the
          +RTS -Dx options).
      
        - -Dx debug messages can be sent to the binary .eventlog file
          by adding +RTS -l.  This should help debugging by reducing
          the impact of debug tracing on execution time.
      
        - Various debug messages that duplicated the information in events
          have been removed.
      a5288c55
  25. 05 Aug, 2009 1 commit
  26. 25 Aug, 2009 1 commit
  27. 02 Aug, 2009 1 commit
    • Simon Marlow's avatar
      RTS tidyup sweep, first phase · a2a67cd5
      Simon Marlow authored
      The first phase of this tidyup is focussed on the header files, and in
      particular making sure we are exposinng publicly exactly what we need
      to, and no more.
      
       - Rts.h now includes everything that the RTS exposes publicly,
         rather than a random subset of it.
      
       - Most of the public header files have moved into subdirectories, and
         many of them have been renamed.  But clients should not need to
         include any of the other headers directly, just #include the main
         public headers: Rts.h, HsFFI.h, RtsAPI.h.
      
       - All the headers needed for via-C compilation have moved into the
         stg subdirectory, which is self-contained.  Most of the headers for
         the rest of the RTS APIs have moved into the rts subdirectory.
      
       - I left MachDeps.h where it is, because it is so widely used in
         Haskell code.
       
       - I left a deprecated stub for RtsFlags.h in place.  The flag
         structures are now exposed by Rts.h.
      
       - Various internal APIs are no longer exposed by public header files.
      
       - Various bits of dead code and declarations have been removed
      
       - More gcc warnings are turned on, and the RTS code is more
         warning-clean.
      
       - More source files #include "PosixSource.h", and hence only use
         standard POSIX (1003.1c-1995) interfaces.
      
      There is a lot more tidying up still to do, this is just the first
      pass.  I also intend to standardise the names for external RTS APIs
      (e.g use the rts_ prefix consistently), and declare the internal APIs
      as hidden for shared libraries.
      a2a67cd5
  28. 02 Jun, 2009 1 commit
  29. 18 Mar, 2009 1 commit
  30. 17 Mar, 2009 1 commit
    • Simon Marlow's avatar
      Add fast event logging · 8b18faef
      Simon Marlow authored
      Generate binary log files from the RTS containing a log of runtime
      events with timestamps.  The log file can be visualised in various
      ways, for investigating runtime behaviour and debugging performance
      problems.  See for example the forthcoming ThreadScope viewer.
      
      New GHC option:
      
        -eventlog   (link-time option) Enables event logging.
      
        +RTS -l     (runtime option) Generates <prog>.eventlog with
                    the binary event information.
      
      This replaces some of the tracing machinery we already had in the RTS:
      e.g. +RTS -vg  for GC tracing (we should do this using the new event
      logging instead).
      
      Event logging has almost no runtime cost when it isn't enabled, though
      in the future we might add more fine-grained events and this might
      change; hence having a link-time option and compiling a separate
      version of the RTS for event logging.  There's a small runtime cost
      for enabling event-logging, for most programs it shouldn't make much
      difference.
      
      (Todo: docs)
      8b18faef
  31. 13 Mar, 2009 1 commit
    • Simon Marlow's avatar
      Use work-stealing for load-balancing in the GC · 4e354226
      Simon Marlow authored
        
      New flag: "+RTS -qb" disables load-balancing in the parallel GC
      (though this is subject to change, I think we will probably want to do
      something more automatic before releasing this).
      
      To get the "PARGC3" configuration described in the "Runtime support
      for Multicore Haskell" paper, use "+RTS -qg0 -qb -RTS".
      
      The main advantage of this is that it allows us to easily disable
      load-balancing altogether, which turns out to be important in parallel
      programs.  Maintaining locality is sometimes more important that
      spreading the work out in parallel GC.  There is a side benefit in
      that the parallel GC should have improved locality even when
      load-balancing, because each processor prefers to take work from its
      own queue before stealing from others.
      4e354226
  32. 23 Nov, 2008 1 commit
  33. 21 Nov, 2008 1 commit
    • Simon Marlow's avatar
      Use mutator threads to do GC, instead of having a separate pool of GC threads · 3ebcd3de
      Simon Marlow authored
      Previously, the GC had its own pool of threads to use as workers when
      doing parallel GC.  There was a "leader", which was the mutator thread
      that initiated the GC, and the other threads were taken from the pool.
      
      This was simple and worked fine for sequential programs, where we did
      most of the benchmarking for the parallel GC, but falls down for
      parallel programs.  When we have N mutator threads and N cores, at GC
      time we would have to stop N-1 mutator threads and start up N-1 GC
      threads, and hope that the OS schedules them all onto separate cores.
      It practice it doesn't, as you might expect.
      
      Now we use the mutator threads to do GC.  This works quite nicely,
      particularly for parallel programs, where each mutator thread scans
      its own spark pool, which is probably in its cache anyway.
      
      There are some flag changes:
      
        -g<n> is removed (-g1 is still accepted for backwards compat).
        There's no way to have a different number of GC threads than mutator
        threads now.
      
        -q1       Use one OS thread for GC (turns off parallel GC)
        -qg<n>    Use parallel GC for generations >= <n> (default: 1)
      
      Using parallel GC only for generations >=1 works well for sequential
      programs.  Compiling an ordinary sequential program with -threaded and
      running it with -N2 or more should help if you do a lot of GC.  I've
      found that adding -qg0 (do parallel GC for generation 0 too) speeds up
      some parallel programs, but slows down some sequential programs.
      Being conservative, I left the threshold at 1.
      
      ToDo: document the new options.
      3ebcd3de
  34. 17 Nov, 2008 1 commit
    • Simon Marlow's avatar
      Attempt to fix #2512 and #2063; add +RTS -xm<address> -RTS option · 11f6f411
      Simon Marlow authored
      On x86_64, the RTS needs to allocate memory in the low 2Gb of the
      address space.  On Linux we can do this with MAP_32BIT, but sometimes
      this doesn't work (#2512) and other OSs don't support it at all
      (#2063).  So to work around this:
      
        - Try MAP_32BIT first, if available.
      
        - Otherwise, try allocating memory from a fixed address (by default
          1Gb)
      
        - We now provide an option to configure the address to allocate
          from.  This allows a workaround on machines where the default
          breaks, and also provides a way for people to test workarounds
          that we can incorporate in future releases.
      11f6f411
  35. 09 Jun, 2008 1 commit
  36. 16 Apr, 2008 1 commit