1. 15 Dec, 2010 1 commit
    • Simon Marlow's avatar
      Implement stack chunks and separate TSO/STACK objects · f30d5273
      Simon Marlow authored
      This patch makes two changes to the way stacks are managed:
      
      1. The stack is now stored in a separate object from the TSO.
      
      This means that it is easier to replace the stack object for a thread
      when the stack overflows or underflows; we don't have to leave behind
      the old TSO as an indirection any more.  Consequently, we can remove
      ThreadRelocated and deRefTSO(), which were a pain.
      
      This is obviously the right thing, but the last time I tried to do it
      it made performance worse.  This time I seem to have cracked it.
      
      2. Stacks are now represented as a chain of chunks, rather than
         a single monolithic object.
      
      The big advantage here is that individual chunks are marked clean or
      dirty according to whether they contain pointers to the young
      generation, and the GC can avoid traversing clean stack chunks during
      a young-generation collection.  This means that programs with deep
      stacks will see a big saving in GC overhead when using the default GC
      settings.
      
      A secondary advantage is that there is much less copying involved as
      the stack grows.  Programs that quickly grow a deep stack will see big
      improvements.
      
      In some ways the implementation is simpler, as nothing special needs
      to be done to reclaim stack as the stack shrinks (the GC just recovers
      the dead stack chunks).  On the other hand, we have to manage stack
      underflow between chunks, so there's a new stack frame
      (UNDERFLOW_FRAME), and we now have separate TSO and STACK objects.
      The total amount of code is probably about the same as before.
      
      There are new RTS flags:
      
         -ki<size> Sets the initial thread stack size (default 1k)  Egs: -ki4k -ki2m
         -kc<size> Sets the stack chunk size (default 32k)
         -kb<size> Sets the stack chunk buffer size (default 1k)
      
      -ki was previously called just -k, and the old name is still accepted
      for backwards compatibility.  These new options are documented.
      f30d5273
  2. 28 Oct, 2010 1 commit
  3. 05 Aug, 2010 1 commit
    • Ian Lynagh's avatar
      Make -rtsopts more flexible · 32073806
      Ian Lynagh authored
      The default is a new "some" state, which allows only known-safe flags
      that we want on by default. Currently this is only "--info".
      32073806
  4. 22 Jun, 2010 1 commit
    • dmp@rice.edu's avatar
      Add support for collecting PAPI native events · d4942f78
      dmp@rice.edu authored
      This patch extends the PAPI support in the RTS to allow collection of native
      events. PAPI can collect data for native events that are exposed by the
      hardware beyond the PAPI present events. The native events supported on your
      hardware can found by using the papi_native_avail tool.
      
      The RTS already allows users to specify PAPI preset events from the command
      line. This patch extends that support to allow users to specify native events.
      The changes needed are:
      
      1) New option (#) for the RTS PAPI flag for native events. For example, to
         collect the native event 0x40000000, use ./a.out +RTS -a#0x40000000 -sstderr
      
      2) Update the PAPI_FLAGS struct to store whether the user specified event is a
         papi preset or a native event
      
      3) Update init_countable_events function to add the native events after parsing
         the event code and decoding the name using PAPI_event_code_to_name
      d4942f78
  5. 26 Apr, 2010 1 commit
  6. 15 Mar, 2010 1 commit
  7. 13 Mar, 2010 1 commit
    • Ian Lynagh's avatar
      Add a link-time flag to en/disable the RTS options · 929d1669
      Ian Lynagh authored
      If RTS options are disabled then:
      * The ghc_rts_opts C code variable is processed as normal
      * The GHCRTS environment variable is ignored and, if it is defined, a
        warning is emitted
      * The +RTS flag gives an error and terminates the program
      929d1669
  8. 26 Feb, 2010 1 commit
    • Simon Marlow's avatar
      Tweak the tracing flags slightly, and clean up error handling and diagnostics · cd6bd66f
      Simon Marlow authored
      Tracing flags are now:
      
         -l[flags]  Log events in binary format to the file <program>.eventlog
         -v[flags]  Log events to stderr
                    where [flags] can contain:
                       s    scheduler events
                       t    add time stamps (only useful with -v)
      
      and there are more helpful error messages when using flags that are
      only available in particular variants of the RTS:
      
       the flag -v requires the program to be built with -debug
       the flag -Ds requires the program to be built with -debug
       the flag -p requires the program to be built with -prof
       the flag -N requires the program to be built with -threaded
       the flag -v requires the program to be built with -debug
      
      Also, I fixed the crash reported in #3874, with +RTS -ls -v.
      cd6bd66f
  9. 02 Dec, 2009 1 commit
  10. 01 Dec, 2009 1 commit
  11. 30 Nov, 2009 1 commit
    • Simon Marlow's avatar
      Implement a new heap-tuning option: -H · 32395093
      Simon Marlow authored
      -H alone causes the RTS to use a larger nursery, but without exceeding
      the amount of memory that the application is already using.  It trades
      off GC time against locality: the default setting is to use a
      fixed-size 512k nursery, but this is sometimes worse than using a very
      large nursery despite the worse locality.
      
      Not all programs get faster, but some programs that use large heaps do
      much better with -H.  e.g. this helps a lot with #3061 (binary-trees),
      though not as much as specifying -H<large>.  Typically using -H<large>
      is better than plain -H, because the runtime doesn't know ahead of
      time how much memory you want to use.
      
      Should -H be on by default?  I'm not sure, it makes some programs go
      slower, but others go faster.
      32395093
  12. 19 Nov, 2009 1 commit
    • Simon Marlow's avatar
      Check upper/lower bounds on various RTS flags (#3633) · a6f61eee
      Simon Marlow authored
      Also, make K mean 1024 rather than 1000, in RTS flags (similarly for M
      and G).  The main reason I want to change it is that otherwise this
      might be confusing:
        
      exp3_8: error in RTS option -H4k: size outside allowed range (4096 - 18446744073709551615)
      
      And I think the original reason for using 1000 instead of 1024,
      worries about direct-mapped caches, is not an issue in this context
      (even if you can find a direct-mapped cache these days).
      a6f61eee
  13. 25 Sep, 2009 1 commit
    • Simon Marlow's avatar
      Add a way to generate tracing events programmatically · 5407ad8e
      Simon Marlow authored
      added:
      
       primop  TraceEventOp "traceEvent#" GenPrimOp
         Addr# -> State# s -> State# s
         { Emits an event via the RTS tracing framework.  The contents
           of the event is the zero-terminated byte string passed as the first
           argument.  The event will be emitted either to the .eventlog file,
           or to stderr, depending on the runtime RTS flags. }
      
      and added the required RTS functionality to support it.  Also a bit of
      refactoring in the RTS tracing code.
      5407ad8e
  14. 15 Sep, 2009 1 commit
    • Simon Marlow's avatar
      Improve the default parallel GC settings, and sanitise the flags (#3340) · 53628e91
      Simon Marlow authored
      Flags (from +RTS -?):
      
        -qg[<n>]  Use parallel GC only for generations >= <n>
                  (default: 0, -qg alone turns off parallel GC)
        -qb[<n>]  Use load-balancing in the parallel GC only for generations >= <n>
                  (default: 1, -qb alone turns off load-balancing)
      
      these are good defaults for most parallel programs.  Single-threaded
      programs that want to make use of parallel GC will probably want +RTS
      -qg1 (this is documented).
      
      I've also updated the docs.
      53628e91
  15. 29 Aug, 2009 1 commit
    • Simon Marlow's avatar
      Unify event logging and debug tracing. · a5288c55
      Simon Marlow authored
        - tracing facilities are now enabled with -DTRACING, and -DDEBUG
          additionally enables debug-tracing.  -DEVENTLOG has been
          removed.
      
        - -debug now implies -eventlog
      
        - events can be printed to stderr instead of being sent to the
          binary .eventlog file by adding +RTS -v (which is implied by the
          +RTS -Dx options).
      
        - -Dx debug messages can be sent to the binary .eventlog file
          by adding +RTS -l.  This should help debugging by reducing
          the impact of debug tracing on execution time.
      
        - Various debug messages that duplicated the information in events
          have been removed.
      a5288c55
  16. 02 Aug, 2009 1 commit
    • Simon Marlow's avatar
      RTS tidyup sweep, first phase · a2a67cd5
      Simon Marlow authored
      The first phase of this tidyup is focussed on the header files, and in
      particular making sure we are exposinng publicly exactly what we need
      to, and no more.
      
       - Rts.h now includes everything that the RTS exposes publicly,
         rather than a random subset of it.
      
       - Most of the public header files have moved into subdirectories, and
         many of them have been renamed.  But clients should not need to
         include any of the other headers directly, just #include the main
         public headers: Rts.h, HsFFI.h, RtsAPI.h.
      
       - All the headers needed for via-C compilation have moved into the
         stg subdirectory, which is self-contained.  Most of the headers for
         the rest of the RTS APIs have moved into the rts subdirectory.
      
       - I left MachDeps.h where it is, because it is so widely used in
         Haskell code.
       
       - I left a deprecated stub for RtsFlags.h in place.  The flag
         structures are now exposed by Rts.h.
      
       - Various internal APIs are no longer exposed by public header files.
      
       - Various bits of dead code and declarations have been removed
      
       - More gcc warnings are turned on, and the RTS code is more
         warning-clean.
      
       - More source files #include "PosixSource.h", and hence only use
         standard POSIX (1003.1c-1995) interfaces.
      
      There is a lot more tidying up still to do, this is just the first
      pass.  I also intend to standardise the names for external RTS APIs
      (e.g use the rts_ prefix consistently), and declare the internal APIs
      as hidden for shared libraries.
      a2a67cd5
  17. 02 Jun, 2009 1 commit
  18. 08 May, 2009 1 commit
  19. 18 Mar, 2009 2 commits
  20. 17 Mar, 2009 2 commits
    • Simon Marlow's avatar
      Add fast event logging · 8b18faef
      Simon Marlow authored
      Generate binary log files from the RTS containing a log of runtime
      events with timestamps.  The log file can be visualised in various
      ways, for investigating runtime behaviour and debugging performance
      problems.  See for example the forthcoming ThreadScope viewer.
      
      New GHC option:
      
        -eventlog   (link-time option) Enables event logging.
      
        +RTS -l     (runtime option) Generates <prog>.eventlog with
                    the binary event information.
      
      This replaces some of the tracing machinery we already had in the RTS:
      e.g. +RTS -vg  for GC tracing (we should do this using the new event
      logging instead).
      
      Event logging has almost no runtime cost when it isn't enabled, though
      in the future we might add more fine-grained events and this might
      change; hence having a link-time option and compiling a separate
      version of the RTS for event logging.  There's a small runtime cost
      for enabling event-logging, for most programs it shouldn't make much
      difference.
      
      (Todo: docs)
      8b18faef
    • Simon Marlow's avatar
      Add getNumberOfProcessors(), FIX MacOS X build problem (hopefully) · 0ee0be10
      Simon Marlow authored
      Somebody needs to implement getNumberOfProcessors() for MacOS X,
      currently it will return 1.
      0ee0be10
  21. 13 Mar, 2009 2 commits
    • Simon Marlow's avatar
      Use work-stealing for load-balancing in the GC · 4e354226
      Simon Marlow authored
        
      New flag: "+RTS -qb" disables load-balancing in the parallel GC
      (though this is subject to change, I think we will probably want to do
      something more automatic before releasing this).
      
      To get the "PARGC3" configuration described in the "Runtime support
      for Multicore Haskell" paper, use "+RTS -qg0 -qb -RTS".
      
      The main advantage of this is that it allows us to easily disable
      load-balancing altogether, which turns out to be important in parallel
      programs.  Maintaining locality is sometimes more important that
      spreading the work out in parallel GC.  There is a side benefit in
      that the parallel GC should have improved locality even when
      load-balancing, because each processor prefers to take work from its
      own queue before stealing from others.
      4e354226
    • Simon Marlow's avatar
  22. 12 Jan, 2009 1 commit
  23. 23 Nov, 2008 1 commit
  24. 21 Nov, 2008 1 commit
    • Simon Marlow's avatar
      Use mutator threads to do GC, instead of having a separate pool of GC threads · 3ebcd3de
      Simon Marlow authored
      Previously, the GC had its own pool of threads to use as workers when
      doing parallel GC.  There was a "leader", which was the mutator thread
      that initiated the GC, and the other threads were taken from the pool.
      
      This was simple and worked fine for sequential programs, where we did
      most of the benchmarking for the parallel GC, but falls down for
      parallel programs.  When we have N mutator threads and N cores, at GC
      time we would have to stop N-1 mutator threads and start up N-1 GC
      threads, and hope that the OS schedules them all onto separate cores.
      It practice it doesn't, as you might expect.
      
      Now we use the mutator threads to do GC.  This works quite nicely,
      particularly for parallel programs, where each mutator thread scans
      its own spark pool, which is probably in its cache anyway.
      
      There are some flag changes:
      
        -g<n> is removed (-g1 is still accepted for backwards compat).
        There's no way to have a different number of GC threads than mutator
        threads now.
      
        -q1       Use one OS thread for GC (turns off parallel GC)
        -qg<n>    Use parallel GC for generations >= <n> (default: 1)
      
      Using parallel GC only for generations >=1 works well for sequential
      programs.  Compiling an ordinary sequential program with -threaded and
      running it with -N2 or more should help if you do a lot of GC.  I've
      found that adding -qg0 (do parallel GC for generation 0 too) speeds up
      some parallel programs, but slows down some sequential programs.
      Being conservative, I left the threshold at 1.
      
      ToDo: document the new options.
      3ebcd3de
  25. 17 Nov, 2008 1 commit
    • Simon Marlow's avatar
      Attempt to fix #2512 and #2063; add +RTS -xm<address> -RTS option · 11f6f411
      Simon Marlow authored
      On x86_64, the RTS needs to allocate memory in the low 2Gb of the
      address space.  On Linux we can do this with MAP_32BIT, but sometimes
      this doesn't work (#2512) and other OSs don't support it at all
      (#2063).  So to work around this:
      
        - Try MAP_32BIT first, if available.
      
        - Otherwise, try allocating memory from a fixed address (by default
          1Gb)
      
        - We now provide an option to configure the address to allocate
          from.  This allows a workaround on machines where the default
          breaks, and also provides a way for people to test workarounds
          that we can incorporate in future releases.
      11f6f411
  26. 11 Aug, 2008 1 commit
  27. 19 Jun, 2008 1 commit
    • Simon Marlow's avatar
      +RTS -S -RTS now sends output to stderr (also -s) · 34e8e513
      Simon Marlow authored
      Previously +RTS -Sstderr -RTS was required to send output to stderr,
      but this is the most common usage and I got tired of typing "stderr".
      The old default behaviour of sending output to <prog>.stat is now gone
      (I don't think we use it anywhere).  
      
      Temporarily we allowed +RTS -S- -RTS to mean stderr; there were
      objections to this, so it is now also gone.
      34e8e513
  28. 09 Jun, 2008 1 commit
  29. 03 Jun, 2008 2 commits
  30. 16 Apr, 2008 1 commit
  31. 31 Jan, 2008 1 commit
  32. 20 Nov, 2007 1 commit
    • simonmar@microsoft.com's avatar
      improvements to PAPI support · 8db56c86
      simonmar@microsoft.com authored
      - major (multithreaded) GC is measured separately from minor GC
      - events to measure can now be specified on the command line, e.g
           prog +RTS -a+PAPI_TOT_CYC
      8db56c86
  33. 31 Oct, 2007 1 commit
    • Simon Marlow's avatar
      Refactoring of the GC in preparation for parallel GC · d5bd3e82
      Simon Marlow authored
        
      This patch localises the state of the GC into a gc_thread structure,
      and reorganises the inner loop of the GC to scavenge one block at a
      time from global work lists in each "step".  The gc_thread structure
      has a "workspace" for each step, in which it collects evacuated
      objects until it has a full block to push out to the step's global
      list.  Details of the algorithm will be on the wiki in due course.
      
      At the moment, THREADED_RTS does not compile, but the single-threaded
      GC works (and is 10-20% slower than before).
      d5bd3e82
  34. 28 Feb, 2008 1 commit
    • Simon Marlow's avatar
      Enable -prof -threaded (#886) · 50c4d039
      Simon Marlow authored
      It turns out that -prof -threaded works (modulo some small changes),
      because all the data structures used in profiling are only accessed by
      one thread at a time, at long as we don't use +RTS -N2 or higher.  So
      this patch enables the use of -prof -threaded, but an error is given
      if you ask for more than one CPU with +RTS -N.
      50c4d039
  35. 23 Nov, 2007 1 commit
  36. 16 Aug, 2007 1 commit