GitLab

Replaces the existing EVENT_RUN/STEAL_SPARK events with 7 new events
covering all stages of the spark lifcycle:
  create, dud, overflow, run, steal, fizzle, gc

The sampled spark events are still available. There are now two event
classes for sparks, the sampled and the fully accurate. They can be
enabled/disabled independently. By default +RTS -l includes the sampled
but not full detail spark events. Use +RTS -lf-p to enable the detailed
'f' and disable the sampled 'p' spark.

Includes work by Mikolaj <mikolaj.konarski@gmail.com>

A new eventlog event containing 7 spark counters/statistics: sparks
created, dud, overflowed, converted, GC'd, fizzled and remaining.
These are maintained and logged separately for each capability.
We log them at startup, on each GC (minor and major) and on shutdown.

The process ID, parent process ID, rts name and version
The program arguments and environment.

This reverts commit 58532eb4.
Turns out it didn't work on Windows and it'll need some non-trivial changes
to make it work on Windows. We'll get it in later once that's sorted out.

So we can now get these in ThreadScope:

  19487000: cap 1: stopping thread 6 (blocked on black hole owned by thread 4)

Note: needs an update to ghc-events.  Older ThreadScopes will just
ignore the new information.

- Defines a DTrace provider, called 'HaskellEvent', that provides a probe
  for every event of the eventlog framework.
- In contrast to the original eventlog, the DTrace probes are available in
  all flavours of the runtime system (DTrace probes have virtually no
  overhead if not enabled); when -DTRACING is defined both the regular
  event log as well as DTrace probes can be used.
- Currently, Mac OS X only.  User-space DTrace probes are implemented
  differently on Mac OS X than in the original DTrace implementation.
  Nevertheless, it shouldn't be too hard to enable these probes on other
  platforms, too.
- Documentation is at http://hackage.haskell.org/trac/ghc/wiki/DTrace

The log file format was still using 32 bits, this just updates the
header file to match; there should be no functional changes.

added:

 primop  TraceEventOp "traceEvent#" GenPrimOp
   Addr# -> State# s -> State# s
   { Emits an event via the RTS tracing framework.  The contents
     of the event is the zero-terminated byte string passed as the first
     argument.  The event will be emitted either to the .eventlog file,
     or to stderr, depending on the runtime RTS flags. }

and added the required RTS functionality to support it.  Also a bit of
refactoring in the RTS tracing code.

This makes events smaller and tracing quicker, and speeds up reading
and sorting the trace file.

HEADS UP: this changes the format of event log files.  Corresponding
changes to the ghc-events package are required (and will be pushed
soon).  Normally we would make backwards-compatible changes, but this
changes the format of every event (to remove the capability) so I'm
breaking the rules this time.  This will be the only time we can do
this, since the format becomes public in 6.12.1.

These indicate the size and time span of a sequence of events in the
event log, to make it easier to sort and navigate a large event log.

  - tracing facilities are now enabled with -DTRACING, and -DDEBUG
    additionally enables debug-tracing.  -DEVENTLOG has been
    removed.

  - -debug now implies -eventlog

  - events can be printed to stderr instead of being sent to the
    binary .eventlog file by adding +RTS -v (which is implied by the
    +RTS -Dx options).

  - -Dx debug messages can be sent to the binary .eventlog file
    by adding +RTS -l.  This should help debugging by reducing
    the impact of debug tracing on execution time.

  - Various debug messages that duplicated the information in events
    have been removed.

The first phase of this tidyup is focussed on the header files, and in
particular making sure we are exposinng publicly exactly what we need
to, and no more.

 - Rts.h now includes everything that the RTS exposes publicly,
   rather than a random subset of it.

 - Most of the public header files have moved into subdirectories, and
   many of them have been renamed.  But clients should not need to
   include any of the other headers directly, just #include the main
   public headers: Rts.h, HsFFI.h, RtsAPI.h.

 - All the headers needed for via-C compilation have moved into the
   stg subdirectory, which is self-contained.  Most of the headers for
   the rest of the RTS APIs have moved into the rts subdirectory.

 - I left MachDeps.h where it is, because it is so widely used in
   Haskell code.
 
 - I left a deprecated stub for RtsFlags.h in place.  The flag
   structures are now exposed by Rts.h.

 - Various internal APIs are no longer exposed by public header files.

 - Various bits of dead code and declarations have been removed

 - More gcc warnings are turned on, and the RTS code is more
   warning-clean.

 - More source files #include "PosixSource.h", and hence only use
   standard POSIX (1003.1c-1995) interfaces.

There is a lot more tidying up still to do, this is just the first
pass.  I also intend to standardise the names for external RTS APIs
(e.g use the rts_ prefix consistently), and declare the internal APIs
as hidden for shared libraries.

Also some tidyups and renaming

Fixed ThreadID to be defined as StgThreadID, not StgWord64.  Changed CapabilityNum to CapNo.  Added helper functions postCapNo() and postThreadID().
ThreadID was StgWord64, but should have been StgThreadID, which is
currently StgWord32.  Changed name from CapabilityNum to CapNo to better
reflect naming in Capability struct where "no" is the capability number.
Modified EventLog.c to use the helper functions postCapNo() and
postThreadID () for CapNo and ThreadID.

Generate binary log files from the RTS containing a log of runtime
events with timestamps.  The log file can be visualised in various
ways, for investigating runtime behaviour and debugging performance
problems.  See for example the forthcoming ThreadScope viewer.

New GHC option:

  -eventlog   (link-time option) Enables event logging.

  +RTS -l     (runtime option) Generates <prog>.eventlog with
              the binary event information.

This replaces some of the tracing machinery we already had in the RTS:
e.g. +RTS -vg  for GC tracing (we should do this using the new event
logging instead).

Event logging has almost no runtime cost when it isn't enabled, though
in the future we might add more fine-grained events and this might
change; hence having a link-time option and compiling a separate
version of the RTS for event logging.  There's a small runtime cost
for enabling event-logging, for most programs it shouldn't make much
difference.

(Todo: docs)