1. 29 Mar, 2010 1 commit
    • Simon Marlow's avatar
      New implementation of BLACKHOLEs · 5d52d9b6
      Simon Marlow authored
      This replaces the global blackhole_queue with a clever scheme that
      enables us to queue up blocked threads on the closure that they are
      blocked on, while still avoiding atomic instructions in the common
      case.
      
      Advantages:
      
       - gets rid of a locked global data structure and some tricky GC code
         (replacing it with some per-thread data structures and different
         tricky GC code :)
      
       - wakeups are more prompt: parallel/concurrent performance should
         benefit.  I haven't seen anything dramatic in the parallel
         benchmarks so far, but a couple of threading benchmarks do improve
         a bit.
      
       - waking up a thread blocked on a blackhole is now O(1) (e.g. if
         it is the target of throwTo).
      
       - less sharing and better separation of Capabilities: communication
         is done with messages, the data structures are strictly owned by a
         Capability and cannot be modified except by sending messages.
      
       - this change will utlimately enable us to do more intelligent
         scheduling when threads block on each other.  This is what started
         off the whole thing, but it isn't done yet (#3838).
      
      I'll be documenting all this on the wiki in due course.
      5d52d9b6
  2. 31 Dec, 2009 1 commit
  3. 02 Dec, 2009 2 commits
  4. 01 Dec, 2009 1 commit
    • Simon Marlow's avatar
      Make allocatePinned use local storage, and other refactorings · 5270423a
      Simon Marlow authored
      This is a batch of refactoring to remove some of the GC's global
      state, as we move towards CPU-local GC.  
      
        - allocateLocal() now allocates large objects into the local
          nursery, rather than taking a global lock and allocating
          then in gen 0 step 0.
      
        - allocatePinned() was still allocating from global storage and
          taking a lock each time, now it uses local storage. 
          (mallocForeignPtrBytes should be faster with -threaded).
          
        - We had a gen 0 step 0, distinct from the nurseries, which are
          stored in a separate nurseries[] array.  This is slightly strange.
          I removed the g0s0 global that pointed to gen 0 step 0, and
          removed all uses of it.  I think now we don't use gen 0 step 0 at
          all, except possibly when there is only one generation.  Possibly
          more tidying up is needed here.
      
        - I removed the global allocate() function, and renamed
          allocateLocal() to allocate().
      
        - the alloc_blocks global is gone.  MAYBE_GC() and
          doYouWantToGC() now check the local nursery only.
      5270423a
  5. 18 Nov, 2009 1 commit
  6. 18 Sep, 2009 1 commit
  7. 20 Aug, 2009 1 commit
  8. 02 Aug, 2009 1 commit
    • Simon Marlow's avatar
      RTS tidyup sweep, first phase · a2a67cd5
      Simon Marlow authored
      The first phase of this tidyup is focussed on the header files, and in
      particular making sure we are exposinng publicly exactly what we need
      to, and no more.
      
       - Rts.h now includes everything that the RTS exposes publicly,
         rather than a random subset of it.
      
       - Most of the public header files have moved into subdirectories, and
         many of them have been renamed.  But clients should not need to
         include any of the other headers directly, just #include the main
         public headers: Rts.h, HsFFI.h, RtsAPI.h.
      
       - All the headers needed for via-C compilation have moved into the
         stg subdirectory, which is self-contained.  Most of the headers for
         the rest of the RTS APIs have moved into the rts subdirectory.
      
       - I left MachDeps.h where it is, because it is so widely used in
         Haskell code.
       
       - I left a deprecated stub for RtsFlags.h in place.  The flag
         structures are now exposed by Rts.h.
      
       - Various internal APIs are no longer exposed by public header files.
      
       - Various bits of dead code and declarations have been removed
      
       - More gcc warnings are turned on, and the RTS code is more
         warning-clean.
      
       - More source files #include "PosixSource.h", and hence only use
         standard POSIX (1003.1c-1995) interfaces.
      
      There is a lot more tidying up still to do, this is just the first
      pass.  I also intend to standardise the names for external RTS APIs
      (e.g use the rts_ prefix consistently), and declare the internal APIs
      as hidden for shared libraries.
      a2a67cd5
  9. 01 Aug, 2009 1 commit
  10. 26 Apr, 2009 1 commit
  11. 13 Mar, 2009 1 commit
    • Simon Marlow's avatar
      Instead of a separate context-switch flag, set HpLim to zero · 304e7fb7
      Simon Marlow authored
      This reduces the latency between a context-switch being triggered and
      the thread returning to the scheduler, which in turn should reduce the
      cost of the GC barrier when there are many cores.
      
      We still retain the old context_switch flag which is checked at the
      end of each block of allocation.  The idea is that setting HpLim may
      fail if the the target thread is modifying HpLim at the same time; the
      context_switch flag is a fallback.  It also allows us to "context
      switch soon" without forcing an immediate switch, which can be costly.
      304e7fb7
  12. 02 Dec, 2008 1 commit
  13. 19 Sep, 2008 1 commit
  14. 02 Jun, 2008 1 commit
  15. 08 Apr, 2008 1 commit
    • Simon Marlow's avatar
      Import libffi-3.0.4, and use it to provide FFI support in GHCi · e0fcf61d
      Simon Marlow authored
      This replaces the hand-rolled architecture-specific FFI support in
      GHCi with the standard libffi as used in GCJ, Python and other
      projects.  I've bundled the complete libffi-3.0.4 tarball in the
      source tree in the same way as we do for GMP, the difference being
      that we always build and install our own libffi regardless of whether
      there's one on the system (it's small, and we don't want
      dependency/versioning headaches).
      
      In particular this means that unregisterised builds will now have a
      fully working GHCi including FFI out of the box, provided libffi
      supports the platform.
      
      There is also code in the RTS to use libffi in place of
      rts/Adjustor.c, but it is currently not enabled if we already have
      support in Adjustor.c for the current platform.  We need to assess the
      performance impact before using libffi here too (in GHCi we don't care
      too much about performance).
      e0fcf61d
  16. 04 Feb, 2008 1 commit
    • Simon Marlow's avatar
      Support for using libffi to implement FFI calls in GHCi (#631) · 937eb1f1
      Simon Marlow authored
      This means that an unregisterised build on a platform not directly
      supported by GHC can now have full FFI support using libffi.
      
      Also in this commit:
      
       - use PrimRep rather than CgRep to describe FFI args in the byte
         code generator.  No functional changes, but PrimRep is more correct.
      
       - change TyCon.sizeofPrimRep to primRepSizeW, which is more useful
      937eb1f1
  17. 27 Nov, 2007 1 commit
  18. 15 Nov, 2007 1 commit
  19. 18 Oct, 2007 1 commit
  20. 10 Oct, 2007 1 commit
    • Simon Marlow's avatar
      GHCi: use non-updatable thunks for breakpoints · 27779403
      Simon Marlow authored
      The extra safe points introduced for breakpoints were previously
      compiled as normal updatable thunks, but they are guaranteed
      single-entry, so we can use non-updatable thunks here.  This restores
      the tail-call property where it was lost in some cases (although stack
      squeezing probably often recovered it), and should improve
      performance.
      27779403
  21. 27 Jul, 2007 1 commit
    • Simon Marlow's avatar
      Pointer Tagging · 6015a94f
      Simon Marlow authored
        
      This patch implements pointer tagging as per our ICFP'07 paper "Faster
      laziness using dynamic pointer tagging".  It improves performance by
      10-15% for most workloads, including GHC itself.
      
      The original patches were by Alexey Rodriguez Yakushev
      <mrchebas@gmail.com>, with additions and improvements by me.  I've
      re-recorded the development as a single patch.
      
      The basic idea is this: we use the low 2 bits of a pointer to a heap
      object (3 bits on a 64-bit architecture) to encode some information
      about the object pointed to.  For a constructor, we encode the "tag"
      of the constructor (e.g. True vs. False), for a function closure its
      arity.  This enables some decisions to be made without dereferencing
      the pointer, which speeds up some common operations.  In particular it
      enables us to avoid costly indirect jumps in many cases.
      
      More information in the commentary:
      
      http://hackage.haskell.org/trac/ghc/wiki/Commentary/Rts/HaskellExecution/PointerTagging
      6015a94f
  22. 15 May, 2007 1 commit
    • Simon Marlow's avatar
      GHCi debugger: new flag -fbreak-on-exception · 17f848e1
      Simon Marlow authored
      When -fbreak-on-exception is set, an exception will cause GHCi to
      suspend the current computation and return to the prompt, where the
      history of the current evaluation can be inspected (if we are in
      :trace).  This isn't on by default, because the behaviour could be
      confusing: for example, ^C will cause a breakpoint.  It can be very
      useful for finding the cause of a "head []" or a "fromJust Nothing",
      though.
      17f848e1
  23. 03 May, 2007 1 commit
  24. 30 Apr, 2007 1 commit
  25. 17 Apr, 2007 1 commit
    • Simon Marlow's avatar
      Re-working of the breakpoint support · cdce6477
      Simon Marlow authored
      This is the result of Bernie Pope's internship work at MSR Cambridge,
      with some subsequent improvements by me.  The main plan was to
      
       (a) Reduce the overhead for breakpoints, so we could enable 
           the feature by default without incurrent a significant penalty
       (b) Scatter more breakpoint sites throughout the code
      
      Currently we can set a breakpoint on almost any subexpression, and the
      overhead is around 1.5x slower than normal GHCi.  I hope to be able to
      get this down further and/or allow breakpoints to be turned off.
      
      This patch also fixes up :print following the recent changes to
      constructor info tables.  (most of the :print tests now pass)
      
      We now support single-stepping, which just enables all breakpoints.
      
        :step <expr>     executes <expr> with single-stepping turned on
        :step            single-steps from the current breakpoint
      
      The mechanism is quite different to the previous implementation.  We
      share code with the HPC (haskell program coverage) implementation now.
      The coverage pass annotates source code with "tick" locations which
      are tracked by the coverage tool.  In GHCi, each "tick" becomes a
      potential breakpoint location.
      
      Previously breakpoints were compiled into code that magically invoked
      a nested instance of GHCi.  Now, a breakpoint causes the current
      thread to block and control is returned to GHCi.
      
      See the wiki page for more details and the current ToDo list:
      
        http://hackage.haskell.org/trac/ghc/wiki/NewGhciDebugger
      cdce6477
  26. 27 Feb, 2007 1 commit
  27. 22 Dec, 2006 1 commit
  28. 21 Dec, 2006 1 commit
  29. 15 Nov, 2006 1 commit
  30. 24 Oct, 2006 1 commit
    • Simon Marlow's avatar
      Split GC.c, and move storage manager into sm/ directory · ab0e778c
      Simon Marlow authored
      In preparation for parallel GC, split up the monolithic GC.c file into
      smaller parts.  Also in this patch (and difficult to separate,
      unfortunatley):
        
        - Don't include Stable.h in Rts.h, instead just include it where
          necessary.
        
        - consistently use STATIC_INLINE in source files, and INLINE_HEADER
          in header files.  STATIC_INLINE is now turned off when DEBUG is on,
          to make debugging easier.
        
        - The GC no longer takes the get_roots function as an argument.
          We weren't making use of this generalisation.
      ab0e778c
  31. 07 Sep, 2006 2 commits
  32. 07 Apr, 2006 1 commit
    • Simon Marlow's avatar
      Reorganisation of the source tree · 0065d5ab
      Simon Marlow authored
      Most of the other users of the fptools build system have migrated to
      Cabal, and with the move to darcs we can now flatten the source tree
      without losing history, so here goes.
      
      The main change is that the ghc/ subdir is gone, and most of what it
      contained is now at the top level.  The build system now makes no
      pretense at being multi-project, it is just the GHC build system.
      
      No doubt this will break many things, and there will be a period of
      instability while we fix the dependencies.  A straightforward build
      should work, but I haven't yet fixed binary/source distributions.
      Changes to the Building Guide will follow, too.
      0065d5ab
  33. 08 Feb, 2006 1 commit
    • Simon Marlow's avatar
      make the smp way RTS-only, normal libraries now work with -smp · beb5737b
      Simon Marlow authored
      We had to bite the bullet here and add an extra word to every thunk,
      to enable running ordinary libraries on SMP.  Otherwise, we would have
      needed to ship an extra set of libraries with GHC 6.6 in addition to
      the two sets we already ship (normal + profiled), and all Cabal
      packages would have to be compiled for SMP too.  We decided it best
      just to take the hit now, making SMP easily accessible to everyone in
      GHC 6.6.
      
      Incedentally, although this increases allocation by around 12% on
      average, the performance hit is around 5%, and much less if your inner
      loop doesn't use any laziness.
      beb5737b
  34. 18 Nov, 2005 1 commit
    • simonmar's avatar
      [project @ 2005-11-18 15:24:12 by simonmar] · c5cd2343
      simonmar authored
      Two improvements to the SMP runtime:
      
        - support for 'par', aka sparks.  Load balancing is very primitive
          right now, but I have seen programs that go faster using par.
      
        - support for backing off when a thread is found to be duplicating
          a computation currently underway in another thread.  This also
          fixes some instability in SMP, because it turned out that when
          an update frame points to an indirection, which can happen if
          a thunk is under evaluation in multiple threads, then after GC
          has shorted out the indirection the update will trash the value.
          Now we suspend the duplicate computation to the heap before this
          can happen.
      
      Additionally:
      
        - stack squeezing is separate from lazy blackholing, and now only
          happens if there's a reasonable amount of squeezing to be done
          in relation to the number of words of stack that have to be moved.
          This means we won't try to shift 10Mb of stack just to save 2
          words at the bottom (it probably never happened, but still).
      
        - update frames are now marked when they have been visited by lazy
          blackholing, as per the SMP paper.
      
        - cleaned up raiseAsync() a bit.
      c5cd2343
  35. 26 Oct, 2005 1 commit
    • simonmar's avatar
      [project @ 2005-10-26 10:42:54 by simonmar] · 7c82b4a8
      simonmar authored
      - change the type of StgRun(): now we return the Capability that the
        thread currently holds.  The return status of the thread is now
        stored in cap->r.rRet (a new slot in the reg table).
      
        This was necessary because on return from StgRun(), the current
        TSO may be blocked, so it no longer belongs to us.  If it is a bound
        thread, then the Task may have been already woken up on another
        Capability, so the scheduler can't use task->cap to find the
        capability it currently owns.
      
      - when shutting down, allow a bound thread to remove its TSO from
        the run queue when exiting (eliminates an error condition in
        releaseCapability()).
      7c82b4a8
  36. 21 Oct, 2005 1 commit
    • simonmar's avatar
      [project @ 2005-10-21 14:02:17 by simonmar] · 03a9ff01
      simonmar authored
      Big re-hash of the threaded/SMP runtime
      
      This is a significant reworking of the threaded and SMP parts of
      the runtime.  There are two overall goals here:
      
        - To push down the scheduler lock, reducing contention and allowing
          more parts of the system to run without locks.  In particular,
          the scheduler does not require a lock any more in the common case.
      
        - To improve affinity, so that running Haskell threads stick to the
          same OS threads as much as possible.
      
      At this point we have the basic structure working, but there are some
      pieces missing.  I believe it's reasonably stable - the important
      parts of the testsuite pass in all the (normal,threaded,SMP) ways.
      
      In more detail:
      
        - Each capability now has a run queue, instead of one global run
          queue.  The Capability and Task APIs have been completely
          rewritten; see Capability.h and Task.h for the details.
      
        - Each capability has its own pool of worker Tasks.  Hence, Haskell
          threads on a Capability's run queue will run on the same worker
          Task(s).  As long as the OS is doing something reasonable, this
          should mean they usually stick to the same CPU.  Another way to
          look at this is that we're assuming each Capability is associated
          with a fixed CPU.
      
        - What used to be StgMainThread is now part of the Task structure.
          Every OS thread in the runtime has an associated Task, and it
          can ask for its current Task at any time with myTask().
      
        - removed RTS_SUPPORTS_THREADS symbol, use THREADED_RTS instead
          (it is now defined for SMP too).
      
        - The RtsAPI has had to change; we must explicitly pass a Capability
          around now.  The previous interface assumed some global state.
          SchedAPI has also changed a lot.
      
        - The OSThreads API now supports thread-local storage, used to
          implement myTask(), although it could be done more efficiently
          using gcc's __thread extension when available.
      
        - I've moved some POSIX-specific stuff into the posix subdirectory,
          moving in the direction of separating out platform-specific
          implementations.
      
        - lots of lock-debugging and assertions in the runtime.  In particular,
          when DEBUG is on, we catch multiple ACQUIRE_LOCK()s, and there is
          also an ASSERT_LOCK_HELD() call.
      
      What's missing so far:
      
        - I have almost certainly broken the Win32 build, will fix soon.
      
        - any kind of thread migration or load balancing.  This is high up
          the agenda, though.
      
        - various performance tweaks to do
      
        - throwTo and forkProcess still do not work in SMP mode
      03a9ff01
  37. 12 Jul, 2005 2 commits