1. 06 Dec, 2016 1 commit
  2. 29 Nov, 2016 1 commit
  3. 09 Sep, 2016 1 commit
    • bitonic's avatar
      Make start address of `osReserveHeapMemory` tunable via command line -xb · 1b5f9207
      bitonic authored
      Summary:
      We stumbled upon a case where an external library (OpenCL) does not work
      if a specific address (0x200000000) is taken.
      
      It so happens that `osReserveHeapMemory` starts trying to mmap at 0x200000000:
      
      ```
              void *hint = (void*)((W_)8 * (1 << 30) + attempt * BLOCK_SIZE);
              at = osTryReserveHeapMemory(*len, hint);
      ```
      
      This makes it impossible to use Haskell programs compiled with GHC 8
      with C functions that use OpenCL.
      
      See this example ​https://github.com/chpatrick/oclwtf for a repro.
      
      This patch allows the user to work around this kind of behavior outside
      our control by letting the user override the starting address through an
      RTS command line flag.
      
      Reviewers: bgamari, Phyx, simonmar, erikd, austin
      
      Reviewed By: Phyx, simonmar
      
      Subscribers: rwbarton, thomie
      
      Differential Revision: https://phabricator.haskell.org/D2513
      1b5f9207
  4. 10 Jun, 2016 1 commit
    • Simon Marlow's avatar
      NUMA support · 9e5ea67e
      Simon Marlow authored
      Summary:
      The aim here is to reduce the number of remote memory accesses on
      systems with a NUMA memory architecture, typically multi-socket servers.
      
      Linux provides a NUMA API for doing two things:
      * Allocating memory local to a particular node
      * Binding a thread to a particular node
      
      When given the +RTS --numa flag, the runtime will
      * Determine the number of NUMA nodes (N) by querying the OS
      * Assign capabilities to nodes, so cap C is on node C%N
      * Bind worker threads on a capability to the correct node
      * Keep a separate free lists in the block layer for each node
      * Allocate the nursery for a capability from node-local memory
      * Allocate blocks in the GC from node-local memory
      
      For example, using nofib/parallel/queens on a 24-core 2-socket machine:
      
      ```
      $ ./Main 15 +RTS -N24 -s -A64m
        Total   time  173.960s  (  7.467s elapsed)
      
      $ ./Main 15 +RTS -N24 -s -A64m --numa
        Total   time  150.836s  (  6.423s elapsed)
      ```
      
      The biggest win here is expected to be allocating from node-local
      memory, so that means programs using a large -A value (as here).
      
      According to perf, on this program the number of remote memory accesses
      were reduced by more than 50% by using `--numa`.
      
      Test Plan:
      * validate
      * There's a new flag --debug-numa=<n> that pretends to do NUMA without
        actually making the OS calls, which is useful for testing the code
        on non-NUMA systems.
      * TODO: I need to add some unit tests
      
      Reviewers: erikd, austin, rwbarton, ezyang, bgamari, hvr, niteria
      
      Subscribers: thomie
      
      Differential Revision: https://phabricator.haskell.org/D2199
      9e5ea67e
  5. 24 May, 2016 1 commit
    • Erik de Castro Lopo's avatar
      Runtime linker: Break m32 allocator out into its own file · fe8a4e5d
      Erik de Castro Lopo authored
      This makes the code a little more modular and allows the removal of some
      CPP hackery. By providing dummy implementations of of the `m32_*`
      functions (which simply call `errorBelch`) it means that the call sites
      for these functions are syntax checked even when `RTS_LINKER_USE_MMAP`
      is `0`.
      
      Also changes some size parameter types from `unsigned int` to `size_t`.
      
      Test Plan: Validate on Linux, OS X and Windows
      
      Reviewers: Phyx, hsyl20, bgamari, simonmar, austin
      
      Reviewed By: simonmar, austin
      
      Subscribers: thomie
      
      Differential Revision: https://phabricator.haskell.org/D2237
      fe8a4e5d
  6. 19 May, 2016 1 commit
  7. 04 May, 2016 1 commit
  8. 01 Nov, 2015 1 commit
    • Ben Gamari's avatar
      rts/posix: Reduce heap allocation amount on mmap failure · 4ad2a8f9
      Ben Gamari authored
      Since the two-step allocator the RTS asks the kernel for a large upfront
      mmap'd region of memory (on the order of terabytes). While we have no
      expectation that this entire region will be backed by physical memory,
      this scheme nevertheless fails on some systems with resource limits.
      Here we use a back-off scheme to reduce our allocation request until we
      find a size agreeable to the kernel. Fixes #10877.
      
      This also fixes a latent bug wherein the heap reservation retry logic
      would fail to free the previously reserved address space, which would
      likely result in a heap allocation failure.
      
      Test Plan:
      set address space limit with `ulimit -v 67108864` and try running
      a compiled program
      
      Reviewers: simonmar, austin
      
      Reviewed By: simonmar
      
      Subscribers: thomie, RyanGlScott
      
      Differential Revision: https://phabricator.haskell.org/D1405
      
      GHC Trac Issues: #10877
      4ad2a8f9
  9. 30 Oct, 2015 1 commit
    • Ben Gamari's avatar
      rts: Make MBLOCK_SPACE_SIZE dynamic · f5974c88
      Ben Gamari authored
      Previously this was introduced in D524 as a compile-time constant.
      Sadly, this isn't flexible enough to allow for environments where
      ulimits restrict the maximum address space size (see, for instance,
      
      Consequently, we are forced to make this dynamic. In principle this
      shouldn't be so terrible as we can place both the beginning and end
      addresses within the same cache line, likely incurring only one or so
      additional instruction in HEAP_ALLOCED.
      
      Test Plan: validate
      
      Reviewers: austin, simonmar
      
      Reviewed By: simonmar
      
      Subscribers: thomie
      
      Differential Revision: https://phabricator.haskell.org/D1353
      
      GHC Trac Issues: #10877
      f5974c88
  10. 22 Jul, 2015 1 commit
    • gcampax's avatar
      Two step allocator for 64-bit systems · 0d1a8d09
      gcampax authored
      Summary:
      The current OS memory allocator conflates the concepts of allocating
      address space and allocating memory, which makes the HEAP_ALLOCED()
      implementation excessively complicated (as the only thing it cares
      about is address space layout) and slow. Instead, what we want
      is to allocate a single insanely large contiguous block of address
      space (to make HEAP_ALLOCED() checks fast), and then commit subportions
      of that in 1MB blocks as we did before.
      This is currently behind a flag, USE_LARGE_ADDRESS_SPACE, that is only enabled for
      certain OSes.
      
      Test Plan: validate
      
      Reviewers: simonmar, ezyang, austin
      
      Subscribers: thomie, carter
      
      Differential Revision: https://phabricator.haskell.org/D524
      
      GHC Trac Issues: #9706
      0d1a8d09
  11. 29 Sep, 2014 1 commit
  12. 28 Jul, 2014 1 commit
  13. 25 Oct, 2013 1 commit
  14. 07 Sep, 2012 1 commit
    • Simon Marlow's avatar
      Deprecate lnat, and use StgWord instead · 41737f12
      Simon Marlow authored
      lnat was originally "long unsigned int" but we were using it when we
      wanted a 64-bit type on a 64-bit machine.  This broke on Windows x64,
      where long == int == 32 bits.  Using types of unspecified size is bad,
      but what we really wanted was a type with N bits on an N-bit machine.
      StgWord is exactly that.
      
      lnat was mentioned in some APIs that clients might be using
      (e.g. StackOverflowHook()), so we leave it defined but with a comment
      to say that it's deprecated.
      41737f12
  15. 01 Nov, 2010 1 commit
  16. 13 Aug, 2010 1 commit
  17. 17 Jun, 2010 1 commit
  18. 09 Sep, 2009 1 commit
  19. 05 Aug, 2009 1 commit
  20. 02 Aug, 2009 1 commit
    • Simon Marlow's avatar
      RTS tidyup sweep, first phase · a2a67cd5
      Simon Marlow authored
      The first phase of this tidyup is focussed on the header files, and in
      particular making sure we are exposinng publicly exactly what we need
      to, and no more.
      
       - Rts.h now includes everything that the RTS exposes publicly,
         rather than a random subset of it.
      
       - Most of the public header files have moved into subdirectories, and
         many of them have been renamed.  But clients should not need to
         include any of the other headers directly, just #include the main
         public headers: Rts.h, HsFFI.h, RtsAPI.h.
      
       - All the headers needed for via-C compilation have moved into the
         stg subdirectory, which is self-contained.  Most of the headers for
         the rest of the RTS APIs have moved into the rts subdirectory.
      
       - I left MachDeps.h where it is, because it is so widely used in
         Haskell code.
       
       - I left a deprecated stub for RtsFlags.h in place.  The flag
         structures are now exposed by Rts.h.
      
       - Various internal APIs are no longer exposed by public header files.
      
       - Various bits of dead code and declarations have been removed
      
       - More gcc warnings are turned on, and the RTS code is more
         warning-clean.
      
       - More source files #include "PosixSource.h", and hence only use
         standard POSIX (1003.1c-1995) interfaces.
      
      There is a lot more tidying up still to do, this is just the first
      pass.  I also intend to standardise the names for external RTS APIs
      (e.g use the rts_ prefix consistently), and declare the internal APIs
      as hidden for shared libraries.
      a2a67cd5
  21. 16 Apr, 2008 1 commit
  22. 17 Oct, 2007 1 commit
  23. 24 Oct, 2006 1 commit
    • Simon Marlow's avatar
      Split GC.c, and move storage manager into sm/ directory · ab0e778c
      Simon Marlow authored
      In preparation for parallel GC, split up the monolithic GC.c file into
      smaller parts.  Also in this patch (and difficult to separate,
      unfortunatley):
        
        - Don't include Stable.h in Rts.h, instead just include it where
          necessary.
        
        - consistently use STATIC_INLINE in source files, and INLINE_HEADER
          in header files.  STATIC_INLINE is now turned off when DEBUG is on,
          to make debugging easier.
        
        - The GC no longer takes the get_roots function as an argument.
          We weren't making use of this generalisation.
      ab0e778c
  24. 30 May, 2006 1 commit
    • Simon Marlow's avatar
      replace stgMallocBytesRWX() with our own allocator · e3c55aeb
      Simon Marlow authored
      See bug #738
      
      Allocating executable memory is getting more difficult these days.  In
      particular, the default SELinux policy on Fedora Core 5 disallows
      making the heap (i.e. malloc()'d memory) executable, although it does
      apparently allow mmap()'ing anonymous executable memory by default.
      
      Previously, stgMallocBytesRWX() used malloc() underneath, and then
      tried to make the page holding the memory executable.  This was rather
      hacky and fails with Fedora Core 5.  
      
      This patch adds a mini-allocator for executable memory, based on the
      block allocator.  We grab page-sized blocks and make them executable,
      then allocate small objects from the page.  There's a simple free
      function, that will free whole pages back to the system when they are
      empty.
      e3c55aeb
  25. 07 Apr, 2006 1 commit
    • Simon Marlow's avatar
      Reorganisation of the source tree · 0065d5ab
      Simon Marlow authored
      Most of the other users of the fptools build system have migrated to
      Cabal, and with the move to darcs we can now flatten the source tree
      without losing history, so here goes.
      
      The main change is that the ghc/ subdir is gone, and most of what it
      contained is now at the top level.  The build system now makes no
      pretense at being multi-project, it is just the GHC build system.
      
      No doubt this will break many things, and there will be a period of
      instability while we fix the dependencies.  A straightforward build
      should work, but I haven't yet fixed binary/source distributions.
      Changes to the Building Guide will follow, too.
      0065d5ab
  26. 12 Sep, 2004 1 commit
  27. 21 Jan, 1999 1 commit
  28. 02 Dec, 1998 1 commit