GitLab

When requesting more than BLOCKS_PER_MBLOCK blocks allocGroup can return a
different number of blocks than requested. Here we use the number of
requested blocks, however arenaFree will subtract the actual number of
blocks we got from arena_blocks (possibly) resulting in a negative value
and triggering ASSERT(arena_blocks >= 0).

(See comments)

This helped me debug one of the bugs in #15508. I'm not sure if this is
a good idea, but it worked for me, so wanted to submit this as a MR.

The `nat` type was an alias for `unsigned int` with a comment saying
it was at least 32 bits. We keep the typedef in case client code is
using it but mark it as deprecated.

Test Plan: Validated on Linux, OS X and Windows

Reviewers: simonmar, austin, thomie, hvr, bgamari, hsyl20

Differential Revision: https://phabricator.haskell.org/D2166

This reverts commit 39b5c1cb.

This will hopefully help ensure some basic consistency in the forward by
overriding buffer variables. In particular, it sets the wrap length, the
offset to 4, and turns off tabs.
Signed-off-by: Austin Seipp <austin@well-typed.com>

Signed-off-by: Austin Seipp <austin@well-typed.com>

lnat was originally "long unsigned int" but we were using it when we
wanted a 64-bit type on a 64-bit machine.  This broke on Windows x64,
where long == int == 32 bits.  Using types of unspecified size is bad,
but what we really wanted was a type with N bits on an N-bit machine.
StgWord is exactly that.

lnat was mentioned in some APIs that clients might be using
(e.g. StackOverflowHook()), so we leave it defined but with a comment
to say that it's deprecated.

Store the *number* of the destination generation in the Bdescr struct,
so that in evacuate() we don't have to deref gen to get it.
This is another improvement ported over from my GC branch.

The GC had a two-level structure, G generations each of T steps.
Steps are for aging within a generation, mostly to avoid premature
promotion.  

Measurements show that more than 2 steps is almost never worthwhile,
and 1 step is usually worse than 2.  In theory fractional steps are
possible, so the ideal number of steps is somewhere between 1 and 3.
GHC's default has always been 2.

We can implement 2 steps quite straightforwardly by having each block
point to the generation to which objects in that block should be
promoted, so blocks in the nursery point to generation 0, and blocks
in gen 0 point to gen 1, and so on.

This commit removes the explicit step structures, merging generations
with steps, thus simplifying a lot of code.  Performance is
unaffected.  The tunable number of steps is now gone, although it may
be replaced in the future by a way to tune the aging in generation 0.

At the moment, this just saves a memory reference in the GC inner loop
(worth a percent or two of GC time).  Later, it will hopefully let me
experiment with partial steps, and simplifying the generation/step
infrastructure.

The first phase of this tidyup is focussed on the header files, and in
particular making sure we are exposinng publicly exactly what we need
to, and no more.

 - Rts.h now includes everything that the RTS exposes publicly,
   rather than a random subset of it.

 - Most of the public header files have moved into subdirectories, and
   many of them have been renamed.  But clients should not need to
   include any of the other headers directly, just #include the main
   public headers: Rts.h, HsFFI.h, RtsAPI.h.

 - All the headers needed for via-C compilation have moved into the
   stg subdirectory, which is self-contained.  Most of the headers for
   the rest of the RTS APIs have moved into the rts subdirectory.

 - I left MachDeps.h where it is, because it is so widely used in
   Haskell code.
 
 - I left a deprecated stub for RtsFlags.h in place.  The flag
   structures are now exposed by Rts.h.

 - Various internal APIs are no longer exposed by public header files.

 - Various bits of dead code and declarations have been removed

 - More gcc warnings are turned on, and the RTS code is more
   warning-clean.

 - More source files #include "PosixSource.h", and hence only use
   standard POSIX (1003.1c-1995) interfaces.

There is a lot more tidying up still to do, this is just the first
pass.  I also intend to standardise the names for external RTS APIs
(e.g use the rts_ prefix consistently), and declare the internal APIs
as hidden for shared libraries.

Now that constructor info tables contain the name of the constructor,
we can generate useful heap profiles without requiring the whole
program and libraries to be compiled with -prof.  So now, "+RTS -hT"
generates a heap profile for any program, dividing the profile by
constructor.  It wouldn't be hard to add support for grouping
constructors by module, or to restrict the profile to certain
constructors/modules/packages.

This means that for the first time we can get heap profiles for GHCi,
which was previously impossible because the byte-code
interpreter and linker don't work with -prof.

In preparation for parallel GC, split up the monolithic GC.c file into
smaller parts.  Also in this patch (and difficult to separate,
unfortunatley):
  
  - Don't include Stable.h in Rts.h, instead just include it where
    necessary.
  
  - consistently use STATIC_INLINE in source files, and INLINE_HEADER
    in header files.  STATIC_INLINE is now turned off when DEBUG is on,
    to make debugging easier.
  
  - The GC no longer takes the get_roots function as an argument.
    We weren't making use of this generalisation.

Most of the other users of the fptools build system have migrated to
Cabal, and with the move to darcs we can now flatten the source tree
without losing history, so here goes.

The main change is that the ghc/ subdir is gone, and most of what it
contained is now at the top level.  The build system now makes no
pretense at being multi-project, it is just the GHC build system.

No doubt this will break many things, and there will be a period of
instability while we fix the dependencies.  A straightforward build
should work, but I haven't yet fixed binary/source distributions.
Changes to the Building Guide will follow, too.

Move include <stdlib.h> later

Removed the annoying "Id" CVS keywords, they're a real PITA when it
comes to merging...

to ease replacement of the underlying allocator used by stgMallocBytes() and friends, provide (&use) RtsUtils.c:stgFree()

arenaAlloc(): tidy up alignment calc

include stdlib.h (to get at NULL)

Always return 8-byte-aligned memory from arenaAlloc().  Fixes problems
with profiling on sparc-sun-solaris2, and might fix problems with
Windows too.

MERGE

Remove most #includes of system headers from Stg.h, and instead
#include any required headers directly in each RTS source file.

The idea is to (a) reduce namespace pollution from system headers that
we don't need, (c) be clearer about dependencies on system things in
the RTS, and (c) improve via-C compilation times (maybe).

In practice though, HsBase.h #includes everything anyway, so the
difference from the point of view of .hc source is minimal.  However,
this makes it easier to move to zero-includes if we wanted to (see
discussion on the FFI list; I'm still not sure that's possible but
at least this is a step in the right direction).

Add a lightweight arena allocation scheme, and use it to speed up
allocation of cost centres and cost-centre stacks in the profiler.