GitLab

This merge does not turn on the new codegen (which only compiles
a select few programs at this point),
but it does introduce some changes to the old code generator.

The high bits:
1. The Rep Swamp patch is finally here.
   The highlight is that the representation of types at the
   machine level has changed.
   Consequently, this patch contains updates across several back ends.
2. The new Stg -> Cmm path is here, although it appears to have a
   fair number of bugs lurking.
3. Many improvements along the CmmCPSZ path, including:
   o stack layout
   o some code for infotables, half of which is right and half wrong
   o proc-point splitting

This patch entails a major restructuring of HscMain and a small bugfix
to MkIface (which required the restructuring in HscMain).

In MkIface:

  - mkIface* no longer outputs orphan warnings directly and also no
    longer quits GHC when -Werror is set.  Instead, errors are
    reported using the common IO (Messages, Maybe result) scheme.

In HscMain:

  - Get rid of the 'Comp' monad.  This monad was mostly GhcMonad + two
    reader arguments, a ModSummary for the currently compiled module
    and a possible old interface.  The latter actually lead to a small
    space-leak since only its hash was needed (to check whether the
    newly-generated interface file was the same as the original one).

    Functions originally of type 'Comp' now only take the arguments
    that they actually need.  This leads to slighly longer argument
    lists in some places, however, it is now much easier to see what
    is actually going on.

  - Get rid of 'myParseModule'.  Rename 'parseFile' to 'hscParse'.

  - Join 'deSugarModule' and 'hscDesugar' (keeping the latter).

  - Rename 'typecheck{Rename}Module{'}' to 'hscTypecheck{Rename}'.
    One variant keeps the renamed syntax, the other doesn't.

  - Parameterise 'HscStatus', so that 'InteractiveStatus' is just a
    different parameterisation of 'HscStatus'.

  - 'hscCompile{OneShot,Batch,Nothing,Interactive}' are now
    implemented using a (local) typeclass called 'HsCompiler'.  The
    idea is to make the common structure more obvious.  Using this
    typeclass we now have two functions 'genericHscCompile' (original
    'hscCompiler') and 'genericHscRecompile' (original 'genComp')
    describing the default pipeline.  The methods of the typeclass
    describe a sort of "hook" interface (in OO-terms this would be
    called the "template method" pattern).

    One problem with this approach is that we parameterise over the
    /result/ type which, in fact, is not actually different for
    "nothing" and "batch" mode.  To avoid functional dependencies or
    associated types, we use type tags to make them artificially
    different and parameterise the type class over the result type.
    A perhaps better approach might be to use records instead.
    
  - Drop some redundant 'HscEnv' arguments.  These were likely
    different from what 'getSession' would return because during
    compilation we temporarily set the module's DynFlags as well as a
    few other fields.  We now use the 'withTempSession' combinator to
    temporarily change the 'HscEnv' and automatically restore the
    original session after the enclosed action has returned (even in
    case of exceptions).

  - Rename 'hscCompile' to 'hscGenHardCode' (since that is what it
    does).

Calls in 'GHC' and 'DriverPipeline' accordingly needed small
adaptions.

A missing case (for equality predicates) in isOverloadedTy make
bindInstsOfLocalFuns/Pats do the wrong thing.  Core Lint nailed it.

Merge to 6.10 branch.

Particularly boolean expresions: the conditional of an 'if', and
guards, were missing their free variables.

Currently it only affects the -t flag output

It now uses the standard warning log and error reporting mechanism.

Previously, the GC had its own pool of threads to use as workers when
doing parallel GC.  There was a "leader", which was the mutator thread
that initiated the GC, and the other threads were taken from the pool.

This was simple and worked fine for sequential programs, where we did
most of the benchmarking for the parallel GC, but falls down for
parallel programs.  When we have N mutator threads and N cores, at GC
time we would have to stop N-1 mutator threads and start up N-1 GC
threads, and hope that the OS schedules them all onto separate cores.
It practice it doesn't, as you might expect.

Now we use the mutator threads to do GC.  This works quite nicely,
particularly for parallel programs, where each mutator thread scans
its own spark pool, which is probably in its cache anyway.

There are some flag changes:

  -g<n> is removed (-g1 is still accepted for backwards compat).
  There's no way to have a different number of GC threads than mutator
  threads now.

  -q1       Use one OS thread for GC (turns off parallel GC)
  -qg<n>    Use parallel GC for generations >= <n> (default: 1)

Using parallel GC only for generations >=1 works well for sequential
programs.  Compiling an ordinary sequential program with -threaded and
running it with -N2 or more should help if you do a lot of GC.  I've
found that adding -qg0 (do parallel GC for generation 0 too) speeds up
some parallel programs, but slows down some sequential programs.
Being conservative, I left the threshold at 1.

ToDo: document the new options.

Parse errors during dependency analysis or options parsing really
shouldn't kill GHC; this is particularly annoying for GHC API clients.

After a deadlock it was possible for the timer signal to remain off,
which meant that the next deadlock would not be detected, and the
system would hang.  Spotted by conc047(threaded2).

Also avoid padding if possible using __attribute__((packed))
Fixes the Windows build

I discovered a new invariant while experimenting (blackholing is not
optional when using parallel GC), so documented it.

Fixes crashes when using reclaimSpark() (not used currently, but may
be in the future).

If we're using the system's installed GMP, we don't want to be picking
up the local gmp.h header file.

Fixes 2469(ghci) for me, because it turns out the system's GMP is more
up-to-date than GHC's version and has a fix for more recent versions
of gcc.  We also need to pull in a more recent GMP, but that's a
separte issue.

There were races between workerTaskStop() and freeTaskManager(): we
need to be sure that all Tasks have exited properly before we start
tearing things down.  This isn't completely straighforward, see
comments for details.

Eager blackholing can improve parallel performance by reducing the
chances that two threads perform the same computation.  However, it
has a cost: one extra memory write per thunk entry.  

To get the best results, any code which may be executed in parallel
should be compiled with eager blackholing turned on.  But since
there's a cost for sequential code, we make it optional and turn it on
for the parallel package only.  It might be a good idea to compile
applications (or modules) with parallel code in with
-feager-blackholing.

ToDo: document -feager-blackholing.

At some point we regressed on detecting simple black-hole loops.  This
happened due to the introduction of duplicate-work detection for
parallelism: a black-hole loop looks very much like duplicate work,
except it's duplicate work being performed by the very same thread.
So we have to detect and handle this case.

On x86_64, the RTS needs to allocate memory in the low 2Gb of the
address space.  On Linux we can do this with MAP_32BIT, but sometimes
this doesn't work (#2512) and other OSs don't support it at all
(#2063).  So to work around this:

  - Try MAP_32BIT first, if available.

  - Otherwise, try allocating memory from a fixed address (by default
    1Gb)

  - We now provide an option to configure the address to allocate
    from.  This allows a workaround on machines where the default
    breaks, and also provides a way for people to test workarounds
    that we can incorporate in future releases.

If we encounter a runnable thread during shutdown, just kill it.  All
the threads are supposed to be dead at this stage, but this catches
threads that might have just returned from a foreign call, or were
finalizers created by the GC.

Fixes memo002(threaded1)

Based on a patch from juhpetersen.

They cause compilation errors (correctly) with newer gccs
Shows up compiling the RTS via C, which happens on Windows