GitLab

This reverts commit b949c96b.

Summary:
In a workload with a large amount of code, zero_static_objects_list()
takes a significant amount of time, and furthermore it is in the
single-threaded part of the GC.

This patch uses a slightly fiddly scheme for marking objects on the
static object lists, using a flag in the low 2 bits that flips between
two states to indicate whether an object has been visited during this
GC or not.  We also have to take into account objects that have not
been visited yet, which might appear at any time due to runtime linking.

Test Plan: validate

Reviewers: austin, bgamari, ezyang, rwbarton

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D1076

This patch solves the scoping problem of CmmTick nodes: If we just put
CmmTicks into blocks we have no idea what exactly they are meant to
cover.  Here we introduce tick scopes, which allow us to create
sub-scopes and merged scopes easily.

Notes:

* Given that the code often passes Cmm around "head-less", we have to
  make sure that its intended scope does not get lost. To keep the amount
  of passing-around to a minimum we define a CmmAGraphScoped type synonym
  here that just bundles the scope with a portion of Cmm to be assembled
  later.

* We introduce new scopes at somewhat random places, aligning with
  getCode calls. This works surprisingly well, but we might have to
  add new scopes into the mix later on if we find things too be too
  coarse-grained.

(From Phabricator D169)

Summary:
This includes pretty much all the changes needed to make `Applicative`
a superclass of `Monad` finally. There's mostly reshuffling in the
interests of avoid orphans and boot files, but luckily we can resolve
all of them, pretty much. The only catch was that
Alternative/MonadPlus also had to go into Prelude to avoid this.

As a result, we must update the hsc2hs and haddock submodules.
Signed-off-by: Austin Seipp <austin@well-typed.com>

Test Plan: Build things, they might not explode horribly.

Reviewers: hvr, simonmar

Subscribers: simonmar

Differential Revision: https://phabricator.haskell.org/D13

Summary:
Previously, both Cabal and GHC defined the type PackageId, and we expected
them to be roughly equivalent (but represented differently).  This refactoring
separates these two notions.

A package ID is a user-visible identifier; it's the thing you write in a
Cabal file, e.g. containers-0.9.  The components of this ID are semantically
meaningful, and decompose into a package name and a package vrsion.

A package key is an opaque identifier used by GHC to generate linking symbols.
Presently, it just consists of a package name and a package version, but
pursuant to #9265 we are planning to extend it to record other information.
Within a single executable, it uniquely identifies a package.  It is *not* an
InstalledPackageId, as the choice of a package key affects the ABI of a package
(whereas an InstalledPackageId is computed after compilation.)  Cabal computes
a package key for the package and passes it to GHC using -package-name (now
*extremely* misnamed).

As an added bonus, we don't have to worry about shadowing anymore.

As a follow on, we should introduce -current-package-key having the same role as
-package-name, and deprecate the old flag.  This commit is just renaming.

The haddock submodule needed to be updated.
Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

Test Plan: validate

Reviewers: simonpj, simonmar, hvr, austin

Subscribers: simonmar, relrod, carter

Differential Revision: https://phabricator.haskell.org/D79

Conflicts:
	compiler/main/HscTypes.lhs
	compiler/main/Packages.lhs
	utils/haddock

In some cases, the layout of the LANGUAGE/OPTIONS_GHC lines has been
reorganized, while following the convention, to

- place `{-# LANGUAGE #-}` pragmas at the top of the source file, before
  any `{-# OPTIONS_GHC #-}`-lines.

- Moreover, if the list of language extensions fit into a single
  `{-# LANGUAGE ... -#}`-line (shorter than 80 characters), keep it on one
  line. Otherwise split into `{-# LANGUAGE ... -#}`-lines for each
  individual language extension. In both cases, try to keep the
  enumeration alphabetically ordered.
  (The latter layout is preferable as it's more diff-friendly)

While at it, this also replaces obsolete `{-# OPTIONS ... #-}` pragma
occurences by `{-# OPTIONS_GHC ... #-}` pragmas.

Documentation in response to Johan's questions

Plus, don't export hpRel from StgCmmHeap, StgCmmLayout
  (it is only used locally in StgCmmLayout)

Also make sure allocHeapClosure updates profiling counters with the
memory allocated.

- Move array representation knowledge into SMRep

- Separate out low-level heap-object allocation so that we can reuse
  it from doNewArrayOp

- remove card-table initialisation, we can safely ignore the card
  table for newly allocated arrays.

I'd like to be able to pack together non-pointer fields that are less
than a word in size, and this is a necessary prerequisite.

Fixes #8585

When emmiting label of a self-recursive tail call (ie. when
performing loopification optimization) we emit the loop header
label after a stack check but before the heap check. The reason is
that tail-recursive functions use constant amount of stack space
so we don't need to repeat the check in every loop. But they can
grow the heap so heap check must be repeated in every call.
See Note [Self-recursive tail calls] and [Self-recursive loop header].

When compiling a function we can determine how much stack space it will
use. We therefore need to perform only a single stack check at the beginning
of a function to see if we have enough stack space. Instead of referring
directly to Sp - as we used to do in the past - the code generator uses
(old + 0) in the stack check. Stack layout phase turns (old + 0) into Sp.

The idea here is that, while we need to perform only one stack check for
each function, we could in theory place more stack checks later in the
function. They would be redundant, but not incorrect (in a sense that they
should not change program behaviour). We need to make sure however that a
stack check inserted after incrementing the stack pointer checks for a
respectively smaller stack space. This would not be the case if the code
generator produced direct references to Sp. By referencing (old + 0) we make
sure that we always check for a correct amount of stack: when converting
(old + 0) to Sp the stack layout phase takes into account changes already
made to stack pointer. The idea for this change came from observations made
while debugging #8275.

Signed-off-by: Edward Z. Yang <ezyang@mit.edu>

This commit removes module StgCmmGran which has only no-op functions.
According to comments in the module, it was used by GpH, but GpH
project seems to be dead for a couple of years now.

A major cleanup of trailing whitespaces and tabs in codeGen/
directory. I also adjusted code formatting in some places.

This includes selector, ap, and constructor thunks. They are still
guarded by the -ticky-dyn-thk flag.

(This is 024df664 with a small bug fix.)

This reverts commit 024df664.

Of course I gaff on my last day...

This includes selector, ap, and constructor thunks. They are still
guarded by the -ticky-dyn-thk flag.

  * the new StgCmmArgRep module breaks a dependency cycle; I also
    untabified it, but made no real changes

  * updated the documentation in the wiki and change the user guide to
    point there

  * moved the allocation enters for ticky and CCS to after the heap check

    * I left LDV where it was, which was before the heap check at least
      once, since I have no idea what it is

  * standardized all (active?) ticky alloc totals to bytes

  * in order to avoid double counting StgCmmLayout.adjustHpBackwards
    no longer bumps ALLOC_HEAP_ctr

  * I resurrected the SLOW_CALL counters

    * the new module StgCmmArgRep breaks cyclic dependency between
      Layout and Ticky (which the SLOW_CALL counters cause)

    * renamed them SLOW_CALL_fast_<pattern> and VERY_SLOW_CALL

  * added ALLOC_RTS_ctr and _tot ticky counters

    * eg allocation by Storage.c:allocate or a BUILD_PAP in stg_ap_*_info

    * resurrected ticky counters for ALLOC_THK, ALLOC_PAP, and
      ALLOC_PRIM

    * added -ticky and -DTICKY_TICKY in ways.mk for debug ways

  * added a ticky counter for total LNE entries

  * new flags for ticky: -ticky-allocd -ticky-dyn-thunk -ticky-LNE

    * all off by default

    * -ticky-allocd: tracks allocation *of* closure in addition to
       allocation *by* that closure

    * -ticky-dyn-thunk tracks dynamic thunks as if they were functions

    * -ticky-LNE tracks LNEs as if they were functions

  * updated the ticky report format, including making the argument
    categories (more?) accurate again

  * the printed name for things in the report include the unique of
    their ticky parent as well as if they are not top-level

Top-level indirections are often generated when there is a cast, e.g.

foo :: T
foo = bar `cast` (some coercion)

For these we were generating a full-blown CAF, which is a fair chunk
of code.

This patch makes these indirections generate a single IND_STATIC
closure (4 words) instead.  This is exactly what the CAF would
evaluate to eventually anyway, we're just shortcutting the whole
process.

The Slow calling convention passes the closure in R1, but we were
ignoring this and hoping it would work, which it often did.  However,
this bug seems to have been the cause of #7192, because the
graph-colouring allocator is more sensitive to having correct liveness
information on jumps.

All Cmm procedures now include the set of global registers that are live on
procedure entry, i.e., the global registers used to pass arguments to the
procedure. Only global registers that are use to pass arguments are included in
this list.

Mostly d -> g (matching DynFlag -> GeneralFlag).
Also renamed if* to when*, matching the Haskell if/when names

The main change here is that the Cmm parser now allows high-level cmm
code with argument-passing and function calls.  For example:

foo ( gcptr a, bits32 b )
{
  if (b > 0) {
     // we can make tail calls passing arguments:
     jump stg_ap_0_fast(a);
  }

  return (x,y);
}

More details on the new cmm syntax are in Note [Syntax of .cmm files]
in CmmParse.y.

The old syntax is still more-or-less supported for those occasional
code fragments that really need to explicitly manipulate the stack.
However there are a couple of differences: it is now obligatory to
give a list of live GlobalRegs on every jump, e.g.

  jump %ENTRY_CODE(Sp(0)) [R1];

Again, more details in Note [Syntax of .cmm files].

I have rewritten most of the .cmm files in the RTS into the new
syntax, except for AutoApply.cmm which is generated by the genapply
program: this file could be generated in the new syntax instead and
would probably be better off for it, but I ran out of enthusiasm.

Some other changes in this batch:

 - The PrimOp calling convention is gone, primops now use the ordinary
   NativeNodeCall convention.  This means that primops and "foreign
   import prim" code must be written in high-level cmm, but they can
   now take more than 10 arguments.

 - CmmSink now does constant-folding (should fix #7219)

 - .cmm files now go through the cmmPipeline, and as a result we
   generate better code in many cases.  All the object files generated
   for the RTS .cmm files are now smaller.  Performance should be
   better too, but I haven't measured it yet.

 - RET_DYN frames are removed from the RTS, lots of code goes away

 - we now have some more canned GC points to cover unboxed-tuples with
   2-4 pointers, which will reduce code size a little.

The current fix is relatively dumb as far as where to add HpLim
checks: it will always perform a check unless we know that we're
returning from a closure or we are doing a non let-no-escape case
analysis.  The performance impact on the nofib suite looks like this:

            Min          +5.7%     -0.0%     -6.5%     -6.4%    -50.0%
            Max          +6.3%     +5.8%     +5.0%     +5.5%     +0.8%
 Geometric Mean          +6.2%     +0.1%     +0.5%     +0.5%     -0.8%

Overall, the executable bloat is the biggest problem, so we keep the old
omit-yields optimization on by default. Remember that if you need an
interruptibility guarantee, you need to recompile all of your libraries
with -fno-omit-yields.

A better fix would involve only inserting the yields necessary to break
loops; this is left as future work.
Signed-off-by: Edward Z. Yang <ezyang@mit.edu>

I've switched to passing DynFlags rather than Platform, as (a) it's
simpler to not have to extract targetPlatform in so many places, and
(b) it may be useful to have DynFlags around in future.

We weren't passing the arguments correctly to the GC functions, which
usually happened to work because the arguments were in the right
registers already.

After this fix the profiling tests go through with the new code
generator.

This saves compile time and can make a big difference in some
pathological cases (T4801)

To explicitly choose whether you want an unregisterised build you now
need to use the "--enable-unregisterised"/"--disable-unregisterised"
configure flags.

Instead of relying on common-block-elimination to share return
continuations in the common case (case-alternative heap checks) we do
it explicitly.  This isn't hard to do, is more robust, and saves some
compilation time.  Full commentary in Note [sharing continuations].

All the flags that 'ways' imply are now dynamic