Simon Peyton Jones authored 4 years ago and Timo von Holtz committed 4 years ago

Specifically:
  #13253 exponential inlining
  #10421 ditto
  #18140 strict constructors
  #18282 another nested-function call case

This patch makes one really significant changes: change the way that
mkDupableCont handles StrictArg.  The details are explained in
GHC.Core.Opt.Simplify Note [Duplicating StrictArg].

Specific changes

* In mkDupableCont, when making auxiliary bindings for the other arguments
  of a call, add extra plumbing so that we don't forget the demand on them.
  Otherwise we haev to wait for another round of strictness analysis. But
  actually all the info is to hand.  This change affects:
  - Make the strictness list in ArgInfo be [Demand] instead of [Bool],
    and rename it to ai_dmds.
  - Add as_dmd to ValArg
  - Simplify.makeTrivial takes a Demand
  - mkDupableContWithDmds takes a [Demand]

There are a number of other small changes

1. For Ids that are used at most once in each branch of a case, make
   the occurrence analyser record the total number of syntactic
   occurrences.  Previously we recorded just OneBranch or
   MultipleBranches.

   I thought this was going to be useful, but I ended up barely
   using it; see Note [Note [Suppress exponential blowup] in
   GHC.Core.Opt.Simplify.Utils

   Actual changes:
     * See the occ_n_br field of OneOcc.
     * postInlineUnconditionally

2. I found a small perf buglet in SetLevels; see the new
   function GHC.Core.Opt.SetLevels.hasFreeJoin

3. Remove the sc_cci field of StrictArg.  I found I could get
   its information from the sc_fun field instead.  Less to get
   wrong!

4. In ArgInfo, arrange that ai_dmds and ai_discs have a simpler
   invariant: they line up with the value arguments beyond ai_args
   This allowed a bit of nice refactoring; see isStrictArgInfo,
   lazyArgcontext, strictArgContext

There is virtually no difference in nofib. (The runtime numbers
are bogus -- I tried a few manually.)

        Program           Size    Allocs   Runtime   Elapsed  TotalMem
--------------------------------------------------------------------------------
            fft          +0.0%     -2.0%    -48.3%    -49.4%      0.0%
     multiplier          +0.0%     -2.2%    -50.3%    -50.9%      0.0%
--------------------------------------------------------------------------------
            Min          -0.4%     -2.2%    -59.2%    -60.4%      0.0%
            Max          +0.0%     +0.1%     +3.3%     +4.9%      0.0%
 Geometric Mean          +0.0%     -0.0%    -33.2%    -34.3%     -0.0%

Test T18282 is an existing example of these deeply-nested strict calls.
We get a big decrease in compile time (-85%) because so much less
inlining takes place.

Metric Decrease:
    T18282

(cherry picked from commit 0bd60059)

toonn authored 4 years ago and Ben Gamari committed 4 years ago

Replace an "as well" missing "as" with "and" in 4.1 Highlights.

Add missing apostrophe in "user's guide", insert space in "work around"
and dash in "cost-center" in 4.2.2 Runtime system.

Tamar Christina authored 4 years ago and Ben Gamari committed 4 years ago

On windows the stack has to be allocated 4k at a time, otherwise we get
a segfault. This is done by using a helper ___chkstk_ms that is provided
by libgcc. The Haskell side already knows how to handle this but we need
to do the same from STG. Previously we would drop the stack in StgRun
but would only make it valid whenever the scheduler loop ran.

This approach was fundamentally broken in that it falls apart when you
take a signal from the OS. We see it less often because you initially
get allocated a 1MB stack block which you have to blow past first.

Concretely this means we must always keep the stack valid.

Fixes #18601.

(cherry picked from commit fd984d68)

!3798 added documentation and semantics for the flags,
but not parsing.

(cherry picked from commit fd302e93)

Fixes #17919.

(cherry picked from commit 587c3c514f05f97082952d613695f1186ff3174e)

Benjamin Maurer authored 4 years ago and Krzysztof Gogolewski committed 4 years ago

_all_ of it, leaving nothing for, e.g., thread stacks.
Fix will only allocate 2/3rds and check whether remainder is at least large
enough for minimum amount of thread stacks.

(cherry picked from commit 74c797f6)

Ben Gamari authored 4 years ago and Krzysztof Gogolewski committed 4 years ago

This struct has a number of fields and we only care that the value is
initialized with zeros. This eliminates the warnings noted in #17905.

(cherry picked from commit 057db94c)

Ben Gamari authored 4 years ago and Krzysztof Gogolewski committed 4 years ago

When updating a TRec for a TVar already part of a transaction we
previously neglected to add the old value to the update remembered set.
I suspect this was the cause of #18587.

(cherry picked from commit 0799b3de)

Ben Gamari authored 4 years ago and Krzysztof Gogolewski committed 4 years ago

As pointed out in #18685, this should be snprintf not vsnprintf. This
appears to be due to a cut-and-paste error.

Fixes #18658.

(cherry picked from commit ce42e187)

Ben Gamari authored 4 years ago and Krzysztof Gogolewski committed 4 years ago

It's incredible that this wasn't noticed until now.

(cherry picked from commit ea1cbb8f)

Moritz Angermann authored 4 years ago and Krzysztof Gogolewski committed 4 years ago

The commit in dff1cb3d incorrectly left
the `debugBelch` function without a comment or IF_DEBUG(linker,)
decoration. This rectifies it.

Needs at least a 8.10 backport, as it was backported in 6471cc6a

(cherry picked from commit 6189cc04)

Ben Gamari authored 4 years ago and Krzysztof Gogolewski committed 4 years ago

(cherry picked from commit 2cdb72a569f6049a390626bca0dd6e362045ed65)

Conflicts:
	testsuite/tests/typecheck/should_compile/all.T

Ben Gamari authored 4 years ago and Krzysztof Gogolewski committed 4 years ago

Previously the desugarer would instead fall over when it realized that
there was no unfolding for an imported function with a SPECIALISE
pragma. We now rather drop the SPECIALISE pragma and throw a warning.

Fixes #18118.

(cherry picked from commit 708e374a8bf108999c11b6cf59c7d27677ed24a8)

Moritz Angermann authored 4 years ago and Ben Gamari committed 4 years ago

In b592bd98 we started using
-dead_strip_dylib on macOS when lining dynamic libraries and binaries.
The underlying reason being the Load Command Size Limit in macOS
Sierra (10.14) and later.

GHC will produce @rpath/libHS... dependency entries together with a
corresponding RPATH entry pointing to the location of the libHS...
library. Thus for every library we produce two Load Commands.  One to
specify the dependent library, and one with the path where to find it.
This makes relocating libraries and binaries easier, as we just need to
update the RPATH entry with the install_name_tool. The dynamic linker
will then subsitute each @rpath with the RPATH entries it finds in the
libraries load commands or the environement, when looking up @rpath
relative libraries.

-dead_strip_dylibs intructs the linker to drop unused libraries. This in
turn help us reduce the number of referenced libraries, and subsequently
the size of the load commands.  This however does not remove the RPATH
entries.  Subsequently we can end up (in extreme cases) with only a
single @rpath/libHS... entry, but 100s or more RPATH entries in the Load
Commands.

This patch rectifies this (slighly unorthodox) by passing *no* -rpath
arguments to the linker at link time, but -headerpad 8000.  The
headerpad argument is in hexadecimal and the maxium 32k of the load
command size.  This tells the linker to pad the load command section
enough for us to inject the RPATHs later.  We then proceed to link the
library or binary with -dead_strip_dylibs, and *after* the linking
inspect the library to find the left over (non-dead-stripped)
dependencies (using otool).  We find the corresponding RPATHs for each
@rpath relative dependency, and inject them into the library or binary
using the install_name_tool.  Thus achieving a deadstripped dylib (and
rpaths) build product.

We can not do this in GHC, without starting to reimplement a dynamic
linker as we do not know which symbols and subsequently libraries are
necessary.

Commissioned-by: Mercury Technologies, Inc. (mercury.com)
(cherry picked from commit 4ff93292)
Signed-off-by: Moritz Angermann <moritz.angermann@iohk.io>

GHC GitLab CI authored 4 years ago and Ben Gamari committed 4 years ago

The fix to #17962 ended up regressing on Windows as it failed to
replicate the logic responsible for overriding the toolchain paths on
Windows. This resulted in a hard-coded path to a directory that likely
doesn't exist on the user's system (#18550).

(cherry picked from commit 34e0fa96)

Valgrind report of the bug when running the test `linker_unload`:

    ==29666== Conditional jump or move depends on uninitialised value(s)
    ==29666==    at 0x369C5B4: setOcInitialStatus (Linker.c:1305)
    ==29666==    by 0x369C6C5: mkOc (Linker.c:1347)
    ==29666==    by 0x36C027A: loadArchive_ (LoadArchive.c:522)
    ==29666==    by 0x36C0600: loadArchive (LoadArchive.c:626)
    ==29666==    by 0x2C144CD: ??? (in /home/omer/haskell/ghc_2/testsuite/tests/rts/linker/linker_unload.run/linker_unload)
    ==29666==
    ==29666== Conditional jump or move depends on uninitialised value(s)
    ==29666==    at 0x369C5B4: setOcInitialStatus (Linker.c:1305)
    ==29666==    by 0x369C6C5: mkOc (Linker.c:1347)
    ==29666==    by 0x369C9F6: preloadObjectFile (Linker.c:1507)
    ==29666==    by 0x369CA8D: loadObj_ (Linker.c:1536)
    ==29666==    by 0x369CB17: loadObj (Linker.c:1557)
    ==29666==    by 0x3866BC: main (linker_unload.c:33)

The problem is `mkOc` allocates a new `ObjectCode` and calls
`setOcInitialStatus` without initializing the `status` field.
`setOcInitialStatus` reads the field as first thing:

    static void setOcInitialStatus(ObjectCode* oc) {
        if (oc->status == OBJECT_DONT_RESOLVE)
          return;

        if (oc->archiveMemberName == NULL) {
            oc->status = OBJECT_NEEDED;
        } else {
            oc->status = OBJECT_LOADED;
        }
    }

`setOcInitialStatus` is unsed in two places for two different purposes:
in `mkOc` where we don't have the `status` field initialized yet (`mkOc`
is supposed to initialize it), and `loadOc` where we do have `status`
field initialized and we want to update it. Instead of splitting the
function into two functions which are both called just once I inline the
functions in the use sites and remove it.

Fixes #18342

(cherry picked from commit 08c1cb0f)

Fixes #18588 and #17956.

rts linker: teach the linker about GLIBC's special handling of *stat, mknod and atexit functions #7072

(cherry picked from commit 0effc57d)

Vladislav Zavialov authored 4 years ago and Krzysztof Gogolewski committed 4 years ago

In preparation for the next version of 'happy', c95920 added a qualified
import to GHC/Parser.y but for some reason neglected GHC/Cmm/Parser.y

This patch adds the missing qualified import to GHC/Cmm/Parser.y and
also adds a clarifying comment to explain why this import is needed.

(cherry picked from commit fddddbf4)

This reverts commit aa2e5863.

We want to only run the check if ld is gold.

Fixes the fix to #17962.

(cherry picked from commit d589ef25)

Previously the code generator could produce corrupt C call sequences due
to register overlap between MachOp lowerings and the platform's calling
convention. We fix this using a hack described in Note [Evaluate C-call
arguments before placing in destination registers].

(cherry picked from commit a647bae8)

Previously to merge a set of object files we would invoke the linker as
usual, adding -r to the command-line. However, this can result in
non-sensical command-lines which causes lld to balk (#17962).

To avoid this we introduce a new tool setting into GHC, -pgmlm, which is
the linker which we use to merge object files.

This reverts commit 2290eb02.

(cherry picked from commit f9b7497d)

Leon Schoorl authored 4 years ago and Ben Gamari committed 4 years ago

Fixes #18070

GHC_STAGE is the stage of the compiler we're building, it should be 1,2(,3?).
But make was generating 0 and 1.

Hadrian does this correctly using a similar `+ 1`:
https://gitlab.haskell.org/ghc/ghc/-/blob/eb8115a8c4cbc842b66798480fefc7ab64d31931/hadrian/src/Rules/Generate.hs#L245

(cherry picked from commit a481afef354ca0383477fd20891c2e8eb9ef4449)

Otherwise we use ld.lld, which fails with

  ld.lld: error: -r and --export-dynamic may not be used together

It appears that this was incorrectly reverted in
4186c713.