The `whereFrom` function provides a Haskell interface for using the
information created by `-finfo-table-map`. Given a Haskell value, the
info table address will be passed to the `lookupIPE` function in order
to attempt to find the source location information for that particular closure.

At the moment it's not possible to distinguish the absense of the map
and a failed lookup.

The `-fdistinct-constructor-tables` flag will generate a fresh info
table for the usage of any data constructor. This is useful for
debugging as now by inspecting the info table, you can determine which
usage of a constructor caused that allocation rather than the old
situation where the info table always mapped to the definition site of
the data constructor which is useless.

In conjunction with `-hi` and `-finfo-table-map` this gives a more fine
grained understanding of where constructor allocations arise from in a
program.

This new flag embeds a lookup table from the address of an info table
to information about that info table.

The main interface for consulting the map is the `lookupIPE` C function

> InfoProvEnt * lookupIPE(StgInfoTable *info)

The `InfoProvEnt` has the following structure:

> typedef struct InfoProv_{
>     char * table_name;
>     char * closure_desc;
>     char * ty_desc;
>     char * label;
>     char * module;
>     char * srcloc;
> } InfoProv;
>
> typedef struct InfoProvEnt_ {
>     StgInfoTable * info;
>     InfoProv prov;
>     struct InfoProvEnt_ *link;
> } InfoProvEnt;

The source positions are approximated in a similar way to the source
positions for DWARF debugging information. They are only approximate but
in our experience provide a good enough hint about where the problem
might be. It is therefore recommended to use this flag in conjunction
with `-g<n>` for more accurate locations.

The lookup table is also emitted into the eventlog when it is available
as it is intended to be used with the `-hi` profiling mode.

Using this flag will significantly increase the size of the resulting
object file but only by a factor of 2-3x in our experience.

This profiling mode creates bands by the address of the info table for
each closure. This provides a much more fine-grained profiling output
than any of the other profiling modes.

The `-hi` profiling mode does not require a profiling build.

Matthias Andreas Benkard authored Jul 17, 2020 and Ben Gamari committed Nov 19, 2020

Adds commentary on the rationale for the changes made in merge request
!3689.

Matthias Andreas Benkard authored Jul 13, 2020 and Ben Gamari committed Nov 19, 2020

With the recent change away from first finding and then loading a
framework, loadFramework had no way of communicating the real reason
why loadDLL failed if it was any reason other than the framework
missing from the file system.  It now collects all loading attempt
errors into a list and concatenates them into a string to return to
the caller.

Matthias Andreas Benkard authored Jul 13, 2020 and Ben Gamari committed Nov 19, 2020

macOS Big Sur makes the following change to how frameworks are shipped
with the OS:

> New in macOS Big Sur 11 beta, the system ships with a built-in
> dynamic linker cache of all system-provided libraries. As part of
> this change, copies of dynamic libraries are no longer present on
> the filesystem. Code that attempts to check for dynamic library
> presence by looking for a file at a path or enumerating a directory
> will fail. Instead, check for library presence by attempting to
> dlopen() the path, which will correctly check for the library in the
> cache. (62986286)

https://developer.apple.com/documentation/macos-release-notes/macos-big-sur-11-beta-release-notes/

Therefore, the previous method of checking whether a library exists
before attempting to load it makes GHC.Runtime.Linker.loadFramework
fail to find frameworks installed at /System/Library/Frameworks.

GHC.Runtime.Linker.loadFramework now opportunistically loads the
framework libraries without checking for their existence first,
failing only if all attempts to load a given framework from any of the
various possible locations fail.

Previously we would allocate a linked list cell for each foreign export.
Now we can avoid this by taking advantage of the fact that they are
already broken into groups.

(cherry picked from commit 40dc9106)

This avoids calling `libc` in the initializers which are responsible for
registering foreign exports. We believe this should avoid the corruption
observed in #18548.

See Note [Tracking foreign exports] in rts/ForeignExports.c for an
overview of the new scheme.

(cherry picked from commit c4921349)

Andreas Klebinger authored Oct 27, 2020 and Ben Gamari committed Nov 18, 2020

We no compare these by doing 64bit subtraction and
checking the resulting flags.

We used to do this differently but the old approach was
broken when the high bits compared equal and the comparison
was one of >= or <=.

The new approach should be both correct and faster.

(cherry picked from commit bb100805)
(cherry picked from commit fda3e50b)

Previously the overflow check for the IMAGE_REL_AMD64_ADDR32NB
relocation failed to account for the signed nature of the value.
Specifically, the overflow check was:

    uint64_t v;
    v = S + A;
    if (v >> 32) { ... }

However, `v` ultimately needs to fit into 32-bits as a signed value.
Consequently, values `v > 2^31` in fact overflow yet this is not caught
by the existing overflow check.

Here we rewrite the overflow check to rather ensure that
`INT32_MIN <= v <= INT32_MAX`. There is now quite a bit of repetition
between the `IMAGE_REL_AMD64_REL32` and `IMAGE_REL_AMD64_ADDR32` cases
but I am leaving fixing this for future work.

This bug was first noticed by @awson.

Fixes #15808.

(cherry picked from commit 6d21ecee535782f01dba9947a49e282afee25724)

This was fixed by 4291bdda.

(cherry picked from commit ee5dcdf9)

toonn authored Oct 06, 2020 and Ben Gamari committed Oct 22, 2020

Replace an "as well" missing "as" with "and" in 4.1 Highlights.

Add missing apostrophe in "user's guide", insert space in "work around"
and dash in "cost-center" in 4.2.2 Runtime system.

Tamar Christina authored Aug 31, 2020 and Ben Gamari committed Oct 19, 2020

On windows the stack has to be allocated 4k at a time, otherwise we get
a segfault. This is done by using a helper ___chkstk_ms that is provided
by libgcc. The Haskell side already knows how to handle this but we need
to do the same from STG. Previously we would drop the stack in StgRun
but would only make it valid whenever the scheduler loop ran.

This approach was fundamentally broken in that it falls apart when you
take a signal from the OS. We see it less often because you initially
get allocated a 1MB stack block which you have to blow past first.

Concretely this means we must always keep the stack valid.

Fixes #18601.

(cherry picked from commit fd984d68)

!3798 added documentation and semantics for the flags,
but not parsing.

(cherry picked from commit fd302e93)

Fixes #17919.

(cherry picked from commit 587c3c514f05f97082952d613695f1186ff3174e)

Benjamin Maurer authored Sep 14, 2020 and Krzysztof Gogolewski committed Oct 16, 2020

_all_ of it, leaving nothing for, e.g., thread stacks.
Fix will only allocate 2/3rds and check whether remainder is at least large
enough for minimum amount of thread stacks.

(cherry picked from commit 74c797f6)

Ben Gamari authored Sep 17, 2020 and Krzysztof Gogolewski committed Oct 16, 2020

This struct has a number of fields and we only care that the value is
initialized with zeros. This eliminates the warnings noted in #17905.

(cherry picked from commit 057db94c)

Ben Gamari authored Aug 21, 2020 and Krzysztof Gogolewski committed Oct 16, 2020

When updating a TRec for a TVar already part of a transaction we
previously neglected to add the old value to the update remembered set.
I suspect this was the cause of #18587.

(cherry picked from commit 0799b3de)

Ben Gamari authored Sep 14, 2020 and Krzysztof Gogolewski committed Oct 16, 2020

As pointed out in #18685, this should be snprintf not vsnprintf. This
appears to be due to a cut-and-paste error.

Fixes #18658.

(cherry picked from commit ce42e187)

Ben Gamari authored Aug 11, 2020 and Krzysztof Gogolewski committed Oct 16, 2020

It's incredible that this wasn't noticed until now.

(cherry picked from commit ea1cbb8f)

Moritz Angermann authored Aug 25, 2020 and Krzysztof Gogolewski committed Oct 16, 2020

The commit in dff1cb3d incorrectly left
the `debugBelch` function without a comment or IF_DEBUG(linker,)
decoration. This rectifies it.

Needs at least a 8.10 backport, as it was backported in 6471cc6a

(cherry picked from commit 6189cc04)

Ben Gamari authored Aug 06, 2020 and Krzysztof Gogolewski committed Oct 16, 2020

(cherry picked from commit 2cdb72a569f6049a390626bca0dd6e362045ed65)

Conflicts:
	testsuite/tests/typecheck/should_compile/all.T

Ben Gamari authored Aug 06, 2020 and Krzysztof Gogolewski committed Oct 16, 2020

Previously the desugarer would instead fall over when it realized that
there was no unfolding for an imported function with a SPECIALISE
pragma. We now rather drop the SPECIALISE pragma and throw a warning.

Fixes #18118.

(cherry picked from commit 708e374a)

Moritz Angermann authored Aug 13, 2020 and Ben Gamari committed Oct 14, 2020

In b592bd98 we started using
-dead_strip_dylib on macOS when lining dynamic libraries and binaries.
The underlying reason being the Load Command Size Limit in macOS
Sierra (10.14) and later.

GHC will produce @rpath/libHS... dependency entries together with a
corresponding RPATH entry pointing to the location of the libHS...
library. Thus for every library we produce two Load Commands.  One to
specify the dependent library, and one with the path where to find it.
This makes relocating libraries and binaries easier, as we just need to
update the RPATH entry with the install_name_tool. The dynamic linker
will then subsitute each @rpath with the RPATH entries it finds in the
libraries load commands or the environement, when looking up @rpath
relative libraries.

-dead_strip_dylibs intructs the linker to drop unused libraries. This in
turn help us reduce the number of referenced libraries, and subsequently
the size of the load commands.  This h...

GHC GitLab CI authored Aug 09, 2020 and Ben Gamari committed Oct 06, 2020

The fix to #17962 ended up regressing on Windows as it failed to
replicate the logic responsible for overriding the toolchain paths on
Windows. This resulted in a hard-coded path to a directory that likely
doesn't exist on the user's system (#18550).

(cherry picked from commit 34e0fa96)

Ömer Sinan Ağacan authored Jun 17, 2020 and Ben Gamari committed Oct 06, 2020

Valgrind report of the bug when running the test `linker_unload`:

    ==29666== Conditional jump or move depends on uninitialised value(s)
    ==29666==    at 0x369C5B4: setOcInitialStatus (Linker.c:1305)
    ==29666==    by 0x369C6C5: mkOc (Linker.c:1347)
    ==29666==    by 0x36C027A: loadArchive_ (LoadArchive.c:522)
    ==29666==    by 0x36C0600: loadArchive (LoadArchive.c:626)
    ==29666==    by 0x2C144CD: ??? (in /home/omer/haskell/ghc_2/testsuite/tests/rts/linker/linker_unload.run/linker_unload)
    ==29666==
    ==29666== Conditional jump or move depends on uninitialised value(s)
    ==29666==    at 0x369C5B4: setOcInitialStatus (Linker.c:1305)
    ==29666==    by 0x369C6C5: mkOc (Linker.c:1347)
    ==29666==    by 0x369C9F6: preloadObjectFile (Linker.c:1507)
    ==29666==    by 0x369CA8D: loadObj_ (Linker.c:1536)
    ==29666==    by 0x369CB17: loadObj (Linker.c:1557)
    ==29666==    by 0x3866BC: main (linker_unload.c:33)

The problem is `mkOc` allocates a new `ObjectCode` and calls
`setOcInitialStatus` without initializing the `status` field.
`setOcInitialStatus` reads the field as first thing:

    static void setOcInitialStatus(ObjectCode* oc) {
        if (oc->status == OBJECT_DONT_RESOLVE)
          return;

        if (oc->archiveMemberName == NULL) {
            oc->status = OBJECT_NEEDED;
        } else {
            oc->status = OBJECT_LOADED;
        }
    }

`setOcInitialStatus` is unsed in two places for two different purposes:
in `mkOc` where we don't have the `status` field initialized yet (`mkOc`
is supposed to initialize it), and `loadOc` where we do have `status`
field initialized and we want to update it. Instead of splitting the
function into two functions which are both called just once I inline the
functions in the use sites and remove it.

Fixes #18342

(cherry picked from commit 08c1cb0f)

Fixes #18588 and #17956.

rts linker: teach the linker about GLIBC's special handling of *stat, mknod and atexit functions #7072

(cherry picked from commit 0effc57d)

Vladislav Zavialov authored Aug 20, 2020 and Krzysztof Gogolewski committed Aug 29, 2020

In preparation for the next version of 'happy', c95920 added a qualified
import to GHC/Parser.y but for some reason neglected GHC/Cmm/Parser.y

This patch adds the missing qualified import to GHC/Cmm/Parser.y and
also adds a clarifying comment to explain why this import is needed.

(cherry picked from commit fddddbf4)

This reverts commit aa2e5863.