Matthew Pickering authored 3 years ago and Zubin committed 3 years ago

The logic didn't account for the fact that the paths could contain
spaces before which led to errors such as the following from
install_name_tool.

Stderr ( T14304 ):
Warning: -rtsopts and -with-rtsopts have no effect with -shared.
    Call hs_init_ghc() from your main() function to set these options.
error: /nix/store/a6j5761iy238pbckxq2xrhqr2d5kra4m-cctools-binutils-darwin-949.0.1/bin/install_name_tool: for: dist/build/libHSp-0.1-ghc8.10.6.dylib (for architecture arm64) option "-add_rpath /Users/matt/ghc/bindisttest/install   dir/lib/ghc-8.10.6/ghc-prim-0.6.1" would duplicate path, file already has LC_RPATH for: /Users/matt/ghc/bindisttest/install   dir/lib/ghc-8.10.6/ghc-prim-0.6.1
`install_name_tool' failed in phase `Install Name Tool'. (Exit code: 1)

Fixes #20212

andreas.abel authored 3 years ago and Zubin committed 3 years ago

Added :since: fields to users_guide on warning, for warnings
introduced starting GHC 6.0.

The data was extracted from the HTML docs on warnings, see

  ghc/ghc#18087

and partially verified by consulting the change logs.

(cherry picked from commit 0b403319)

andreas.abel authored 3 years ago and Zubin committed 3 years ago

New warnings are only released in major versions, it seems.
One way or the other, a .1 minor version can always be dropped.

(cherry picked from commit ebcdf3fa)

andreas.abel authored 3 years ago and Zubin committed 3 years ago

(cherry picked from commit 20b352eb)

Ben Gamari authored 4 years ago and Zubin committed 3 years ago

Instead of relying on RTS_LINKER_USE_MMAP

(cherry picked from commit c35d0e03)

Ben Gamari authored 4 years ago and Zubin committed 3 years ago

Previously we would check only that the *start* of the mapping was in
the bottom 32-bits of address space. However, we need the *entire*
mapping to be in low memory. Fix this.

Noticed by @Phyx.

Ben Gamari authored 4 years ago and Zubin committed 3 years ago

Previously most of the uses of mmapForLinker were mapping anonymous
memory, resulting in a great deal of unnecessary repetition. Factor this
out into a new helper.

Also fixes a few places where error checking was missing or suboptimal.

(cherry picked from commit add0aeae)

Ben Gamari authored 4 years ago and Zubin committed 3 years ago

Previously, in an attempt to reduce fragmentation, each new allocator
would map a region of M32_MAX_PAGES fresh pages to seed itself. However,
this ends up being extremely wasteful since it turns out that we often
use fewer than this.  Consequently, these pages end up getting freed
which, ends up fragmenting our address space more than than we would
have if we had naively allocated pages on-demand.

Here we refactor m32 to avoid this waste while achieving the
fragmentation mitigation previously desired. In particular, we move all
page allocation into the global m32_alloc_page, which will pull a page
from the free page pool. If the free page pool is empty we then refill
it by allocating a region of M32_MAP_PAGES and adding them to the pool.

Furthermore, we do away with the initial seeding entirely. That is, the
allocator starts with no active pages: pages are rather allocated on an
as-needed basis.

On the whole this ends up being a pleasingly simple change,
simultaneously making m32 more efficient, more robust, and simpler.

Fixes #18980.

(cherry picked from commit 085bfdeb)

Matthew Pickering authored 3 years ago and Zubin committed 3 years ago

The symlink structure now looks like:

```
lrwxrwxrwx 1 matt users       16 Aug  3 16:27 _build/bindist/ghc-9.3.20210721-x86_64-unknown-linux/bin/ghc -> ghc-9.3.20210721
-rwxr-xr-x 1 matt users  1750336 Aug  3 15:00 _build/bindist/ghc-9.3.20210721-x86_64-unknown-linux/bin/ghc-9.3.20210721
lrwxrwxrwx 1 matt users       22 Aug  3 16:27 _build/bindist/ghc-9.3.20210721-x86_64-unknown-linux/bin/ghc-iserv -> ghc-iserv-9.3.20210721
-rwxr-xr-x 1 matt users 31703176 Aug  3 15:00 _build/bindist/ghc-9.3.20210721-x86_64-unknown-linux/bin/ghc-iserv-9.3.20210721
lrwxrwxrwx 1 matt users       26 Aug  3 16:27 _build/bindist/ghc-9.3.20210721-x86_64-unknown-linux/bin/ghc-iserv-dyn -> ghc-iserv-dyn-9.3.20210721
-rwxr-xr-x 1 matt users    40808 Aug  3 15:00 _build/bindist/ghc-9.3.20210721-x86_64-unknown-linux/bin/ghc-iserv-dyn-9.3.20210721
lrwxrwxrwx 1 matt users       20 Aug  3 16:27 _build/bindist/ghc-9.3.20210721-x86_64-unknown-linux/bin/ghc-pkg -> ghc-pkg-9.3.20210721
-rwxr-xr-x 1 matt users   634872 Aug  3 15:00 _build/bindist/ghc-9.3.20210721-x86_64-unknown-linux/bin/ghc-pkg-9.3.20210721
lrwxrwxrwx 1 matt users       14 Aug  3 16:27 _build/bindist/ghc-9.3.20210721-x86_64-unknown-linux/bin/haddock -> haddock-2.24.0
-rwxr-xr-x 1 matt users  4336664 Aug  3 15:00 _build/bindist/ghc-9.3.20210721-x86_64-unknown-linux/bin/haddock-2.24.0
lrwxrwxrwx 1 matt users        9 Aug  3 16:27 _build/bindist/ghc-9.3.20210721-x86_64-unknown-linux/bin/hp2ps -> hp2ps-0.1
-rwxr-xr-x 1 matt users    49312 Aug  3 15:00 _build/bindist/ghc-9.3.20210721-x86_64-unknown-linux/bin/hp2ps-0.1
lrwxrwxrwx 1 matt users        8 Aug  3 16:27 _build/bindist/ghc-9.3.20210721-x86_64-unknown-linux/bin/hpc -> hpc-0.68
-rwxr-xr-x 1 matt users   687896 Aug  3 15:00 _build/bindist/ghc-9.3.20210721-x86_64-unknown-linux/bin/hpc-0.68
lrwxrwxrwx 1 matt users       13 Aug  3 16:27 _build/bindist/ghc-9.3.20210721-x86_64-unknown-linux/bin/hsc2hs -> hsc2hs-0.68.8
-rwxr-xr-x 1 matt users   729904 Aug  3 15:00 _build/bindist/ghc-9.3.20210721-x86_64-unknown-linux/bin/hsc2hs-0.68.8
lrwxrwxrwx 1 matt users       19 Aug  3 16:27 _build/bindist/ghc-9.3.20210721-x86_64-unknown-linux/bin/runghc -> runghc-9.3.20210721
-rwxr-xr-x 1 matt users    57672 Aug  3 15:00 _build/bindist/ghc-9.3.20210721-x86_64-unknown-linux/bin/runghc-9.3.20210721
lrwxrwxrwx 1 matt users        9 Aug  3 16:27 _build/bindist/ghc-9.3.20210721-x86_64-unknown-linux/bin/unlit -> unlit-0.1
-rwxr-xr-x 1 matt users    14896 Aug  3 15:00 _build/bindist/ghc-9.3.20210721-x86_64-unknown-linux/bin/unlit-0.1
```

Fixes #20198

(cherry picked from commit e50267d9)

Matthew Pickering authored 3 years ago and Zubin committed 3 years ago

(cherry picked from commit 7b0ceafb)

Matthew Pickering authored 3 years ago and Zubin committed 3 years ago

The package.cache needs to have a later mod-time than all of the .conf
files. This invariant can be destroyed by `cp -r` and so we run `ghc-pkg
recache` to ensure the package database which is distributed is
consistent.

If you are installing a relocatable bindist, for example, on windows,
you should preserve mtimes by using cp -a or run ghc-pkg recache after
installing.

(cherry picked from commit 957fe359)

Matthew Pickering authored 3 years ago and Zubin committed 3 years ago

(cherry picked from commit 45f05554)

Matthew Pickering authored 3 years ago and Zubin committed 3 years ago

(cherry picked from commit 22a16b0f)

Matthew Pickering authored 3 years ago and Zubin committed 3 years ago

Fixes #19868

(cherry picked from commit 5412730e)

Matthew Pickering authored 3 years ago and Zubin committed 3 years ago

(cherry picked from commit acc47bd2)

Matthew Pickering authored 3 years ago and Zubin committed 3 years ago

Before we would just copy the unversioned executable into the bindist.
Now the actual executable is copied into the bindist and a version
suffix is added. Then a wrapper or symlink is added which points to the
versioned executable.

Fixes #20074

(cherry picked from commit f481c189)

Matthew Pickering authored 3 years ago and Zubin committed 3 years ago

(cherry picked from commit 172fd5d1)

Matthew Pickering authored 3 years ago and Zubin committed 3 years ago

(cherry picked from commit e4c25261)

Matthew Pickering authored 3 years ago and Zubin committed 3 years ago

Exes
----
Before: The whole bin/ folder was copied which could contain random old/stale/testsuite executables
After: Be precise

Wrappers
--------
Before: Wrappers were created for everything in the bin folder,
including internal executables such as "unlit"
After: Only create wrappers for the specific things which we want to
include in the user's path.

This makes the hadrian bindists match up more closely with the make
bindists.

(cherry picked from commit 888eadb9)

Moritz Angermann authored 3 years ago and Zubin committed 3 years ago

In `checkBlockingQueues` we must always untag the `bh` field of an `StgBlockingQueue`.
While at first glance it might seem a sensible assumption that `bh` will
always be a blackhole and therefore never be tagged, the GC could
shortcut the indirection and put a tagged pointer into the indirection.

This blew up on aarch64-darwin with a misaligned access. `bh` pointed
to an address that always ended in 0xa. On architectures that
are a little less strict about alignment, this would have read
a garbage info table pointer, which very, very unlikely would have been equal to
`stg_BLACKHOLE_info` and therefore things accidentally worked. However,
on AArch64, the read of the info table pointer resulted in a SIGBUS due
to misaligned read.

Fixes #20093.

(cherry picked from commit 1832676a)

Sylvain Henry authored 3 years ago and Zubin committed 3 years ago

* Pthread based timer was initialized started while some other parts of
  the RTS assume it is initialized stopped, e.g. in hs_init_ghc:

     /* Start the "ticker" and profiling timer but don't start until the
     * scheduler is up. However, the ticker itself needs to be initialized
     * before the scheduler to ensure that the ticker mutex is initialized as
     * moreCapabilities will attempt to acquire it.
     */

* after a fork, don't start the timer before the IOManager is
  initialized: the timer handler (handle_tick) might call wakeUpRts to
  perform an idle GC, which calls wakeupIOManager/ioManagerWakeup

Found while debugging #18033/#20132 but I couldn't confirm if it fixes
them.

(cherry picked from commit 5f3991c7)

Matthew Pickering authored 3 years ago and Zubin committed 3 years ago

This fixes a very subtle bug in withEncodedCString where a reference would
be kept to the whole continuation until the continuation had finished
executing. This was because the call to tryFillBufferAndCall could fail,
if the buffer was already full and so the `go` helper would be
recursively called on failure which necessitated keeping a reference to
`act`.

The failure could only happen during the initial checking phase of the
function but not during the call to the continuation. Therefore the fix
is to first perform the size check, potentially recursively and then
finally calling tail calling the continuation.

In the real world, this broke writing lazy bytestrings because a
reference to the head of the bytestring would be retained in the
continuation until the whole string had been written to a file.

Fixes #20107

(cherry picked from commit 509445b5)

For some architectures the C calling convention is that any integer
shorter than 64 bits is replaced by its 64 bits representation using
sign or zero extension.

This is basically a backport of 0ac5860e
but adjusted for the 8.10 branch.

Fixes #20120

GHC GitLab CI authored 3 years ago and Zubin committed 3 years ago

Previously we branched unnecessarily on
IF_NONMOVING_WRITE_BARRIER_ENABLED on every trip through the array
barrier push loop.

(cherry picked from commit 30f233fe)

GHC GitLab CI authored 3 years ago and Zubin committed 3 years ago

Previously the code generator's logic for invoking the nonmoving write
barrier was inconsistent with the write barrier itself. Namely, the code
generator treated the header size argument as being in words whereas the
barrier expected bytes. This was the cause of #19715.

Fixes #19715.

(cherry picked from commit 221a104f)

Matthew Pickering authored 3 years ago and Zubin committed 3 years ago

(cherry picked from commit f926ecfd)

Tamar Christina authored 3 years ago and Zubin committed 3 years ago

(cherry picked from commit 74c87414)

Moritz Angermann authored 3 years ago and Zubin committed 3 years ago

We need to be careful about the sign bit for BR26 relocation
otherwise we end up encoding a large positive number and reading
back a large negative number.

(cherry picked from commit d6ab9c60)