GitLab

Since some ELF fields ran out of range to represent that many sections,
they've been extended with magic numbers that indicate that the full
value is stored in another field.

This will be necessary for GHCi with -split-sections on ELF platforms
that don't use GNU ld.

Reviewers: austin, bgamari, simonmar, erikd

Reviewed By: bgamari, simonmar, erikd

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D1357

GHC Trac Issues: #8405

Test Plan: Validate on powerpc/linux, x86_64/linux and x86_64/darwin

Reviewers: austin, bgamari, thomie

Reviewed By: thomie

Subscribers: Phyx, thomie

Differential Revision: https://phabricator.haskell.org/D1398

The code:

    uint64_t c = __sync_sub_and_fetch((uint64_t*)addr, 1);

was causing GCC to emit atomic instructions for 64 bit values which
are not available on PowerPC. However, since PowerPC only has a 32
bit address space, use of a 64 bit value is superflous.

Switching the type from `uint64_t` to `uintptr_t` should simply do
the correct thing on all 32 and 64 bit architectures.

Reviewers: austin, bgamari, simonmar

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D1399

GHC Trac Issues: #11036

Drop support for old OS X (OS X 10.2 and earlier) dynamic linking.
10.3 was released in 2003.

Test Plan: Validate on OS X and Linux.

Reviewers: bgamari, thomie, austin

Reviewed By: thomie, austin

Differential Revision: https://phabricator.haskell.org/D1393

Build system support for Cygwin was removed in b6be81b8.

Test Plan:
 - Validate on x86_64/linux
 - Cross-compile rts/RtsSymbols.c and rts/Linker.c to Windows using the
   i686-w64-mingw32-gcc and x86_64-w64-mingw32-gcc cross compilers.

Reviewers: hvr, austin, bgamari

Reviewed By: bgamari

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D1371

Test Plan: validate

Reviewers: austin, bgamari

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D1366

Pull the RtsSymbolVal typedef and rtsSyms[] array out into a separate
header and C file. No change in functionality.

Test Plan: validate

Reviewers: simonmar, austin, bgamari

Subscribers: Phyx, thomie

Differential Revision: https://phabricator.haskell.org/D1362

Summary: Spotted by @erikd

Test Plan: validate

Reviewers: austin, bgamari, erikd

Subscribers: thomie, erikd

Differential Revision: https://phabricator.haskell.org/D1345

Reviewers: bgamari, austin

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D1341

Silence the unconditional debugBelch statements recently added to HEAD
which on Windows cause debug information to always be printed.

Differential Revision: https://phabricator.haskell.org/D1338

Test Plan: Validate on x86_64, PowerPC and Arm

Reviewers: simonmar, austin, bgamari

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D1330

GHC Trac Issues: #10977

Add a missing #ifdef

Reviewed By: simonmar

Differential Revision: https://phabricator.haskell.org/D1328

On 64-bit ELF we need to link object files into the low 2GB due to the
small memory model.  Previously we would map the entire object file
using MAP_32BIT, but the object file can consist of 75% or more
symbols, which only need to be present during linking, so this is
wasteful.  In our particular application, we're already running out of
space here.

This patch changes the way we load object files on ELF platforms so
that the object is first mapped above the 2GB boundary, parsed, and
then the important sections are re-mapped into the low 2GB area.

Test Plan:
validate
(also needs testing on OS X & Windows, preferably 32 & 64 bit)

Reviewers: Phyx, trommler, bgamari, austin

Subscribers: hsyl20, thomie, bgamari

Differential Revision: https://phabricator.haskell.org/D975

Arm has two instruction sets, Arm and Thumb, and an execution mode for each.
Executing Arm code in Thumb mode or vice-versa will likely result in an
Illegal instruction exception.

Furthermore, Haskell code compiled via LLVM was generating Arm instructions
while C code compiled via GCC was generating Thumb code by default. When
these two object code types were being linked by the system linker, all was
fine, because the system linker knows how to jump and call from one
instruction set to the other.

The first problem was with GHCi's object code loader which did not know
about Thumb vs Arm. When loading an object file `StgCRun` would jump
into the loaded object which could change the mode causing a crash after
it returned. This was fixed by forcing all C code to generate Arm
instructions by passing `-marm` to GCC.

The second problem was the `mkJumpToAddr` function which was generating
Thumb instructions. Changing that to generate Arm instructions instead
results in a working GHCi on Arm.

Test Plan: validate on x86_64 and arm

Reviewers: bgamari, austin, hvr

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D1323

GHC Trac Issues: #10375

The Windows Linker has 3 main parts that this patch changes.

1) Identification and classification of sections
2) Adding of symbols to the symbols tables
3) Reallocation of sections

1.
Previously section identification used to be done on a whitelisted
basis. It was also exclusively being done based on the names of the
sections. This meant that there was a bit of a cat and mouse game
between `GCC` and `GHC`. Every time `GCC` added new sections there was a
good chance `GHC` would break. Luckily this hasn't happened much in the
past because the `GCC` versions `GHC` used were largely unchanged.

The new code instead treats all new section as `CODE` or `DATA`
sections, and changes the classifications based on the `Characteristics`
flag in the PE header. By doing so we no longer have the fragility of
changing section names. The one exception to this is the `.ctors`
section, which has no differentiating flag in the PE header, but we know
we need to treat it as initialization data.

The check to see if the sections are aligned by `4` has been removed.
The reason is that debug sections often time are `1 aligned` but do have
relocation symbols. In order to support relocations of `.debug` sections
this check needs to be gone. Crucially this assumption doesn't seem to
be in the rest of the code. We only check if there are at least 4 bytes
to realign further down the road.

2.
The second loop is iterating of all the symbols in the file and trying
to add them to the symbols table. Because the classification of the
sections we did previously are (currently) not available in this phase
we still have to exclude the sections by hand. If they don't we will
load in symbols from sections we've explicitly ignored the in # 1. This
whole part should rewritten to avoid this. But didn't want to do it in
this commit.

3.
Finally the sections are relocated. But for some reason the PE files
contain a Linux relocation constant in them `0x0011` This constant as
far as I can tell does not come from GHC (or I couldn't find where it's
being set). I believe this is probably a bug in GAS. But because the
constant is in the output we have to handle it. I am thus mapping it to
the constant I think it should be `0x0003`.

Finally, static linking *should* work, but won't. At least not if you
want to statically link `libgcc` with exceptions support. Doing so would
require you to link `libgcc` and `libstd++` but also `libmingwex`. The
problem is that `libmingwex` also defines a lot of symbols that the RTS
automatically injects into the symbol table. Presumably because they're
symbols that it needs. like `coshf`. The these symbols are not in a
section that is declared with weak symbols support. So if we ever want
to get this working, we should either a) Ask mingw to declare the
section as such, or b) treat all a imported symbols as being weak.
Though this doesn't seem like it's a good idea..

Test Plan:
Running ./validate for both x86 and x86_64

Also running the specific test case for #10672

make TESTS="T10672_x86 T10672_x64"

Reviewed By: ezyang, thomie, austin

Differential Revision: https://phabricator.haskell.org/D1244

GHC Trac Issues: #9907, #10672, #10563

This patch does a few things

- Moved GHC x86 to MinGW-w64 (Using Awson's patch)
- Moves Both GHCs to MSYS2 toolchains
- Completely removes the dependencies on the git tarball repo
  - Downloads only the required tarball for the architecture for
    which we are building
  - Downloads the perl tarball is missing as well
  - Fixed a few bugs in the linker to fix tests on Windows

The links currently point to repo.msys2.org and GitHub, it might be
more desirable to mirror them on
http://downloads.haskell.org/~ghc/mingw/ as with the previous patch
attempt.

For more details on what the MSYS2 packages I include see #10726
(Awson's comment). but it should contain all we need
and no python or fortran, which makes the uncompressed tar a 1-2
hundreds mb smaller.

The `GCC 5.2.0` in the package supports `libgcc` as a shared library,
this is a problem since
when compiling with -shared the produced dll now has a dependency on
`libgcc_s_sjlj-1.dll`.
To solve this the flag `-static-libgcc` is now being used for all GCC
calls on windows.

Test Plan:
./validate was ran both on x86 and x86_64 windows and compared against
the baseline.

A few test were failing due to Ld no longer being noisy. These were
updated.

The changes to the configure script *should* be validated by the build
bots for the other platforms before landing

Reviewers: simonmar, awson, bgamari, austin, thomie

Reviewed By: thomie

Subscribers: #ghc_windows_task_force, thomie, awson

Differential Revision: https://phabricator.haskell.org/D1123

GHC Trac Issues: #10726, #9014, #9218, #10435

Summary:
The patch disables check for .init_array section on OpenBSD.
It is provided in OpenBSD ports tree and was done by Matthias Kilian.

Reviewers: austin

Subscribers: thomie, bgamari

Differential Revision: https://phabricator.haskell.org/D1023

Commit a93ab43a
enabled support for proper PIC relocations from
assembler.

Commit adds support for relocations of type:
    R_PPC_REL16_HI
    R_PPC_REL16_HA
    R_PPC_REL16_LO
    R_PPC_PLTREL24

They are used only when GHC is built in
    DYNAMIC_GHC_PROGRAMS = NO
mode.

Verified by running the following test:

    // cat a.c
    #include <stdio.h>

    void ffi_a_hello (int i) {
        fprintf (stderr, "WEEEEEEEE: i=%d\n", i);
    }

    -- cat A.hs
    {-# LANGUAGE ForeignFunctionInterface #-}
    module A where

    import Foreign.C

    foreign import ccall "ffi_a_hello" a :: CInt -> IO ()

    # ghc -fPIC -c a.c -fforce-recomp
    # ghc -fPIC -c A.hs -fforce-recomp
    # ghc --interactive ./a.o A
    ...
    *A> a 42
    WEEEEEEEE: i=42
See gory details in Trac #10402.
Signed-off-by: Colin Watson <cjwatson@debian.org>
Signed-off-by: Sergei Trofimovich <siarheit@google.com>

Reviewed By: bgamari, austin

Differential Revision: https://phabricator.haskell.org/D996

GHC Trac Issues: #10402

In a situaion where we have some statically-linked code and we want to
load and unload a series of objects, we need the CAFs in the
statically-linked code to be retained indefinitely, while the CAFs in
the dynamically-linked code should be GC'd as normal, so that we can
detect when the code is unloadable.  This was wrong before - we GC'd
CAFs in the static code, leading to a crash in the rare case where we
use a CAF, GC it, and then load a new object that uses it again.

I also did some tidy up: RtsConfig now has a field keep_cafs to
indicate whether we want CAFs to be retained in static code.

The patch is a last major piece to make unregisterised GHC build
under GCC's link-time optimizer.

Before the patch we imported everything external as functions.
Now we distinct between global variables and functions.

The difference is crucial on ia64 and a complement to fixes:
    > d82f5925
    > CMM: add a mechanism to import C .data labels
    > e18525fa
    > pprC: declare extern cmm primitives as functions, not data
Signed-off-by: Sergei Trofimovich <siarheit@google.com>

Reviewed By: austin

Differential Revision: https://phabricator.haskell.org/D797

Summary:
Hooks rely on static linking semantics, and are broken by -Bsymbolic
which we need when using dynamic linking.

Test Plan: Built it

Reviewers: austin, hvr, tibbe

Differential Revision: https://phabricator.haskell.org/D8

This fixes more testsuite failures on Windows.
Signed-off-by: Austin Seipp <austin@well-typed.com>

Summary: Currently the linker tries to see if it understands/knows every section in the PE file before it continues. If it encounters a section it doesn't know about it errors out. Every time there's a change in MinGW compiler that adds a new section to the PE file this will break the ghc linker. The new sections don't need to be understood by `ghc` to continue so instead of erroring out the section is just ignored. When running with `-debug` the sections that are ignored will be printed.

Test Plan:
See the file `ghcilinkerbug.zip` in #9907.

 1) unzip file content.
 2) open examplecpp.cabal and change base <4.8 to <4.9.
 3) execute cabal file with cabal repl.

Applying the patch makes `cabal repl` in step 3) work.

Note that the file will fail on a `___mingw_vprintf` not being found. This is because of the `cc-options` specifying `-std=c++0x`, which will also require `libmingwex.a` to be linked in but wasn't specified in the cabal file. To fix this, remove the `cc-options` which defaults to c99.

Reviewers: austin

Reviewed By: austin

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D671

GHC Trac Issues: #9907, #7103, #10051, #7056, #8546

Summary:
A mutex is used to protect the SPT.

unsafeLookupStaticPtr and staticPtrKeys in GHC.StaticPtr are made
monadic.

SPT entries are removed in a destructor function of modules.
Authored-by: Facundo Domínguez <facundo.dominguez@tweag.io>
Authored-by: Alexander Vershilov <alexander.vershilov@tweag.io>

Test Plan: ./validate

Reviewers: austin, simonpj, hvr

Subscribers: carter, thomie, qnikst, mboes

Differential Revision: https://phabricator.haskell.org/D587

GHC Trac Issues: #9878

Summary:
As proposed in [1], this extension introduces a new syntactic form
`static e`, where `e :: a` can be any closed expression. The static form
produces a value of type `StaticPtr a`, which works as a reference that
programs can "dereference" to get the value of `e` back. References are
like `Ptr`s, except that they are stable across invocations of a
program.

The relevant wiki pages are [2, 3], which describe the motivation/ideas
and implementation plan respectively.

[1] Jeff Epstein, Andrew P. Black, and Simon Peyton-Jones. Towards
Haskell in the cloud. SIGPLAN Not., 46(12):118–129, September 2011. ISSN
0362-1340.
[2] https://ghc.haskell.org/trac/ghc/wiki/StaticPointers
[3] https://ghc.haskell.org/trac/ghc/wiki/StaticPointers/ImplementationPlanAuthored-by: Facundo Domínguez <facundo.dominguez@tweag.io>
Authored-by: Mathieu Boespflug <m@tweag.io>
Authored-by: Alexander Vershilov <alexander.vershilov@tweag.io>

Test Plan: `./validate`

Reviewers: hvr, simonmar, simonpj, austin

Reviewed By: simonpj, austin

Subscribers: qnikst, bgamari, mboes, carter, thomie, goldfire

Differential Revision: https://phabricator.haskell.org/D550

GHC Trac Issues: #7015

This reverts commit 7932b2ad.

Also includes a fix for the segfaults on Windows caused by the original
version of this patch.

This reverts commit 4b51194d.

This reverts commit b5e8b3b1.

I reverted it because one of these two patches
  9e6e4796 Add purgeObj()
  b5e8b3b1 Make the linker API thread-safe

causes a seg-fault on Windows.  The seg-fault happens immediately
the linker is invoked, in ghci or in Template Haskell.

I believe that it is the "linker API thread-safe" commit that causes
the seg-fault; it happens even if the "purgeObj" commit alone is
reverted.  But since the two patches mess with the same code, to
revert the "linker API" patch I had revert both.

This reverts commit 9e6e4796.

I reverted it because one of these two patches
  9e6e4796 Add purgeObj()
  b5e8b3b1 Make the linker API thread-safe

causes a seg-fault on Windows.  The seg-fault happens immediately
the linker is invoked, in ghci or in Template Haskell.

I believe that it is the "linker API thread-safe" commit that causes
the seg-fault; it happens even if the "purgeObj" commit alone is
reverted.  But since the two patches mess with the same code, to
revert the "linker API" patch I had revert both.

Summary:
In a statically linked GHCi symbol `environ` resolves to NULL when
called from a Haskell script.

When resolving symbols in a Haskell script we need to search the
executable program and its dependent (DT_NEEDED) shared libraries
first and then search the loaded libraries.

We want to be able to override functions in loaded libraries later.
Libraries must be opened with local scope (RTLD_LOCAL) and not global.
The latter adds all symbols to the executable program's symbols where
they are then searched in loading order. We want reverse loading order.

When libraries are loaded with local scope the dynamic linker
cannot use symbols in that library when resolving the dependencies
in another shared library. This changes the way files compiled to
object code must be linked into temporary shared libraries. We link
with the last temporary shared library created so far if it exists.
Since each temporary shared library is linked to the previous temporary
shared library the dynamic linker finds the latest definition of a
symbol by following the dependency chain.

See also Note [RTLD_LOCAL] for a summary of the problem and solution.

Cherry-picked commit 2f8b4c

Changed linker argument ordering

On some ELF systems GNU ld (and others?) default to
--as-needed and the order of libraries in the link
matters.

The last temporary shared library, must appear
before all other libraries. Switching the position
of extra_ld_inputs and lib_path_objs does that.

Fixes #8935 and #9186

Reviewers: austin, hvr, rwbarton, simonmar

Reviewed By: simonmar

Subscribers: thomie, carter, simonmar

Differential Revision: https://phabricator.haskell.org/D349

GHC Trac Issues: #8935, #9186, #9480

This allows us to replace an object without actually unloading the old
object, which is necessary when we know we have references to the old
object so it can't be completely unloaded.  Using unloadObj() would
cause the GC (CheckUnload) to repeatedly and fruitlessly try to unload
the old object.

We used to be able to rely on the client to use the API in a
single-threaded way, but now that the GC calls into the linker to
unload objects this isn't a safe assumption.

This reverts commit f0fcc41d.

New changes: now works on 32-bit platforms too.  I added some basic
support for 64-bit subtraction and comparison operations to the x86
NCG.

Summary:
swprintf has different signatures in mingw32, where it does not include the
buffer size, and in mingw-w64, where it does. That of course breaks the code
as mingw-w64 treats the pointer to the format string as a size_t.

snwprintf is available in both environments and is consistent, so use that
instead.

Reviewers: simonmar, austin

Reviewed By: austin

Subscribers: #ghc_windows_task_force, thomie, carter, simonmar

Differential Revision: https://phabricator.haskell.org/D372

GHC Trac Issues: #9726

The warning was breaking validate.sh runs due to -Wall.

Reviewers: austin

Reviewed By: austin

Subscribers: #ghc_windows_task_force, thomie, carter, simonmar

Differential Revision: https://phabricator.haskell.org/D400

Authored-by: Simon Marlow <marlowsd@gmail.com>
Signed-off-by: Austin Seipp <austin@well-typed.com>

This reverts commit 35672072.

Conflicts:
	compiler/main/DriverPipeline.hs

This reverts commit 2fc0c6cf.