Ryan Scott authored 1 year ago and Zubin committed 1 year ago

This partially reverts some of the changes in !9475 to make `base` and
`ghc-prim` depend on the `mingwex` library on Windows. It also restores the
RTS's stubs for `mingwex`-specific symbols such as `_lock_file`.

This is done because the C runtime provides `libmingwex` nowadays, and
moreoever, not linking against `mingwex` requires downstream users to link
against it explicitly in difficult-to-predict circumstances. Better to always
link against `mingwex` and prevent users from having to do the guesswork
themselves.

See ghc/ghc!10360 (comment 495873) for
the discussion that led to this.

(cherry picked from commit 2b1a4abe)
(cherry picked from commit ff73fc3f)

Metric Decrease:
  T13035
  T15164
  T1969
  T9961
  WWRec
  T12707
  T13379

Metric Increase:
  T17123

Matthew Pickering authored 2 years ago and Zubin committed 1 year ago

This mirrors some existing logic for the bootstrap_target which
influences how TargetPlatform is set.

As described on #21970 not storing this led to `LlvmTarget` being set incorrectly
and hence the wrong `--target` flag being passed to the C compiler.

Towards #21970

(cherry picked from commit 64286132)
(cherry picked from commit 48a9e688)

Ryan Scott authored 2 years ago and Zubin committed 1 year ago

The clang based toolchain uses ucrt as its math library
and so mingwex is no longer needed.  In fact using mingwex
will cause incompatibilities as the default routines in both
have differing ULPs and string formatting modifiers.

```
$ LIBRARY_PATH=/mingw64/lib ghc/_build/stage1/bin/ghc Bug.hs -fforce-recomp && ./Bug.exe
[1 of 2] Compiling Main             ( Bug.hs, Bug.o )
ghc.exe:  | C:\Users\winferno\Software\ghc\_build\stage1\lib\x86_64-windows-ghc-9.5.20220908\base-4.17.0.0\libHSbase-4.17.0.0.a: unknown symbol `__imp___p__environ'
ghc.exe:  | C:\Users\winferno\Software\ghc\_build\stage1\lib\x86_64-windows-ghc-9.5.20220908\base-4.17.0.0\libHSbase-4.17.0.0.a: unknown symbol `__hscore_get_errno'
ghc.exe:  | C:\Users\winferno\Software\ghc\_build\stage1\lib\x86_64-windows-ghc-9.5.20220908\base-4.17.0.0\libHSbase-4.17.0.0.a: unknown symbol `base_ForeignziCziError_errnoToIOError_info'
ghc.exe:  | C:\Users\winferno\Software\ghc\_build\stage1\lib\x86_64-windows-ghc-9.5.20220908\base-4.17.0.0\libHSbase-4.17.0.0.a: unknown symbol `base_GHCziWindows_failIf2_closure'
ghc.exe:  | C:\Users\winferno\Software\ghc\_build\stage1\lib\x86_64-windows-ghc-9.5.20220908\base-4.17.0.0\libHSbase-4.17.0.0.a: unknown symbol `base_GHCziIOziEncodingziCodePageziAPI_mkCodePageEncoding_info'
ghc.exe:  | C:\Users\winferno\Software\ghc\_build\stage1\lib\x86_64-windows-ghc-9.5.20220908\base-4.17.0.0\libHSbase-4.17.0.0.a: unknown symbol `base_GHCziIOziEncodingziCodePage_currentCodePage_closure'
ghc.exe:  | C:\Users\winferno\Software\ghc\_build\stage1\lib\x86_64-windows-ghc-9.5.20220908\base-4.17.0.0\libHSbase-4.17.0.0.a: unknown symbol `base_GHCziIOziEncoding_getForeignEncoding_closure'
ghc.exe:  | C:\Users\winferno\Software\ghc\_build\stage1\lib\x86_64-windows-ghc-9.5.20220908\base-4.17.0.0\libHSbase-4.17.0.0.a: unknown symbol `base_ForeignziCziString_withCStringLen1_info'
ghc.exe:  | C:\Users\winferno\Software\ghc\_build\stage1\lib\x86_64-windows-ghc-9.5.20220908\base-4.17.0.0\libHSbase-4.17.0.0.a: unknown symbol `base_GHCziIOziHandleziInternals_zdwflushCharReadBuffer_info'
ghc.exe:  | C:\Users\winferno\Software\ghc\_build\stage1\lib\x86_64-windows-ghc-9.5.20220908\base-4.17.0.0\libHSbase-4.17.0.0.a: unknown symbol `base_GHCziIOziHandleziText_hGetBuf1_info'
ghc.exe:  | C:\Users\winferno\Software\ghc\_build\stage1\lib\x86_64-windows-ghc-9.5.20220908\base-4.17.0.0\libHSbase-4.17.0.0.a: unknown symbol `base_GHCziFingerprint_fingerprintString_closure'
ghc.exe:  | C:\Users\winferno\Software\ghc\_build\stage1\lib\x86_64-windows-ghc-9.5.20220908\base-4.17.0.0\libHSbase-4.17.0.0.a: unknown symbol `base_DataziTypeableziInternal_mkTrCon_closure'
ghc.exe:  | C:\Users\winferno\Software\ghc\_build\stage1\lib\x86_64-windows-ghc-9.5.20220908\base-4.17.0.0\libHSbase-4.17.0.0.a: unknown symbol `base_GHCziException_errorCallWithCallStackException_closure'
ghc.exe:  | C:\Users\winferno\Software\ghc\_build\stage1\lib\x86_64-windows-ghc-9.5.20220908\base-4.17.0.0\libHSbase-4.17.0.0.a: unknown symbol `base_GHCziErr_error_info'
ghc.exe:  | C:\Users\winferno\Software\ghc\_build\stage1\lib\x86_64-windows-ghc-9.5.20220908\template-haskell-2.19.0.0\libHStemplate-haskell-2.19.0.0.a: unknown symbol `base_DataziMaybe_fromJust1_info'
ghc.exe:  | C:\Users\winferno\Software\ghc\_build\stage1\lib\x86_64-windows-ghc-9.5.20220908\template-haskell-2.19.0.0\libHStemplate-haskell-2.19.0.0.a: unknown symbol `templatezmhaskell_LanguageziHaskellziTHziSyntax_IntPrimL_con_info'
ghc.exe: ^^ Could not load 'templatezmhaskell_LanguageziHaskellziTHziLibziInternal_stringL_closure', dependency unresolved. See top entry above.

<no location info>: error:

GHC.ByteCode.Linker.lookupCE
During interactive linking, GHCi couldn't find the following symbol:
  templatezmhaskell_LanguageziHaskellziTHziLibziInternal_stringL_closure
This may be due to you not asking GHCi to load extra object files,
archives or DLLs needed by your current session.  Restart GHCi, specifying
the missing library using the -L/path/to/object/dir and -lmissinglibname
flags, or simply by naming the relevant files on the GHCi command line.
Alternatively, this link failure might indicate a bug in GHCi.
If you suspect the latter, please report this as a GHC bug:
  https://www.haskell.org/ghc/reportabug
```

(cherry picked from commit de1d1512)

sheaf authored 2 years ago and Zubin committed 1 year ago

We now always merge .a archives when ar supports -L.
This change is necessary in order to bootstrap GHC using GHC 9.4
on Windows, as nested archives aren't supported.
Not doing so triggered bug #21990 when trying to use the Win32
package, with errors such as:

  Not a x86_64 PE+ file.
  Unknown COFF 4 type in getHeaderInfo.

  ld.lld: error: undefined symbol: Win32zm2zi12zi0zi0_SystemziWin32ziConsoleziCtrlHandler_withConsoleCtrlHandler1_info

We have to be careful about which ar is meant: in stage 0, the check
should be done on the system ar (system-ar in system.config).

(cherry picked from commit 545ff490)

Matthew Pickering authored 2 years ago and Zubin committed 1 year ago

Recent versions of MacOS use a version of ld where `-fixup_chains` is on by default.
This is incompatible with our usage of `-undefined dynamic_lookup`. Therefore we
explicitly disable `fixup-chains` by passing `-no_fixup_chains` to the linker on
darwin. This results in a warning of the form:

ld: warning: -undefined dynamic_lookup may not work with chained fixups

The manual explains the incompatible nature of these two flags:

     -undefined treatment
             Specifies how undefined symbols are to be treated. Options are: error, warning,
             suppress, or dynamic_lookup.  The default is error. Note: dynamic_lookup that
             depends on lazy binding will not work with chained fixups.

A relevant ticket is #22429

Here are also a few other links which are relevant to the issue:

Official comment: https://developer.apple.com/forums/thread/719961

More relevant links:

https://openradar.appspot.com/radar?id=5536824084660224

https://github.com/python/cpython/issues/97524

Note in release notes: https://developer.apple.com/documentation/xcode-release-notes/xcode-13-releas    e-notes

(cherry picked from commit 8c0ea25f)

Ben Gamari authored 2 years ago and Matthew Pickering committed 2 years ago

Since 6be2c5a7 we would probe for LD
rather early in `configure`. However, it turns out that this breaks
`configure`'s `ld`-override logic, which assumes that `LD` was set by
the user and aborts.

Fixes #21778.

(cherry picked from commit 995ea44d)

Ben Gamari authored 2 years ago and Matthew Pickering committed 2 years ago

Here we introduce a new "virtual" package into the initial package
database, `system-cxx-std-lib`. This gives users a convenient, platform
agnostic way to link against C++ libraries, addressing #20010.

Fixes #20010.

(cherry picked from commit 0ef249aa)

Ben Gamari authored 2 years ago and Matthew Pickering committed 2 years ago

Here we introduce proper support for compilation of C++ objects. This
includes:

 * logic in `configure` to detect the C++ toolchain and propagating this
   information into the `settings` file
 * logic in the driver to use the C++ toolchain when compiling C++
   sources

(cherry picked from commit fb579e15)

Ben Gamari authored 2 years ago and Matthew Pickering committed 2 years ago

Here we deprecate the eventlogging RTS ways and instead enable eventlog
support in the remaining ways. This simplifies packaging and reduces GHC
compilation times (as we can eliminate two whole compilations of the RTS)
while simplifying the end-user story. The trade-off is a small increase
in binary sizes in the case that the user does not want eventlogging
support, but we think that this is a fine trade-off.

This also revealed a latent RTS bug: some files which included `Cmm.h`
also assumed that it defined various macros which were in fact defined
by `Config.h`, which `Cmm.h` did not include. Fixing this in turn
revealed that `StgMiscClosures.cmm` failed to import various spinlock
statistics counters, as evidenced by the failed unregisterised build.

Closes #18948.

(cherry picked from commit ee11d043)

Matthew Pickering authored 2 years ago and Ben Gamari committed 2 years ago

In the make bindists we generate documentation in docs/ghc-<VER> but the
hadrian bindists generate docs/ghc/ so the path to the GHC API docs was
wrong in the index.html file.

Rather than make the hadrian and make bindists the same it was easier to
assume that if you're using the mkDocs script that you're using hadrian
bindists.

Fixes #21509

(cherry picked from commit 68d1ea5f)

We previously only checked the stage 1/2 compiler
for --target support. We got away with this for quite a while but it
eventually caught up with us in #21579, where `bytestring`'s new NEON
implementation was unbuildable on Darwin due to Rosetta's seemingly
random logic for determining which executable image to execute. This
lead to a confusing failure to build `bytestring`'s cbits, when `clang`
tried to compile NEON builtins while targetting x86-64.

Fix this by checking CC_STAGE0 for --target support.

Fixes #21579.

(cherry picked from commit 222eb83e)

It appears that the version of sphinx shipped on CentOS 7
reports a version string of `Sphinx v1...`. Accept the `v`.

Pull forward minimum version to match 9.2.

* Bump win32-tarballs to 0.7
* Move Windows toolchain autoconf logic into separate file
* Use clang and LLVM utilities as described in #21019
* Disable object merging as lld doesn't support -r
* Drop --oformat=pe-bigobj-x86-64 arguments from ld flags as LLD detects
  that the output is large on its own.
* Drop gcc wrapper since Clang finds its root fine on its own.

Ben Gamari authored 2 years ago and Marge Bot committed 2 years ago

Since object files may in fact be archive files, we must ensure that
their contents are merged rather than constructing an
archive-of-an-archive.

See #21068.

Cheng Shao authored 3 years ago and Marge Bot committed 2 years ago

LLVM 13.0.0 is released in Oct 2021, and latest head validates against
LLVM 13 just fine if LlvmMaxVersion is bumped.

Zubin authored 3 years ago and Matthew Pickering committed 3 years ago

This patch allows ghc and its dependencies to be built using a normal
invocation of cabal-install. Each componenent which relied on generated
files or additional configuration now has a Setup.hs file.

There are also various fixes to the cabal files to satisfy
cabal-install.

There is a new hadrian command which will build a stage2 compiler and
then a stage3 compiler by using cabal.

```
./hadrian/build build-cabal
```

There is also a new CI job which tests running this command.

For the 9.4 release we will upload all the dependent executables to
hackage and then end users will be free to build GHC and GHC executables
via cabal.

There are still some unresolved questions about how to ensure soundness
when loading plugins into a reinstalled GHC (#20742) which will be
tighted up in due course.

Fixes #19896

PHO authored 3 years ago and Marge Bot committed 3 years ago

Like -threaded, stage0 isn't guaranteed to have an event-logging RTS.

Ben Gamari authored 3 years ago and Marge Bot committed 3 years ago

Previously we would unconditionally provide a declaration for `environ`,
even if `<unistd.h>` already provided one. This would result in
`-Werror` builds failing on some platforms.

Also `#include <unistd.h>` to ensure that the declaration is visible.

Fixes #20861.

Zubin authored 3 years ago and Marge Bot committed 3 years ago

We use `git ls-files` to get the list of files to include in the source distribution.

Also implements the `-testsuite` and `-extra-tarballs` distributions.

Matthew Pickering authored 3 years ago and Marge Bot committed 3 years ago

As noted in #20794 the parts of libiserv and iserv-proxy depend on
network, therefore are never built nor tested during CI.

Due to this iserv-proxy had bitrotted due to the bound on bytestring
being out of date.

Given we don't test this code it seems undesirable to distribute it.
Therefore, it's removed and an external maintainer can be responsible
for testing it (via head.hackage if desired).

Fixes #20794

PHO authored 3 years ago and Marge Bot committed 3 years ago

Not all platforms have per-thread locales. NetBSD doesn't have uselocale() in particular. Using setlocale() is of course not a safe thing to do, but it would be better than no GHC at all.

Get rid of `USE_INPLACE_MINGW_TOOLCHAIN` and use a settings file entry
instead.

The CPP setting was originally introduced in f065b6b0.

John Ericson authored 3 years ago and Marge Bot committed 3 years ago

This is, rather unintuitively, part of the goal of making the packages
that make of the GHC distribution more freestanding. `ghcversion.h` is
very simple, so we easily can move it out of the main build systems
(make and Hadrian). By doing so, the RTS becomes less of a special case
to those build systems as the header, already existing in the source
tree, appears like any other.

We could do this with the upcomming RTS configure, but it hardly matters
because there is nothing platform-specific here, it is just versioning
information like the other files the top-level configure can be
responsible for.