(cherry picked from commit 3655c3fd)
(cherry picked from commit b6d5cce5)

Bryan R authored 1 year ago and Cheng Shao committed 1 month ago

(cherry picked from commit 069729d3)

(cherry picked from commit 42826a89)
(cherry picked from commit 3ea15740)
(cherry picked from commit 30d4f0ce)

This patch bumps macOS minimum SDK version to 11.0 for x86_64-darwin
to align it with aarch64-darwin. This allows us to get rid of the
horrible -Wl,-U,___darwin_check_fd_set_overflow hack, which is causing
linker warnings and testsuite failures on macOS 15. Fixes #25504.

(cherry picked from commit 88c4fe1d)
(cherry picked from commit 5c7e7695)
(cherry picked from commit 77c8c3eb)

This patch makes test-bootstrap related ci jobs only depend on
hadrian-ghc-in-ghci job to finish, consistent with other jobs in the
full-build stage generated by gen_ci.hs. This allows the jobs to be
spawned earlier and improve overall pipeline parallelism.

(cherry picked from commit e684c406)
(cherry picked from commit 4884eb08)
(cherry picked from commit f3a9d4ec)

This patch improves the compile_flags.txt config used to power clangd
for the rts C codebase. The flags in the file are sampled & deduped
from a real stage1 build with clang-19 and vastly improves the IDE
accuracy when hacking the rts.

For maximum code coverage under the default settings,
compile_flags.txt defaults to threaded+profiled+dynamic+debug way.
This does not mean profdyn needs to be actually built in _build/stage1
for IDE to work. To activate IDE for other RTS ways, simply remove one
of the -D flags at the end of compile_flags.txt and restart clangd.

(cherry picked from commit 59e0a770)
(cherry picked from commit 222a08ae)
(cherry picked from commit 039e2086)

This patch removes the obsolete cross-port script in the tree. The
script was based on the legacy Make build system which has been pruned
from the tree long ago. For hadrian, proper support for two-stage
bootstrapping onto a new unsupported platform is a work in progress in
!11444.

(cherry picked from commit 00d551bf)
(cherry picked from commit 329a26da)
(cherry picked from commit 3f3be277)

The executable wrappers are handled by hadrian and bindist Makefile.
The various .wrapper scripts in the tree are unused since removal of
Make build system, so this patch removes them all.

(cherry picked from commit 9ede97f3)
(cherry picked from commit 737e9e0b)
(cherry picked from commit c57730cb)

This patch removes unused `USE_REPORT_PRELUDE` code paths from the
tree. They have been present since the first git revision
4fb94ae5, and might have been useful
for debugging purposes many years ago, but these code paths are never
actually built. Removing these ease maintenance of relevant modules in
the future, and also allows us to get rid of `CPP` extension in those
modules as a nice byproduct.

(cherry picked from commit 573cad4b)

This patch fixes foreign stub handling logic in `hscParsedDecls`.
Previously foreign stubs were simply ignored here, so any feature that
involve foreign stubs would not work in ghci (e.g. CApiFFI). The patch
reuses `generateByteCode` logic and eliminates a large chunk of
duplicate logic that implements Core to bytecode generation pipeline
here. Fixes #25414.

(cherry picked from commit 677e3aa56e905524071fc9717a88ad2cd1bc2951)
(cherry picked from commit 0db7ace8)

This commit adds T25414 test case to demonstrate #25414. It is marked
as broken and will be fixed by the next commit.

(cherry picked from commit b1eed26f)

This patch removes unused `hscDecls`/`hscDeclsWithLocation` functions
from the compiler, to reduce maintenance burden when doing
refactorings related to ghci.

(cherry picked from commit e3496ef6)

This commit fixes an oversight of hadrian wrapper generation logic:
when doing cross compilation, `wrapper` is called on executable names
with cross prefix, therefore we must use `isSuffixOf` when matching to
take the cross prefix into account. Also add missing cross prefix to
ghci wrapper content and fix hsc2hs wrapper logic.

(cherry picked from commit b57c2174cf8b4804d400c7db774565b2be4ff493)

This patch fixes an unnoticed undefined behavior in the bytecode
interpreter. It can be caught by building `rts/Interpreter.c` with
`-fsanitize=pointer-overflow`, the warning message is something like:

```
rts/Interpreter.c:1369:13: runtime error: addition of unsigned offset to 0x004200197660 overflowed to 0x004200197658
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior rts/Interpreter.c:1369:13
rts/Interpreter.c:1265:13: runtime error: addition of unsigned offset to 0x004200197660 overflowed to 0x004200197658
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior rts/Interpreter.c:1265:13
rts/Interpreter.c:1645:13: runtime error: addition of unsigned offset to 0x0042000b22f8 overflowed to 0x0042000b22f0
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior rts/Interpreter.c:1645:13
```

Whenever we do something like `SpW(-1)`, the negative argument is
implicitly converted to an unsigned integer type and causes pointer
arithmetic overflow. It happens to be harmless for most targets since
overflowing would wrap the result to desired value, but it's still
coincidental and undefined behavior. Furthermore, it causes real
damage to the wasm backend, given clang-20 will emit invalid wasm code
that crashes at run-time for this kind of C code! (see
https://github.com/llvm/llvm-project/issues/108770)

The fix here is adding some explicit casts to ensure we always use the
signed `ptrdiff_t` type as right hand operand of pointer arithmetic.

(cherry picked from commit 5bcfefd5)
(cherry picked from commit 990d79a0)

(cherry picked from commit 94673d41)
(cherry picked from commit f13a2942)

This commit implements the config.cross field in the testsuite driver.
It comes from the "cross compiling" ghc info field for both
in-tree/out-of-tree GHC, and is an accurate predicate of whether we're
cross-compiling or not (compared to the precense of target emulator),
and is useful to implement predicates to assert the precense of
internal interpreter (only available on non-cross GHC) for tests that
do require it (e.g. plugins).

(cherry picked from commit 2956a3f7)
(cherry picked from commit 5500e1b3)

This commit bumps ci-image revision to use updated wasm toolchain, and
use host_fully_static instead of fully_static for wasm jobs so to
ensure wasm shared libraries can be properly built.

(cherry picked from commit 5043507c)
(cherry picked from commit 0ae9f0d5)

Matthew Pickering authored 8 months ago and Cheng Shao committed 1 month ago

alpine 3.20 was recently released and uses a new python and sphinx
toolchain which could be useful to test.

(cherry picked from commit 7bcda869)
(cherry picked from commit eac492c9)

This commit adds the host_fully_static flavour transformer to hadrian,
which ensures stage0 is fully statically linked while still permitting
stage1 libdir to contain shared libraries. This is intended to be used
by the wasm backend to build portable linux bindists that contain wasm
shared libraries.

(cherry picked from commit b45080a3)
(cherry picked from commit 91c0dbd4)

This commit disable the internal-interpreter flag of ghc library when
cross compiling, only external interpreter works in such cases.

(cherry picked from commit f232c872)
(cherry picked from commit 4e7d0cee)

This commit fixes conflicting StgRun definition when building dynamic
ways of RTS for wasm in unregisterised mode.

(cherry picked from commit bef94bde)
(cherry picked from commit 6f844296)

This commit wraps the predefined GlobalRegs in Wasm.S under a CPP
guard to prevent building for PIC mode. When building dynamic ways of
RTS, the wasm globals that represent STG GlobalRegs will be created
and supplied by dyld.mjs. The current wasm dylink convention doesn't
properly support exporting relocatable wasm globals at all, any wasm
global exported by a .so is assumed to be a GOT.mem entry.

(cherry picked from commit 98a32ec5)
(cherry picked from commit cc67ef51)

This commit drops interpretBCO support from non dynamic rts ways on
wasm. The bytecode interpreter is only useful when the RTS linker also
works, and on wasm it only works for dynamic ways anyway. An
additional benefit of dropping interpretBCO is reduction in code size
of linked wasm modules, especially since interpretBCO references
ffi_call which is an auto-generated large function in libffi-wasm and
unused by most user applications.

(cherry picked from commit 90a35c41)
(cherry picked from commit 7a16cf7c)

This patch avoids using mmap() to allocate executable memory for
mkConInfoTable on platforms without tables-next-to-code, see added
comment for explanation.

(cherry picked from commit 839ac52e)
(cherry picked from commit e54e3335)

This patch drops obsolete libffi Makefile from the tree, given it's
completely unused since removal of make build system in !7094.

(cherry picked from commit 3fe843c7)
(cherry picked from commit 81ddb1e7)

This patch removes the ghciWithDebugger field from flavour config
since it's actually not used anywhere.

(cherry picked from commit c6e5fd3d)
(cherry picked from commit 022d7565)

a.out is a configure script byproduct. It was mistakenly checked into
the tree in !13118. This patch removes it, and include it in
.gitignore to prevent a similar error in the future.

(cherry picked from commit f0408eeb)
(cherry picked from commit 8abb0f19)

This commit fixes isMinTTY.h casing in isMinTTY.c that's compiled for
Windows targets. While this looks harmless given Windows filesystems
are case-insensitive by default, it does cause a compilation warning
with recent versions of clang, so we might as well fix the casing:

```
driver\ghci\isMinTTY.c:10:10: error:
     warning: non-portable path to file '"isMinTTY.h"'; specified path differs in case from file name on disk [-Wnonportable-include-path]
   |
10 | #include "isMINTTY.h"
   |          ^

 #include "isMINTTY.h"
         ^~~~~~~~~~~~
         "isMinTTY.h"
1 warning generated.
```

(cherry picked from commit 3a145315)
(cherry picked from commit 7bcc3dc6)

This patch removes the -fasm flag from test ways, except ways like
optasm that explicitly state they are meant to be compiled with NCG
backend. Most test ways should use the default codegen backend, and
the precense of -fasm can cause stderr mismatches like this when GHC
is configured with the unregisterised backend:

```
--- /dev/null
+++ /tmp/ghctest-3hydwldj/test   spaces/testsuite/tests/profiling/should_compile/prof-late-cc.run/prof-late-cc.comp.stderr.normalised
@@ -0,0 +1,2 @@
+when making flags consistent: warning: [GHC-74335] [-Winconsistent-flags (in -Wdefault)]
+    Target platform uses unregisterised ABI, so compiling via C
*** unexpected failure for prof-late-cc(prof_no_auto)
```

This has been breaking the wasm unreg nightly job since !12595 landed.

(cherry picked from commit 88488847)
(cherry picked from commit f603ac9c)

(cherry picked from commit 64fba310)
(cherry picked from commit b35707a9)

This patch changes mblock size to page size on wasm. It allows us to
simplify our wasi-libc fork, makes it much easier to test third party
libc allocators like emmalloc/mimalloc, as well as experimenting with
threaded RTS in wasm.

(cherry picked from commit 558353f4)
(cherry picked from commit f3ea9fb8)

This patch removes pre-C11 legacy code paths related to
INLINE_HEADER/STATIC_INLINE/EXTERN_INLINE macros, ensure EXTERN_INLINE
is treated as static inline in most cases (fixes #24945), and also
corrects the comments accordingly.

(cherry picked from commit 35a64220)
(cherry picked from commit 241f401d)

objc-hi/objcxx-hi should be skipped when cross compiling. The existing
opsys('darwin') predicate only asserts the host system is darwin but
tells us nothing about the target, hence the oversight.

(cherry picked from commit 595c0894)
(cherry picked from commit 5273d3d0)

(cherry picked from commit 2eee65e1)
(cherry picked from commit 3f7b718f)

(cherry picked from commit ae50a8eb)
(cherry picked from commit b7e93bc9)

This patch makes hadrian build in-tree GMP with the
--enable-alloca=malloc-notreentrant configure option. We will only
need malloc-reentrant when we have threaded RTS and SMP support on
wasm32, which will take some time to happen, before which we should
use malloc-notreentrant to avoid undesired runtime overhead.

(cherry picked from commit 06277d56)
(cherry picked from commit 0f9a8aaf)

This patch disables in-tree GMP FFT code paths for wasm32 target in
order to give up some performance of multiplying very large operands
in exchange for reduced code size.

(cherry picked from commit 1a32f828)
(cherry picked from commit 097ea6e2)

This patch disables PIC for in-tree GMP on wasm32 target. Enabling PIC
unconditionally adds undesired code size and runtime overhead for
wasm32.

(cherry picked from commit f9c1ae12)
(cherry picked from commit 5f728fc4)

This patch fixes missing -ddump-cmm-raw output when compiling .cmm,
which is useful for debugging cmm related codegen issues.

(cherry picked from commit 6346c669)
(cherry picked from commit e5a1dd43)