This patch removes redundant autoconf check for the context diff
program given it isn't actually been used anywhere, especially since
make removal.

(cherry picked from commit b7bcf729)
(cherry picked from commit 0e9d2a0f)
(cherry picked from commit 00d58ab2)

This patch fixes I/O manager compilation errors for win32 target
discovered when cross-compiling to win32 using recent clang:

```
rts/win32/ThrIOManager.c:117:7: error:
     error: call to undeclared function 'is_io_mng_native_p'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
      117 |   if (is_io_mng_native_p ()) {
          |       ^
    |
117 |   if (is_io_mng_native_p ()) {
    |       ^

1 error generated.
`x86_64-w64-mingw32-clang' failed in phase `C Compiler'. (Exit code: 1)

rts/fs.c:143:28: error:
     error: a function declaration without a prototype is deprecated in all versions of C [-Werror,-Wstrict-prototypes]
      143 | int setErrNoFromWin32Error () {
          |                            ^
          |                             void
    |
143 | int setErrNoFromWin32Error () {
    |                            ^

1 error generated.
`x86_64-w64-mingw32-clang' failed in phase `C Compiler'. (Exit code: 1)

rts/win32/ConsoleHandler.c:227:9: error:
     error: call to undeclared function 'interruptIOManagerEvent'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
      227 |         interruptIOManagerEvent ();
          |         ^
    |
227 |         interruptIOManagerEvent ();
    |         ^

rts/win32/ConsoleHandler.c:227:9: error:
     note: did you mean 'getIOManagerEvent'?
    |
227 |         interruptIOManagerEvent ();
    |         ^

rts/include/rts/IOInterface.h:27:10: error:
     note: 'getIOManagerEvent' declared here
       27 | void *   getIOManagerEvent  (void);
          |          ^
   |
27 | void *   getIOManagerEvent  (void);
   |          ^

1 error generated.
`x86_64-w64-mingw32-clang' failed in phase `C Compiler'. (Exit code: 1)

rts/win32/ConsoleHandler.c:196:9: error:
     error: call to undeclared function 'setThreadLabel'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
      196 |         setThreadLabel(cap, t, "signal handler thread");
          |         ^
    |
196 |         setThreadLabel(cap, t, "signal handler thread");
    |         ^

rts/win32/ConsoleHandler.c:196:9: error:
     note: did you mean 'postThreadLabel'?
    |
196 |         setThreadLabel(cap, t, "signal handler thread");
    |         ^

rts/eventlog/EventLog.h:118:6: error:
     note: 'postThreadLabel' declared here
      118 | void postThreadLabel(Capability    *cap,
          |      ^
    |
118 | void postThreadLabel(Capability    *cap,
    |      ^

1 error generated.
`x86_64-w64-mingw32-clang' failed in phase `C Compiler'. (Exit code: 1)
```

(cherry picked from commit 710665bd)
(cherry picked from commit 9499ed96)
(cherry picked from commit e19bea4a)

(cherry picked from commit b1e0c313)
(cherry picked from commit 34882192)
(cherry picked from commit 94adf918)

This commit removes obsolete deletection logic of the patch command
from autoconf scripts, given we no longer need to patch anything in
the GHC build process.

(cherry picked from commit 71f28958)
(cherry picked from commit 1e77ded1)
(cherry picked from commit e5506d09)

This commit removes obsolete Patch logic from hadrian, given we no
longer need to patch the gmp tarball when building in-tree GMP.

(cherry picked from commit 65b4b92f)
(cherry picked from commit 92065500)
(cherry picked from commit e3d00525)

This commit removes an obsolete ln script in ghc-bignum/gmp. See
060251c2 for its original intention,
but it's been obsolete for a long time, especially since the removal
of the make build system. Hence the house cleaning.

(cherry picked from commit c62dc317)
(cherry picked from commit 5f4848c0)
(cherry picked from commit 5b9561d6)

This commit adds an assertion to Bdescr() to assert the pointer is
indeed heap allocated. This is useful to rule out RTS bugs that
attempt to access non-existent block descriptor of a static closure, #24492
being one such example.

(cherry picked from commit d19441d7)
(cherry picked from commit 8af2c3fd)

This commit exposes HeapAlloc.h as a public header. The intention is
to expose HEAP_ALLOCED/HEAP_ALLOCED_GC, so they can be used in
assertions in other public headers, and they may also be useful for
user code.

(cherry picked from commit dedcf102)
(cherry picked from commit 8d38b450)

Ever since the removal of the make build system, the in tree lndir
hasn't been actually built, so this patch removes it.

(cherry picked from commit d3a050d2)
(cherry picked from commit 04860c24)

This commit enables parallelism when the hadrian/ghci scripts are
called in CI. The time bottleneck is in the hadrian build step, but
previously the build step wasn't parallelized.

(cherry picked from commit 50bfdb46)
(cherry picked from commit b5df6e69)

This commit passes -j to hadrian in the hadrian/hie-bios scripts. When
the user starts HLS in a fresh clone that has just been configured, it
takes quite a while for hie-bios to pick up the ghc flags and start
actual indexing, due to the fact that the hadrian build step defaulted
to -j1, so -j speeds things up and improve HLS user experience in GHC.
Also add -j flag to .ghcid to speed up ghcid, and sets the Windows
build root to .hie-bios which also works and unifies with other
platforms, the previous build root _hie-bios was missing from
.gitignore anyway.

(cherry picked from commit c71bfdff)
(cherry picked from commit 57d706ed)

This commit removes the redundant logic of initializing each
Capability's rCCCS to CCS_SYSTEM in initProfiling(). Before
initProfiling() is called during RTS startup, each Capability's rCCCS
has already been assigned CCS_SYSTEM when they're first initialized.

(cherry picked from commit a7569495)
(cherry picked from commit 3dace6e1)

hs_try_putmvar002 includes pthread.h and doesn't work on targets
without this header (e.g. wasm32). It doesn't need to include this
header at all. This was previously unnoticed by wasm CI, though recent
toolchain upgrade brought in upstream changes that completely removes
pthread.h in the single-threaded wasm32-wasi sysroot, therefore we
need to handle that change.

(cherry picked from commit 2ce2a493)
(cherry picked from commit f581edb0)

This commit cleans up how we include the xxhash.h header and only
define XXH_INLINE_ALL, which is sufficient to inline the xxHash
functions without symbol collision.

(cherry picked from commit ee01de7d)
(cherry picked from commit 2c52e59b)

This commit enables XXH3_64bits hash to be used on all 64-bit
platforms. Previously it was only enabled on x86_64, so platforms like
aarch64 silently falls back to using XXH32 which degrades the hashing
function quality.

(cherry picked from commit 4a97bdb8)
(cherry picked from commit 8ed990f5)

(cherry picked from commit b19ec331)
(cherry picked from commit 7c0a6f59)

This commit explicitly disables the ncgPIC flag for the wasm32 target.
The wasm backend doesn't support PIC for the time being.

(cherry picked from commit 87e34888)
(cherry picked from commit f46220a9)

On posix platforms, when performing read/write on FDs, we check the
nonblocking flag first. For FDs without this flag (e.g. stdout), we
call fdReady() first, which in turn calls poll() to wait for I/O to be
available on that FD. This is problematic for wasm32-wasi: although
select()/poll() is supported via the poll_oneoff() wasi syscall, that
syscall is rather heavyweight and runtime behavior differs in
different wasi implementations. The issue is even worse when targeting
browsers, given there's no satisfactory way to implement async I/O as
a synchronous syscall, so existing JS polyfills for wasi often give up
and simply return ENOSYS.

Before we have a proper I/O manager that avoids poll_oneoff() for
async I/O on wasm, this patch improves the status quo a lot by merely
pretending all FDs are "nonblocking". Read/write on FDs will directly
invoke read()/write(), which are much more reliably handled in
existing wasi implementations, especially those in browsers.

Fixes #23275 and the following test cases: T7773 isEOF001 openFile009
T4808 cgrun025

Approved by CLC proposal #234:
https://github.com/haskell/core-libraries-committee/issues/234

(cherry picked from commit 2eca52b4)
(cherry picked from commit 288692dd)

(cherry picked from commit af771148)
(cherry picked from commit 84c875d1)

We used to have MIN_UPD_SIZE macro that describes the minimum reserved
size for thunks, so that the thunk can be overwritten in place as
indirections or blackholes. However, this macro has not been actually
defined or used anywhere since a long time ago; StgThunkHeader already
reserves a padding word for this purpose. Hence this patch which drops
stale mentions of MIN_UPD_SIZE.

(cherry picked from commit c1e3719c)
(cherry picked from commit 8d9efe4d)

(cherry picked from commit a4785b33)
(cherry picked from commit 45935f5d)

(cherry picked from commit bd575a13)

See added comment for details. Closes #24423.

(cherry picked from commit 20f80b77)

This patch fixes the STG_FIELD_OFFSET macro definition by using
__builtin_offsetof, which is what gcc/clang uses to implement offsetof
in standard C. The previous definition that uses NULL pointer involves
subtle undefined behavior in C and thus reported by
UndefinedBehaviorSanitizer as well:

```
rts/Capability.h:243:58: runtime error: member access within null pointer of type 'Capability' (aka 'struct Capability_')
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior rts/Capability.h:243:58
```

(cherry picked from commit 83341bbc)

The wasm backend didn't properly make use of all Cmm global registers
due to #24347. Now that it is fixed, this patch re-enables full
register mapping for wasm32, and we can now generate smaller & faster
wasm modules that doesn't always spill arguments onto the stack. Fixes #22460 #24152.

(cherry picked from commit 0cda2b8b)
(cherry picked from commit f1f5068b398b1effb837add38ecc5303dc9a381f)
(cherry picked from commit 1e695750)

ghc-api tests for wasm32 are more likely to timeout due to the large
wasm module sizes, especially when testing with wasm native tail
calls, given wasmtime's handling of tail call opcodes are suboptimal
at the moment. It makes sense to increase timeout specifically for
these tests on wasm32. This doesn't affect other targets, and for
wasm32 we don't increase timeout for all tests, so not to risk letting
major performance regressions slip through the testsuite.

(cherry picked from commit 07ab5cc1)
(cherry picked from commit c4d1edb1)

Fixes #22981.

(cherry picked from commit 1e6861dd)

This patch includes all wasm32-specific testsuite fixes.

(cherry picked from commit bd2bfdec)

Sylvain Henry authored 2 years ago and Cheng Shao committed 2 months ago

req_cmm is more informative than js_skip

(cherry picked from commit 239202a2)

This patch adds missing annotations (req_th, req_dynamic_lib_support,
req_rts_linker) to some tests. They were discovered when testing
wasm32, though it's better to be explicit about what features they
require, rather than simply adding when(arch('wasm32'), skip).

(cherry picked from commit b174a110)

This patch adds the req_host_target_ghc predicate to the testsuite to
assert the ghc compiler being tested can compile both host/target
code. When testing cross GHCs this is not supported yet, but it may
change in the future.

(cherry picked from commit cb933665)

This patch adds the req_ghc_with_threaded_rts predicate to the
testsuite to assert the platform has threaded RTS, and mark some tests
as req_ghc_with_threaded_rts. Also makes ghc_with_threaded_rts a
config field instead of a global variable.

(cherry picked from commit aa6afe8a)

(cherry picked from commit a580722e)
(cherry picked from commit 4f317bfd)
(cherry picked from commit d94d0176)

Sylvain Henry authored 2 years ago and Cheng Shao committed 2 months ago

See #22630 and !9552

This commit:
 - splits req_smp into req_target_smp and req_ghc_smp
 - changes the testsuite driver to calculate req_ghc_smp
 - changes a handful of tests to use req_target_smp instead of req_smp
 - changes a handful of tests to use req_host_smp when needed

The problem:
 - the problem this solves is the ambiguity surrounding req_smp
 - on master req_smp was used to express the constraint that the program
 being compiled supports smp _and_ that the host RTS (i.e., the RTS used
 to compile the program) supported smp. Normally that is fine, but in
 cross compilation this is not always the case as was discovered in #22630.

The solution:
 - Differentiate the two constraints:
   - use req_target_smp to say the RTS the compiled program is linked
   with (and the platform) supports smp
   - use req_host_smp to say the RTS the host is linked with supports smp

WIP: fix req_smp (target vs ghc)

add flag to separate bootstrapper

split req_smp -> req_target_smp and req_ghc_smp

update tests smp flags

cleanup and add some docstrings

only set ghc_with_smp to bootstrapper on S1 or CC

Only set ghc_with_smp to bootstrapperWithSMP of when testing stage 1
and cross compiling

test the RTS in config/ghc not hadrian

re-add ghc_with_smp

fix and align req names

fix T11760 to use req_host_smp

test the rts directly, avoid python 3.5 limitation

test the compiler in a try block

align out of tree and in tree withSMP flags

mark failing tests as host req smp

testsuite: req_host_smp --> req_ghc_smp

Fix ghc vs host, fix ghc_with_smp leftover

(cherry picked from commit 06036d93)

When the testsuite driver copy files instead of symlinking them, it
should also copy the permission bits, otherwise there'll be permission
denied errors. Also, enforce file copying when testing wasm32, since
wasmtime doesn't handle host symlinks quite well
(https://github.com/bytecodealliance/wasmtime/issues/6227).

(cherry picked from commit 791cce64)

This patch implements logic to automatically exclude ghci ways when
there is no rts linker. It's way better than having to annotate
individual test cases.

(cherry picked from commit e6416b10)

This patch fixes hp2ps related framework failures when testing the
wasm backend by including target exe extension in heap profile
filenames.

(cherry picked from commit 6f511c36)

This patch fixes cross prefix stripping in the testsuite driver. The
normalization logic used to only handle prefixes of the triple form
<arch>-<vendor>-<os>, now it's relaxed to allow any number of tokens
in the prefix tuple, so the cross prefix stripping logic would work
when ghc is configured with something like --target=wasm32-wasi.

(cherry picked from commit b5f00811)

This patch enables bignum native & unregisterised wasm32 jobs as
manual jobs in validate pipelines, which can be useful to prevent
breakage when working on wasm32 related patches.

(cherry picked from commit 533d075e)

For the time being, we still need to use in-tree mode and can't test
the bindist yet.

(cherry picked from commit d5c4629b)