This commit implements the synchronous flavour of the wasm backend
JSFFI exports:

- `foreign export javascript "foo sync"` exports a top-level Haskell
  binding as a synchronous JS function
- `foreign import javascript "wrapper sync"` dynamically exports a
  Haskell function closure as a synchronous JS function
- `foreign import javascript unsafe` is now re-entrant by lowering to
  a safe ccall
- Also fix the issue that JSFFI dynamic exports didn't really work in
  TH & ghci (#25473)

(cherry picked from commit 03ebab52)
(cherry picked from commit 8cefd046)

This commit allows arbitrary label string to appear in a foreign
export declaration, as long as the calling convention is javascript.
Well, doesn't make sense to enforce it's a C function symbol for a
JSFFI declaration anyway, and it gets in the way of implementing the
"sync" flavour of exports.

(cherry picked from commit a204df3a)
(cherry picked from commit 18dd15dc)

This commit fixes the runIO/runNonIO closure type signatures in the
RTS which should be extern StgClosure. This allows us to remove an
unnecessary type cast in the C foreign desugaring logic, as well as
unneeded complications of JSFFI desugaring logic that also needs to
generate C stubs that may refer to those top handler closures.
Otherwise, we'll have to take special care to avoid generating "extern
StgClosure" declarations for them as we would for other closures, just
to avoid conflicting type signature error at stub compile time.

(cherry picked from commit c78d8f55)
(cherry picked from commit 8ebb0348)

Ben Gamari authored 2 months ago and Cheng Shao committed 2 months ago

(cherry picked from commit 12752f0c)
(cherry picked from commit a73d6c7f)

Ben Gamari authored 2 months ago and Cheng Shao committed 2 months ago

To silence warnings with GHC 9.10

(cherry picked from commit a7e23f01)
(cherry picked from commit 1078f402)

ghc-heap defines HalfWord as Word32/Word16 depending on host word
size. For cross GHC with different host/target word sizes, the Binary
instances are incompatible and breaks iserv serialization of any
message type that involves HalfWord, breaking the ghci debugger. This
patch fixes the issue and has been tested to fix ghci debugger
functionality of the wasm backend. Fixes #25420 #25781.

(cherry picked from commit b228fcb5)
(cherry picked from commit 25e75c1f)

GHCi.BreakArray.showBreakArray is not used anywhere, hence the
housecleaning.

(cherry picked from commit 2d6a63ab)
(cherry picked from commit 8e0bb14d)

Hadrian used to omit the docs target as a part of binary-dist-dir for
cross targets. This commit enables docs to be built as a part of cross
bindists and it works just fine in CI.

(cherry picked from commit a794e733)
(cherry picked from commit a0af1500)

This commit enables building stage1 haddock for cross ghc. Without
this change, hadrian would panic with "Unknown program" error when
building the _build/stage1/bin/cross-prefix-haddock program needed by
the docs-haddock target, even if it only needs to copy from
_build/stage0/bin/cross-prefix-haddock.

(cherry picked from commit 272eaef0)
(cherry picked from commit 0b775a8a)

Fangyi Zhou authored 2 months ago and Cheng Shao committed 2 months ago

- Add new `WasmInstr` constructor `WasmSqrt` for sqrt, corresponding to
  primitivie operations in wasm.
- When lowering CallishMachOp, use `WasmAbs` and `WasmSqrt` for F32 and
  F64 fabs and sqrt.

(cherry picked from commit 64b0d4d0)
(cherry picked from commit d6b9dab5)

Bumps libffi submodule.

(cherry picked from commit c318be56)
(cherry picked from commit c4b9fc99)

This patch adds a note to explain how the magic variables like
`__ghc_wasm_jsffi_dyld` are brought into scope of JSFFI code snippets,
as follow up work of !13583.

(cherry picked from commit f3bfe31e)
(cherry picked from commit 6f411dc5)

For the wasm target, the driver calls `wasm32-wasi-clang
--print-search-dirs` and finds the sysroot directory that contains
libc.so etc, then passes the directory path to dyld. However, when GHC
is configured with -flto as a part of C/C++ compiler flags, the clang
driver would insert a llvm-lto specific directory in the
--print-search-dirs output and the driver didn't take that into
account. This patch fixes it and always selects the non-lto sysroot
directory to be passed to dyld. This is one small step towards
supporting building all cbits with lto for wasm.

(cherry picked from commit 7eea38c8)
(cherry picked from commit d842f750)

This patch fixes wasm dyld script for shared libraries created by llvm
20.x. The __wasm_apply_data_relocs function is now optional and may be
omitted for shared libraries without any runtime relocatable data
segments, so only call __wasm_apply_data_relocs when it's present.

(cherry picked from commit cb60da24)
(cherry picked from commit 6bd4fd72)

This patch uses fromAscList (with O(n) complexity) instead of fromList
(with O(nlogn) complexity) in certain Binary instances. It's safe to
do so since the corresponding serialization logic is based on toList
(same as toAscList).

(cherry picked from commit 1c8e608a)
(cherry picked from commit 048b5404)

The `InternalCounters` test case fails when ghc is built with
`+debug_ghc`. This patch skips it in that case and allows the
testsuite to pass for the `+debug_ghc` flavour transformer.

(cherry picked from commit 59b9307b)
(cherry picked from commit 70003f90)

This patch adds a +text_simdutf flavour transformer to hadrian to
allow downstream packagers and users that build from source to opt-in
simdutf support for text, in order to benefit from SIMD speedup at
run-time. It's still disabled by default for the time being.

(cherry picked from commit 44e2abfb)

(cherry picked from commit c35281f9)

amesgen authored 3 months ago and Cheng Shao committed 2 months ago

(cherry picked from commit 7202a02c0a5238682de6a3a06a9b5137f02ad70c)
(cherry picked from commit 5e60fd64)

This patch bumps macOS minimum SDK version to 11.0 for x86_64-darwin
to align it with aarch64-darwin. This allows us to get rid of the
horrible -Wl,-U,___darwin_check_fd_set_overflow hack, which is causing
linker warnings and testsuite failures on macOS 15. Fixes #25504.

(cherry picked from commit 88c4fe1d)
(cherry picked from commit 5c7e7695)

This patch makes test-bootstrap related ci jobs only depend on
hadrian-ghc-in-ghci job to finish, consistent with other jobs in the
full-build stage generated by gen_ci.hs. This allows the jobs to be
spawned earlier and improve overall pipeline parallelism.

(cherry picked from commit e684c406)
(cherry picked from commit 4884eb08)

This patch fixes foreign import javascript "wrapper" in wasm backend's
TH/ghci by fixing the handling of dyld/finalization_registry magic
variables. Fixes T25473 and closes #25473.

(cherry picked from commit bd0a8b7e)
(cherry picked from commit 610b0f77)

This commit adds regression test T25473 marked as broken due to #25473.
It will be fixed in the subsequent commit.

(cherry picked from commit ed2ed6c5)
(cherry picked from commit fb470cb6)

This patch improves the compile_flags.txt config used to power clangd
for the rts C codebase. The flags in the file are sampled & deduped
from a real stage1 build with clang-19 and vastly improves the IDE
accuracy when hacking the rts.

For maximum code coverage under the default settings,
compile_flags.txt defaults to threaded+profiled+dynamic+debug way.
This does not mean profdyn needs to be actually built in _build/stage1
for IDE to work. To activate IDE for other RTS ways, simply remove one
of the -D flags at the end of compile_flags.txt and restart clangd.

(cherry picked from commit 59e0a770)
(cherry picked from commit 222a08ae)

This patch fixes one incoherent place between bindist makefile and
hadrian logic: I forgot to include wasi/wasm32 in
OsSupportsGHCi/ArchSupportsGHCi as well. And this results in incorrect
settings file generated after installing the bindist, and "Use
interpreter"/"Have interpreter" fields incorrectly have "NO" values
where they should be "YES" like --info output of in-tree version.

(cherry picked from commit 75a2eae4)
(cherry picked from commit 735f3f9a)

This patch removes the obsolete cross-port script in the tree. The
script was based on the legacy Make build system which has been pruned
from the tree long ago. For hadrian, proper support for two-stage
bootstrapping onto a new unsupported platform is a work in progress in
!11444.

(cherry picked from commit 00d551bf)
(cherry picked from commit 329a26da)

This patch fixes FinalizationRegistry related logic for Cloudflare
Workers in wasm backend js post linker. Cloudflare Workers doesn't
support FinalizationRegistry, in this case we use a dummy
implementation that doesn't do anything.

(cherry picked from commit bea8ea4c)
(cherry picked from commit 0346e7b7)

This patch fixes setImmediate() implementation for Cloudflare Workers
in the wasm backend's js prelude script. Cloudflare Workers doesn't
support the MessageChannel API, and we use a setTimeout() based
fallback implementation in this case.

(cherry picked from commit c37b96fa)
(cherry picked from commit 6bd2d9ad)

The executable wrappers are handled by hadrian and bindist Makefile.
The various .wrapper scripts in the tree are unused since removal of
Make build system, so this patch removes them all.

(cherry picked from commit 9ede97f3)
(cherry picked from commit 737e9e0b)

This patch removes unused `USE_REPORT_PRELUDE` code paths from the
tree. They have been present since the first git revision
4fb94ae5, and might have been useful
for debugging purposes many years ago, but these code paths are never
actually built. Removing these ease maintenance of relevant modules in
the future, and also allows us to get rid of `CPP` extension in those
modules as a nice byproduct.

(cherry picked from commit 573cad4b)

This patch fixes foreign stub handling logic in `hscParsedDecls`.
Previously foreign stubs were simply ignored here, so any feature that
involve foreign stubs would not work in ghci (e.g. CApiFFI). The patch
reuses `generateByteCode` logic and eliminates a large chunk of
duplicate logic that implements Core to bytecode generation pipeline
here. Fixes #25414.

(cherry picked from commit 677e3aa56e905524071fc9717a88ad2cd1bc2951)

This commit adds T25414 test case to demonstrate #25414. It is marked
as broken and will be fixed by the next commit.

(cherry picked from commit 82213395dc2fbdc8b452336da0909896b4300218)

This patch removes unused `hscDecls`/`hscDeclsWithLocation` functions
from the compiler, to reduce maintenance burden when doing
refactorings related to ghci.

(cherry picked from commit e3496ef6)

This patch fixes the wasm backend JSFFI prelude script to avoid
calling `import("node:timers")` on non-deno hosts. Safari doesn't like
it and would print an error message to the console. Fixes
https://gitlab.haskell.org/ghc/ghc-wasm-meta/-/issues/13.

(cherry picked from commit 301c3b54)

(cherry picked from commit 589fea7f)

This patch disables internal-interpreter flag for stage0 ghc-bin when
not cross compiling, see added comment for explanation. Fixes #25406.

(cherry picked from commit fde12aba)

This patch adds `--max-old-space-size=8192` to wasm dyld shebang
arguments to bump V8 heap size limit. The default limit
(`heap_size_limit` returned by `v8.getHeapStatistics()`) is
dynamically determined and a bit too low under certain workloads, and
V8 would waste too much CPU time to garbage collect old generation
heap more aggressively. Bumping the limit to 8G doesn't imply dyld
would really take that much memory at run-time, but it lessens V8 heap
stress significantly.

(cherry picked from commit 14c5143899d164c7ac1213d918b4819684538c4b)

Luite Stegeman authored 6 months ago and Cheng Shao committed 2 months ago

This adds locking to communication with the external interpreter
to prevent concurrent tasks interfering with each other. This
fixes Template Haskell with the external interpreter in parallel (-j)
builds.

Fixes #25083

This commit flips on support for shared libs, TH & ghci for wasm in
the testsuite, given support has been landed in previous commits.

(cherry picked from commit 74a1f681)

This commit fixes shared library size tests (e.g. array_so in
testsuite/tests/perf/size/all.T) when testing cross ghc. Previously,
if shared library file extension of host and target differs, those
tests will fail with framework errors due to not finding the right
files.

(cherry picked from commit 05e40406)
(cherry picked from commit d00c26a5)
(cherry picked from commit 908d6621)