The wasm dyld script used to only run in node and directly uses
setImmediate in globalThis. In case of browsers, it needs to import
setImmediate from the prelude, hence this commit.

(cherry picked from commit 9a697181)

This commit refactors the simple binary parser in the dyld script in
charge of parsing the dylink.0 custom section. Previously the parser
was synchronous and operated on the entire input buffer; this was
simple and easy and worked well enough when the input wasm modules are
instantly read from local filesystem.

However, when running dyld in the browser, the wasm modules are
transferred via fetch() requests. The host ghc and the browser might
not be on the same machine, so slow network uplink does need to be
considered. We only need to parse dylink.0 custom section to extract
dependency info, and dylink.0 is the very first custom section in the
wasm shared library binary payload, so the parsing process should not
require fetch() to complete and should return the parsing result asap.

Hence the refactorings in this commit: asyncify the parser, make it
only consume as many bytes as needed by invoking an async consumer
callback. The input is a readable stream from the fetch() response;
once the response is available, the async wasm compilation can start
in the background, and dylink.0 parsing shall end asap which results
in more wasm shared libraries to be loaded earlier. Profit.

(cherry picked from commit 929df0ba)

This commit uses console.assert() instead of node-specific strict
assert in the dyld script, in order to make it runnable in the
browser. console.assert() only warns and doesn't crash upon assertion
failure, but this is fine; we can always trivially define a strict
assert function shall it be necessary when debugging, and there hasn't
been such an assertion failure seen in the wild for long enough.

(cherry picked from commit 27bb73c6)

The wasm ghci browser mode needs to run dyld.mjs in the browser which
imports post-link.mjs. This script makes post-link.mjs runnable in the
browser by deferring node-specific module imports to their actual use
sites.

(cherry picked from commit efcebed6)

This commit fixes ^C handling for wasm iserv. Previously we didn't
handle it at all, so ^C would kill the node process and host ghc would
then crash as well. But native ghc with external interpreter can
handle ^C just fine and wasm should be no different. Hence the fix:
wasm iserv exports its signal handler as a js callback to be handled
by the dyld script. Also see added note for details.

(cherry picked from commit fa2fbd2b)

This commit makes wasm iserv take advantage of the Pipe refactoring by
passing binary receiver/sender js callbacks from the dyld script. This
paves the way for piping the binary messages through WebSockets in
order to run wasm iserv in the browser, but more importantly, it
allows us to get rid of a horrible hack in the dyld script: we no
longer have to fake magical wasi file descriptors that are backed by
nodejs blocking I/O! The legacy hack was due to these facts:

- iserv only supported exchanging binary messages via handles backed
  by file descriptors
- In wasi you can't access host file descriptors passed by host ghc
- The nodejs wasi implementation only allows mapping host directories
  into the wasi vfs, not host file descriptors
- Named pipes with file paths (mkfifo) doesn't work well with nodejs
  wasi implementation, causes spurious testsuite failures on macos

But starting from this point, we can fully take advantage of
non-blocking I/O on the js side.

(cherry picked from commit a2103fd2)

This commit makes the Pipe type in ghci opaque, and introduce the
mkPipeFromHandles constructor for creating a Pipe from a pair of
Handles. Pipe is now just a pair of receiver/sender continuations
under the hood. This allows a Pipe to be potentially backed by other
IPC mechanisms (e.g. WebSockets) which is essential for wasm ghci
browser mode.

(cherry picked from commit 7d18c19b)

Brandon Chinn authored 3 months ago and Cheng Shao committed 1 month ago

This INLINE pragma for alexScanUser was added in 9.12, but then I
ported the change to alex in 3.5.2.0
(https://github.com/haskell/alex/pull/262).

I didn't realize that GHC errors on duplicate INLINE pragmas, so
this ended up being a breaking change.

This change should be backported into 9.12

(cherry picked from commit a1d92378)

This patch revamps the wasm backend's JSFFI internal implementation
and documentation:

- `JSValManager` logic to allocate a key is simplified to simple
  bumping. According to experiments with all major browsers, the
  internal `Map` would overflow the heap much earlier before we really
  exhaust the 32-bit key space, so there's no point in the extra
  complexity.
- `freeJSVal` is now idempotent and safe to call more than once. This
  is achieved by attaching the `StablePtr#` to the `JSVal#` closure
  and nullifying it when calling `freeJSVal`, so the same stable
  pointer cannot be double freed.
- `mkWeakJSVal` no longer exposes the internal `Weak#` pointer and
  always creates a new `Weak#` on the fly. Otherwise by finalizing
  that `Weak#`, user could accidentally drop the `JSVal`, but
  `mkWeakJSVal` is only supposed to create a `Weak` that observes the
  `JSVal`'s liveliness without actually interfering it.
- `PromisePendingException` is no longer exported since it's never
  meant to be caught by user code; it's a severe bug if it's actually
  raised at runtime.
- Everything exported by user-facing `GHC.Wasm.Prim` now has proper
  haddock documentation.
- Note [JSVal representation for wasm] has been updated to reflect the
  new JSVal# memory layout.

(cherry picked from commit fd40eaa1)

There's a much more efficient way to convert an unsigned i32 to a
signed one. Thanks, o3-mini-high.

(cherry picked from commit 75fcc5c9)

The wasm dynamic linker used to depend on v8's experimental wasm type
reflection support to generate stub functions when treating GOT.func
items that aren't exported by any loaded library yet. However, as we
work towards wasm ghci browser mode (#25399), we need to ensure the
wasm dyld logic is portable across browsers. So this commit removes
the usage of wasm type reflection in wasm dyld, and it shall only be
added many months later when this feature is widely available in
browsers.

(cherry picked from commit cca68421)

This commit adds a mkWeakJSVal function that can be used to set up a
Weak pointer with a JSVal key to observe the key's lifetime and
optionally attach a finalizer.

(cherry picked from commit 55af20e6)

JSVal has an internal Weak# with the unlifted JSVal# object as key to
arrange its builtin finalization logic. The Weak# used to designate
Unit_closure as a dummy value; now this commit designates the lifted
JSVal closure as the Weak# value. This allows the implementation of
mkWeakJSVal which can be used to observe the liveliness of a JSVal and
attach a user-specified finalizer.

(cherry picked from commit 4f342431)

This commit makes JSVal an abstract type in the export list of
GHC.Wasm.Prim. JSVal's internal representation is supposed to be a non
user facing implementation detail subject to change at any time. We
should only expose things that are newtypes of JSVal, not JSVal
itself.

(cherry picked from commit 8037f487)

This patch makes `putWithTables` avoid overwriting all existing
UserData writers in the handle. This is crucial for GHC API users that
use putWithUserData/getWithUserData for serialization logic that
involve Names.

(cherry picked from commit c331eebf)

This commit attaches an error message to WouldBlockException, for now
the error message consists of the JS async import code snippet that
thunk is trying to block for. This is useful for debugging synchronous
callbacks that accidentally call an async JS function.

(cherry picked from commit 9b54eecb)

This commit updates wasm backend documentation to reflect the new
JSFFI sync exports feature.

(cherry picked from commit edae2874)

This commit repurposes some existing JSFFI test cases to make them
cover JSFFI sync exports as well.

(cherry picked from commit b6ae908b)

This commit implements the synchronous flavour of the wasm backend
JSFFI exports:

- `foreign export javascript "foo sync"` exports a top-level Haskell
  binding as a synchronous JS function
- `foreign import javascript "wrapper sync"` dynamically exports a
  Haskell function closure as a synchronous JS function
- `foreign import javascript unsafe` is now re-entrant by lowering to
  a safe ccall
- Also fix the issue that JSFFI dynamic exports didn't really work in
  TH & ghci (#25473)

(cherry picked from commit 03ebab52)

This commit allows arbitrary label string to appear in a foreign
export declaration, as long as the calling convention is javascript.
Well, doesn't make sense to enforce it's a C function symbol for a
JSFFI declaration anyway, and it gets in the way of implementing the
"sync" flavour of exports.

(cherry picked from commit a204df3a)

This commit fixes the runIO/runNonIO closure type signatures in the
RTS which should be extern StgClosure. This allows us to remove an
unnecessary type cast in the C foreign desugaring logic, as well as
unneeded complications of JSFFI desugaring logic that also needs to
generate C stubs that may refer to those top handler closures.
Otherwise, we'll have to take special care to avoid generating "extern
StgClosure" declarations for them as we would for other closures, just
to avoid conflicting type signature error at stub compile time.

(cherry picked from commit c78d8f55)

ghc-heap defines HalfWord as Word32/Word16 depending on host word
size. For cross GHC with different host/target word sizes, the Binary
instances are incompatible and breaks iserv serialization of any
message type that involves HalfWord, breaking the ghci debugger. This
patch fixes the issue and has been tested to fix ghci debugger
functionality of the wasm backend. Fixes #25420 #25781.

(cherry picked from commit b228fcb5)

GHCi.BreakArray.showBreakArray is not used anywhere, hence the
housecleaning.

(cherry picked from commit 2d6a63ab)

Hadrian used to omit the docs target as a part of binary-dist-dir for
cross targets. This commit enables docs to be built as a part of cross
bindists and it works just fine in CI.

(cherry picked from commit a794e733)

This commit enables building stage1 haddock for cross ghc. Without
this change, hadrian would panic with "Unknown program" error when
building the _build/stage1/bin/cross-prefix-haddock program needed by
the docs-haddock target, even if it only needs to copy from
_build/stage0/bin/cross-prefix-haddock.

(cherry picked from commit 272eaef0)

Fangyi Zhou authored 2 months ago and Cheng Shao committed 1 month ago

- Add new `WasmInstr` constructor `WasmSqrt` for sqrt, corresponding to
  primitivie operations in wasm.
- When lowering CallishMachOp, use `WasmAbs` and `WasmSqrt` for F32 and
  F64 fabs and sqrt.

(cherry picked from commit 64b0d4d0)

Bumps libffi submodule.

(cherry picked from commit c318be56)

This patch adds a note to explain how the magic variables like
`__ghc_wasm_jsffi_dyld` are brought into scope of JSFFI code snippets,
as follow up work of !13583.

(cherry picked from commit f3bfe31e)

For the wasm target, the driver calls `wasm32-wasi-clang
--print-search-dirs` and finds the sysroot directory that contains
libc.so etc, then passes the directory path to dyld. However, when GHC
is configured with -flto as a part of C/C++ compiler flags, the clang
driver would insert a llvm-lto specific directory in the
--print-search-dirs output and the driver didn't take that into
account. This patch fixes it and always selects the non-lto sysroot
directory to be passed to dyld. This is one small step towards
supporting building all cbits with lto for wasm.

(cherry picked from commit 7eea38c8)

This patch fixes wasm dyld script for shared libraries created by llvm
20.x. The __wasm_apply_data_relocs function is now optional and may be
omitted for shared libraries without any runtime relocatable data
segments, so only call __wasm_apply_data_relocs when it's present.

(cherry picked from commit cb60da24)

This patch uses fromAscList (with O(n) complexity) instead of fromList
(with O(nlogn) complexity) in certain Binary instances. It's safe to
do so since the corresponding serialization logic is based on toList
(same as toAscList).

(cherry picked from commit 1c8e608a)

The `InternalCounters` test case fails when ghc is built with
`+debug_ghc`. This patch skips it in that case and allows the
testsuite to pass for the `+debug_ghc` flavour transformer.

(cherry picked from commit 59b9307b)

This patch removes the unused assembleOneBCO function from the
bytecode assembler.

(cherry picked from commit 02794411)

This patch bumps hadrian index-state to fix bootstrap with 9.12.1.

(cherry picked from commit e16eae65)

Mateusz Goślinowski authored 3 months ago and Cheng Shao committed 1 month ago

(cherry picked from commit 14f8a7ec)

amesgen authored 3 months ago and Cheng Shao committed 1 month ago

This fixes the following esbuild error:

    ✘ [ERROR] Could not resolve "node:timers"

        www/ghc_wasm_jsffi.js:66:25:
          66 │     return (await import("node:timers")).setImmediate;
             ╵                          ~~~~~~~~~~~~~

      The package "node:timers" wasn't found on the file system but is built into node. Are you trying
      to bundle for node? You can use "--platform=node" to do that, which will remove this error.

Previously (i.e. after !13503), one had to work around this by passing
`--external:node:timers`.

(cherry picked from commit f6493dbc)

(cherry picked from commit 42826a89)

This patch makes test-bootstrap related ci jobs only depend on
hadrian-ghc-in-ghci job to finish, consistent with other jobs in the
full-build stage generated by gen_ci.hs. This allows the jobs to be
spawned earlier and improve overall pipeline parallelism.

(cherry picked from commit e684c406)

This patch makes some minor improvements re nix-in-docker logic in the
ci configuration:

- Update `nixos/nix` to the latest version
- Apply $CPUS to `cores`/`max-jobs` to avoid oversubscribing while
  allowing a reasonable degree of parallelism
- Remove redundant `--extra-experimental-features nix-command` in
  later `nix shell` invocations, it's already configured in
  `/etc/nix/nix.conf`

(cherry picked from commit 291388e1)