(cherry picked from commit 3655c3fd)

(cherry picked from commit 7318d779)

(cherry picked from commit afdd3fe7)
(cherry picked from commit f4d21490)

This commit adds a test case to test the wasm backend
promise.throwTo() logic.

(cherry picked from commit 7f80455e)
(cherry picked from commit bbbaba0e)

This commit implements promise.throwTo() for wasm backend JSFFI
exports. This allows the JavaScript side to interrupt Haskell
computation by raising an async exception. See subsequent docs/test
commits for more details.

(cherry picked from commit dc904bfd)
(cherry picked from commit 2162cb15)

We used to use keepAlive# to pin the raiseJSException closure when
blocking on a JSFFI import thunk, since it can potentially be used by
RTS. But raiseJSException may be used in other places as well (e.g.
the promise.throwTo logic), and it's better to simply unconditionally
pin it in the JSFFI initialization logic.

(cherry picked from commit da34f0aa)
(cherry picked from commit feecaeed)

Previously, when blocking on a JSFFI import, we push a custom
stg_jsffi_block stack frame and arrange the `promise.then` callback to
write to that stack frame. It turns out we can simply use the good old
MVar to implement the blocking logic, with a few benefits:

- Less maintenance burden. We can drop the stg_jsffi_block related Cmm
  code without loss of functionality.
- It interacts better with existing async exception mechanism. throwTo
  would properly block the caller if the target thread is masking
  async exceptions.

(cherry picked from commit 9cd9f347)
(cherry picked from commit f8a96987)

This commit adds hs_try_putmvar_with_value to rts. It allows more
flexibility than hs_try_putmvar by taking an additional value argument
as a closure to be put into the MVar. This function is used & tested
by the wasm backend runtime, though it makes sense to expose it as a
public facing RTS API function as well.

(cherry picked from commit f75e823e)
(cherry picked from commit e8afc767)

This commit implements GHC driver flags that enable the wasm ghci
browser mode.

(cherry picked from commit c6a3bc8f)
(cherry picked from commit 4f5fcfc3)

This commit updates Note [The Wasm Dynamic Linker] to reflect recent
developments, in particular the wasm ghci browser mode.

(cherry picked from commit 37381bcf)
(cherry picked from commit 5bdcce7b)

This commit updates the user manual to add wasm ghci subsection.

(cherry picked from commit 6ef5c0d2)
(cherry picked from commit 9b1259b9)

This commit adds support for testing the wasm ghci browser mode in the
testsuite, as well as a simple first test case browser001 that makes
use of TH, JSFFI and browser-specific DOM API. See added note and
comments for details.

(cherry picked from commit ac70e643)
(cherry picked from commit f3e0f4bd)

This commit adds brotli compression for wasm shared libraries for ghci
browser mode. With BROTLI_MIN_QUALITY, the overhead is negligible, and
it helps reducing amount of transferred data when the browser connects
to the server over a slow connection.

(cherry picked from commit 731217ce)
(cherry picked from commit e67dd992)

This commit adds optional support for redirecting wasi console
stdout/stderr back to the host when running wasm ghci browser mode. By
default, the wasi console outputs are only available under F12
devtools console, but in case of testing against a mobile browser, the
devtools console may not be readily available, and it would be more
convenient to at least get wasi console output on the host side.

The redirection logic is simple, just adding another two WebSockets
connections that pump the line-buffered textual messages back to
host.

(cherry picked from commit ad7e271d)
(cherry picked from commit afd60048)

This commit adds support for using puppeteer/playwright for
automatically launching a headless browser that backs the ghci browser
mode. This is useful for testing the ghci browser mode as a part of
GHC testsuite, and it's also convenient for local development since
the step to start iserv can be automated away.

(cherry picked from commit fc576798)
(cherry picked from commit 760cb5f9)

This commit implements the rest of dyld logic that delivers the ghci
browser mode:

- The dyld script can now fully run in the browser. It communicates
  back with dyld-on-nodejs via WebSockets and also plain HTTP 1.1
  requests.
- The host dyld starts a server and acts as a broker between the GHC
  process and the browser side. GHC doesn't need to know anything
  about the browser mode; no driver flags need to be added and no
  recompilation needs to happen, the GHC driver continues to use the
  original iserv binary messages protocol.
- The dyld broker doesn't need to parse any message between the
  browser dyld and GHC; it merely sets up WebSockets connections to
  redirect these messages as well as ^C signals.
- Plain HTTP 1.1 is used for IPC requests (e.g. downloading a wasm
  module).
- The dyld broker serves a main.js script that bootstraps iserv in the
  browser, and a main.html empty page playground for testing. CORS is
  enabled so it could be possible to inject iserv into other websites
  and use ghci to play with them.
- All the RPC logic is opaque to the DyLD class, the majority of the
  wasm dynamic linker code is already portable and runs fine in
  firefox/chrome/webkit.

Closes #25399.

(cherry picked from commit e93fc33d)
(cherry picked from commit 1f67f423)

This commit spins out a DyLDHost class from DyLD that handles side
effects that must be run in the same host environment that runs
wasm32-wasi-ghc. When the dyld script runs in the browser, it'll need
to do IPC to find libraries, fetch wasm library, etc, and the other
side of dyld that runs on nodejs would simply expose the DyLDHost
methods as endpoints for WebSockets/HTTP.

(cherry picked from commit 22ba2a78)
(cherry picked from commit 9d22e71d)

As we move towards supporting running the dyld script in the browser,
this commit implements the isNode module-level binding which is true
if dyld is running in nodejs. The nodejs-specific bits are gated under
isNode.

For the browser case, this commit introduces @bjorn3/browser_wasi_shim
as the wasi implementation; we already use it in quite a few projects
and it simply works.

(cherry picked from commit 7003a399)
(cherry picked from commit 9156318f)

The wasm dyld downsweep logic used to rely on nodejs path module to
handle filepaths. That's not available in browsers, so this commit
implements poor man's filepath handling in js, which is not elegant
for sure but works for both nodejs and the browser.

(cherry picked from commit d9b71e82)
(cherry picked from commit cb8d428f)

The wasm dyld script used to only run in node and directly uses
setImmediate in globalThis. In case of browsers, it needs to import
setImmediate from the prelude, hence this commit.

(cherry picked from commit 9a697181)
(cherry picked from commit 5f261dcd)

This commit refactors the simple binary parser in the dyld script in
charge of parsing the dylink.0 custom section. Previously the parser
was synchronous and operated on the entire input buffer; this was
simple and easy and worked well enough when the input wasm modules are
instantly read from local filesystem.

However, when running dyld in the browser, the wasm modules are
transferred via fetch() requests. The host ghc and the browser might
not be on the same machine, so slow network uplink does need to be
considered. We only need to parse dylink.0 custom section to extract
dependency info, and dylink.0 is the very first custom section in the
wasm shared library binary payload, so the parsing process should not
require fetch() to complete and should return the parsing result asap.

Hence the refactorings in this commit: asyncify the parser, make it
only consume as many bytes as needed by invoking an async consumer
callback. The input is a readable stream from the fetch() response;
once the response is available, the async wasm compilation can start
in the background, and dylink.0 parsing shall end asap which results
in more wasm shared libraries to be loaded earlier. Profit.

(cherry picked from commit 929df0ba)
(cherry picked from commit 48d0ea21)

This commit uses console.assert() instead of node-specific strict
assert in the dyld script, in order to make it runnable in the
browser. console.assert() only warns and doesn't crash upon assertion
failure, but this is fine; we can always trivially define a strict
assert function shall it be necessary when debugging, and there hasn't
been such an assertion failure seen in the wild for long enough.

(cherry picked from commit 27bb73c6)
(cherry picked from commit 17ef371d)

The wasm ghci browser mode needs to run dyld.mjs in the browser which
imports post-link.mjs. This script makes post-link.mjs runnable in the
browser by deferring node-specific module imports to their actual use
sites.

(cherry picked from commit efcebed6)
(cherry picked from commit 2b6ab4cc)

This commit fixes ^C handling for wasm iserv. Previously we didn't
handle it at all, so ^C would kill the node process and host ghc would
then crash as well. But native ghc with external interpreter can
handle ^C just fine and wasm should be no different. Hence the fix:
wasm iserv exports its signal handler as a js callback to be handled
by the dyld script. Also see added note for details.

(cherry picked from commit fa2fbd2b)
(cherry picked from commit 6797c0a4)

This commit makes wasm iserv take advantage of the Pipe refactoring by
passing binary receiver/sender js callbacks from the dyld script. This
paves the way for piping the binary messages through WebSockets in
order to run wasm iserv in the browser, but more importantly, it
allows us to get rid of a horrible hack in the dyld script: we no
longer have to fake magical wasi file descriptors that are backed by
nodejs blocking I/O! The legacy hack was due to these facts:

- iserv only supported exchanging binary messages via handles backed
  by file descriptors
- In wasi you can't access host file descriptors passed by host ghc
- The nodejs wasi implementation only allows mapping host directories
  into the wasi vfs, not host file descriptors
- Named pipes with file paths (mkfifo) doesn't work well with nodejs
  wasi implementation, causes spurious testsuite failures on macos

But starting from this point, we can fully take advantage of
non-blocking I/O on the js side.

(cherry picked from commit a2103fd2)
(cherry picked from commit 81aba64d)

This commit makes the Pipe type in ghci opaque, and introduce the
mkPipeFromHandles constructor for creating a Pipe from a pair of
Handles. Pipe is now just a pair of receiver/sender continuations
under the hood. This allows a Pipe to be potentially backed by other
IPC mechanisms (e.g. WebSockets) which is essential for wasm ghci
browser mode.

(cherry picked from commit 7d18c19b)
(cherry picked from commit 4148bc13)

This patch revamps the wasm backend's JSFFI internal implementation
and documentation:

- `JSValManager` logic to allocate a key is simplified to simple
  bumping. According to experiments with all major browsers, the
  internal `Map` would overflow the heap much earlier before we really
  exhaust the 32-bit key space, so there's no point in the extra
  complexity.
- `freeJSVal` is now idempotent and safe to call more than once. This
  is achieved by attaching the `StablePtr#` to the `JSVal#` closure
  and nullifying it when calling `freeJSVal`, so the same stable
  pointer cannot be double freed.
- `mkWeakJSVal` no longer exposes the internal `Weak#` pointer and
  always creates a new `Weak#` on the fly. Otherwise by finalizing
  that `Weak#`, user could accidentally drop the `JSVal`, but
  `mkWeakJSVal` is only supposed to create a `Weak` that observes the
  `JSVal`'s liveliness without actually interfering it.
- `PromisePendingException` is no longer exported since it's never
  meant to be caught by user code; it's a severe bug if it's actually
  raised at runtime.
- Everything exported by user-facing `GHC.Wasm.Prim` now has proper
  haddock documentation.
- Note [JSVal representation for wasm] has been updated to reflect the
  new JSVal# memory layout.

(cherry picked from commit fd40eaa1)
(cherry picked from commit 38410f28)

There's a much more efficient way to convert an unsigned i32 to a
signed one. Thanks, o3-mini-high.

(cherry picked from commit 75fcc5c9)
(cherry picked from commit 733b4c56)

The wasm dynamic linker used to depend on v8's experimental wasm type
reflection support to generate stub functions when treating GOT.func
items that aren't exported by any loaded library yet. However, as we
work towards wasm ghci browser mode (#25399), we need to ensure the
wasm dyld logic is portable across browsers. So this commit removes
the usage of wasm type reflection in wasm dyld, and it shall only be
added many months later when this feature is widely available in
browsers.

(cherry picked from commit cca68421)
(cherry picked from commit a398fae2)

This commit adds a mkWeakJSVal function that can be used to set up a
Weak pointer with a JSVal key to observe the key's lifetime and
optionally attach a finalizer.

(cherry picked from commit 55af20e6)
(cherry picked from commit 2788a597)

JSVal has an internal Weak# with the unlifted JSVal# object as key to
arrange its builtin finalization logic. The Weak# used to designate
Unit_closure as a dummy value; now this commit designates the lifted
JSVal closure as the Weak# value. This allows the implementation of
mkWeakJSVal which can be used to observe the liveliness of a JSVal and
attach a user-specified finalizer.

(cherry picked from commit 4f342431)
(cherry picked from commit 4941e126)

This commit makes JSVal an abstract type in the export list of
GHC.Wasm.Prim. JSVal's internal representation is supposed to be a non
user facing implementation detail subject to change at any time. We
should only expose things that are newtypes of JSVal, not JSVal
itself.

(cherry picked from commit 8037f487)
(cherry picked from commit a834b03d)

This commit attaches an error message to WouldBlockException, for now
the error message consists of the JS async import code snippet that
thunk is trying to block for. This is useful for debugging synchronous
callbacks that accidentally call an async JS function.

(cherry picked from commit 9b54eecb)
(cherry picked from commit a0995a9d)

This commit updates wasm backend documentation to reflect the new
JSFFI sync exports feature.

(cherry picked from commit edae2874)
(cherry picked from commit 1440292c)

This commit repurposes some existing JSFFI test cases to make them
cover JSFFI sync exports as well.

(cherry picked from commit b6ae908b)
(cherry picked from commit b7dce5cd)

This commit implements the synchronous flavour of the wasm backend
JSFFI exports:

- `foreign export javascript "foo sync"` exports a top-level Haskell
  binding as a synchronous JS function
- `foreign import javascript "wrapper sync"` dynamically exports a
  Haskell function closure as a synchronous JS function
- `foreign import javascript unsafe` is now re-entrant by lowering to
  a safe ccall
- Also fix the issue that JSFFI dynamic exports didn't really work in
  TH & ghci (#25473)

(cherry picked from commit 03ebab52)
(cherry picked from commit 8cefd046)

This commit allows arbitrary label string to appear in a foreign
export declaration, as long as the calling convention is javascript.
Well, doesn't make sense to enforce it's a C function symbol for a
JSFFI declaration anyway, and it gets in the way of implementing the
"sync" flavour of exports.

(cherry picked from commit a204df3a)
(cherry picked from commit 18dd15dc)

This commit fixes the runIO/runNonIO closure type signatures in the
RTS which should be extern StgClosure. This allows us to remove an
unnecessary type cast in the C foreign desugaring logic, as well as
unneeded complications of JSFFI desugaring logic that also needs to
generate C stubs that may refer to those top handler closures.
Otherwise, we'll have to take special care to avoid generating "extern
StgClosure" declarations for them as we would for other closures, just
to avoid conflicting type signature error at stub compile time.

(cherry picked from commit c78d8f55)
(cherry picked from commit 8ebb0348)

Ben Gamari authored 1 month ago and Cheng Shao committed 1 week ago

(cherry picked from commit 12752f0c)
(cherry picked from commit a73d6c7f)

Ben Gamari authored 1 month ago and Cheng Shao committed 1 week ago

To silence warnings with GHC 9.10

(cherry picked from commit a7e23f01)
(cherry picked from commit 1078f402)