This commit updates Note [The Wasm Dynamic Linker] to reflect recent
developments, in particular the wasm ghci browser mode.

This commit updates the user manual to add wasm ghci subsection.

This commit adds support for testing the wasm ghci browser mode in the
testsuite, as well as a simple first test case browser001 that makes
use of TH, JSFFI and browser-specific DOM API. See added note and
comments for details.

This commit adds brotli compression for wasm shared libraries for ghci
browser mode. With BROTLI_MIN_QUALITY, the overhead is negligible, and
it helps reducing amount of transferred data when the browser connects
to the server over a slow connection.

This commit adds optional support for redirecting wasi console
stdout/stderr back to the host when running wasm ghci browser mode. By
default, the wasi console outputs are only available under F12
devtools console, but in case of testing against a mobile browser, the
devtools console may not be readily available, and it would be more
convenient to at least get wasi console output on the host side.

The redirection logic is simple, just adding another two WebSockets
connections that pump the line-buffered textual messages back to
host.

This commit adds support for using puppeteer/playwright for
automatically launching a headless browser that backs the ghci browser
mode. This is useful for testing the ghci browser mode as a part of
GHC testsuite, and it's also convenient for local development since
the step to start iserv can be automated away.

This commit implements the rest of dyld logic that delivers the ghci
browser mode:

- The dyld script can now fully run in the browser. It communicates
  back with dyld-on-nodejs via WebSockets and also plain HTTP 1.1
  requests.
- The host dyld starts a server and acts as a broker between the GHC
  process and the browser side. GHC doesn't need to know anything
  about the browser mode; no driver flags need to be added and no
  recompilation needs to happen, the GHC driver continues to use the
  original iserv binary messages protocol.
- The dyld broker doesn't need to parse any message between the
  browser dyld and GHC; it merely sets up WebSockets connections to
  redirect these messages as well as ^C signals.
- Plain HTTP 1.1 is used for IPC requests (e.g. downloading a wasm
  module).
- The dyld broker serves a main.js script that bootstraps iserv in the
  browser, and a main.html empty page playground for testing. CORS is
  enabled so it could be possible to inject iserv into other websites
  and use ghci to play with them.
- All the RPC logic is opaque to the DyLD class, the majority of the
  wasm dynamic linker code is already portable and runs fine in
  firefox/chrome/webkit.

Closes #25399.

This commit spins out a DyLDHost class from DyLD that handles side
effects that must be run in the same host environment that runs
wasm32-wasi-ghc. When the dyld script runs in the browser, it'll need
to do IPC to find libraries, fetch wasm library, etc, and the other
side of dyld that runs on nodejs would simply expose the DyLDHost
methods as endpoints for WebSockets/HTTP.

As we move towards supporting running the dyld script in the browser,
this commit implements the isNode module-level binding which is true
if dyld is running in nodejs. The nodejs-specific bits are gated under
isNode.

For the browser case, this commit introduces @bjorn3/browser_wasi_shim
as the wasi implementation; we already use it in quite a few projects
and it simply works.

The wasm dyld downsweep logic used to rely on nodejs path module to
handle filepaths. That's not available in browsers, so this commit
implements poor man's filepath handling in js, which is not elegant
for sure but works for both nodejs and the browser.

The wasm dyld script used to only run in node and directly uses
setImmediate in globalThis. In case of browsers, it needs to import
setImmediate from the prelude, hence this commit.

This commit refactors the simple binary parser in the dyld script in
charge of parsing the dylink.0 custom section. Previously the parser
was synchronous and operated on the entire input buffer; this was
simple and easy and worked well enough when the input wasm modules are
instantly read from local filesystem.

However, when running dyld in the browser, the wasm modules are
transferred via fetch() requests. The host ghc and the browser might
not be on the same machine, so slow network uplink does need to be
considered. We only need to parse dylink.0 custom section to extract
dependency info, and dylink.0 is the very first custom section in the
wasm shared library binary payload, so the parsing process should not
require fetch() to complete and should return the parsing result asap.

Hence the refactorings in this commit: asyncify the parser, make it
only consume as many bytes as needed by invoking an async consumer
callback. The input is a readable stream from the fetch() response;
once the response is available, the async wasm compilation can start
in the background, and dylink.0 parsing shall end asap which results
in more wasm shared libraries to be loaded earlier. Profit.

This commit uses console.assert() instead of node-specific strict
assert in the dyld script, in order to make it runnable in the
browser. console.assert() only warns and doesn't crash upon assertion
failure, but this is fine; we can always trivially define a strict
assert function shall it be necessary when debugging, and there hasn't
been such an assertion failure seen in the wild for long enough.

The wasm ghci browser mode needs to run dyld.mjs in the browser which
imports post-link.mjs. This script makes post-link.mjs runnable in the
browser by deferring node-specific module imports to their actual use
sites.

This commit fixes ^C handling for wasm iserv. Previously we didn't
handle it at all, so ^C would kill the node process and host ghc would
then crash as well. But native ghc with external interpreter can
handle ^C just fine and wasm should be no different. Hence the fix:
wasm iserv exports its signal handler as a js callback to be handled
by the dyld script. Also see added note for details.

This commit makes wasm iserv take advantage of the Pipe refactoring by
passing binary receiver/sender js callbacks from the dyld script. This
paves the way for piping the binary messages through WebSockets in
order to run wasm iserv in the browser, but more importantly, it
allows us to get rid of a horrible hack in the dyld script: we no
longer have to fake magical wasi file descriptors that are backed by
nodejs blocking I/O! The legacy hack was due to these facts:

- iserv only supported exchanging binary messages via handles backed
  by file descriptors
- In wasi you can't access host file descriptors passed by host ghc
- The nodejs wasi implementation only allows mapping host directories
  into the wasi vfs, not host file descriptors
- Named pipes with file paths (mkfifo) doesn't work well with nodejs
  wasi implementation, causes spurious testsuite failures on macos

But starting from this point, we can fully take advantage of
non-blocking I/O on the js side.

This commit makes the Pipe type in ghci opaque, and introduce the
mkPipeFromHandles constructor for creating a Pipe from a pair of
Handles. Pipe is now just a pair of receiver/sender continuations
under the hood. This allows a Pipe to be potentially backed by other
IPC mechanisms (e.g. WebSockets) which is essential for wasm ghci
browser mode.

Ben Gamari authored 1 month ago and Marge Bot committed 1 month ago

Previously it was not made clear that the directory name is significant.

jeffrey young authored 6 months ago and jeffrey young committed 1 month ago

-- see CLC #289

jeffrey young authored 6 months ago and jeffrey young committed 1 month ago

-- see CLC #289

Matthew Pickering authored 1 month ago and Marge Bot committed 1 month ago

This reverts commit 64ea68d9.

See #25889

This commit refactors the 'childGREPriority' function which is used when
renaming subordinate items in export lists and class declarations.

Instead of having a complicated LookupChild parameter, we now simply pass the
GREInfo of the parent, which allows us to decide what is a valid child:

  - classes can have children that are in the type constructor namespace,
  - promoted data constructors should be treated the same as normal data
    constructors.

Fixes #24027

Alan Zimmerman authored 1 month ago and Marge Bot committed 1 month ago

This commit fixes two minor issues with exactprinting of the
SPECIALISE pragma after !12319 landed

- The span for the RHS did not include the optional signature type
- The `::` was printed twice when the legacy path was used

Closes #25885

Teo Camarasu authored 2 months ago and Marge Bot committed 1 month ago

This module is purely used for the implementation of TH quote
desugarring.

Historically this needed to be exposed from `template-haskell`, since
that's the package that the desugarred expressions referenced but since
!12479, this is no
longer the case.

Now these identifiers are in `ghc-internal`.

Note that this module has carried the following warning for a long time:

> This is not a part of the public API, and as such, there are no API guarantees for this
module from version to version.

Resolves #24766

sheaf authored 1 month ago and Marge Bot committed 1 month ago

This commit fixes a serious bug in the padLiveArgs function, which
was incorrectly computing too many padding registers. This caused
segfaults, e.g. in the UnboxedTuples test.

Fixes #25770
Fixes #25773

sheaf authored 1 month ago and Marge Bot committed 1 month ago

This commit ensures we do not add solved Wanted constraints that mention
HasCallStack or HasExceptionContext constraints to the set of solved
Wanted dictionary constraints: caching them is invalid, because re-using
such cached dictionaries means using an old call-stack instead of
constructing a new one, as was reported in #25529.

Fixes #25529.

Cheng Shao authored 1 year ago and Marge Bot committed 1 month ago

T7919 may also fail i386 CI jobs with test timeout.

sheaf authored 1 month ago and Marge Bot committed 1 month ago

This commit modifies rnClsInstDecl so that, when renaming, we reject a class
instance declaration in which the head is not a class. Before this change, it
would get rejected in the type-checker, but that meant that the renamer could
emit unhelpful error messages, e.g.:

  data Foo m a
  instance Foo m where
    fmap _ x = case x of

would rather unhelpfully say:

    ‘fmap’ is not a (visible) method of class ‘Foo’

when of course 'Foo' is not even a class. We now reject the above program
with the following error message:

    Illegal instance for data type ‘Foo’.
    Instance heads must be of the form
      C ty_1 ... ty_n
    where ‘C’ is a class.

Fixes #22688

sheaf authored 1 month ago and Marge Bot committed 1 month ago

This commit removes the remaining SDocs from the HsDocContext data type.
It adds the following constructors:

  ClassInstanceCtx  -- Class instances
  ClassMethodSigCtx -- Class method signatures
  SpecialiseSigCtx  -- SPECIALISE pragmas
  PatSynSigCtx      -- Pattern synonym signatures

We now report a bit more information for errors while renaming class instances,
which slightly improves renamer-emitted error messages.

sheaf authored 1 month ago and Marge Bot committed 1 month ago

This commit replaces unstructured SDoc arguments in error message constructors
with uses of the following two datatypes:

  - SigLike: for different kinds of signatures (e.g. standalone kind signatures,
    fixity signatures, COMPLETE pragmas, etc)
  - Subordinate: for class methods, associated types, and record fields

The following error message constructors now no longer have any SDocs in them:

  - TcRnIllegalBuiltinSyntax: SDoc -> SigLike
  - MissingBinding: SDoc -> SigLike
  - UnknownSubordinate: SDoc -> (Name, Subordinate)
  - SuggestMoveToDeclarationSite: SDoc -> SigLike

Simon Peyton Jones authored 1 month ago and Marge Bot committed 1 month ago

This big MR entirely removes the "flattener" that took a type and
replaced each type-family application with a fresh type variable.
The flattener had its origin in the paper
     Injective type families for Haskell

But (a) #25657 showed that flattening doesn't really work.
    (b) since we wrote the paper we have introduced the so-called
        "fine-grained" unifier GHC.Core.Unify, which can return
                 * SurelyApart
                 * Unifiable subst
                 * MaybeApart subst
        where the MaybeApart says that the two types are not unifiable by a
        substitution, but could (perhaps) be unified "later" after some type
        family reductions.  This turns out to subsume flattening.

This MR does a major refactor of GHC.Core.Unify to make it capable of
subsuming flattening.   The main payload is described in
       Note [Apartness and type families]
and its many wrinkles.

The key (non-refactoring) implementation change is to add `um_fam_env`
to the `UMState` in the unification monad.

Careful review with Richard revealed various bugs in the treament of
`kco`, the kind coercion carried around by the unifier, so that is
substantially fixed too: see Note [Kind coercions in Unify].

Compile-time performance is improved by 0.1% with a few improvements over
1% and one worsening by 1.3% namely T9872a.  (I have not investigated the
latter.)

Metric Decrease:
    T9872b
    T9872c
    TcPlugin_RewritePerf
Metric Increase:
    T9872a

Matthew Craven authored 1 month ago and Marge Bot committed 1 month ago

As with ghc-prim, it makes sense to have some protection against
accidental interface changes to this package caused by changes
in ghc-internal.

Ticat Fp authored 1 month ago and Marge Bot committed 1 month ago

Ensure that GHC-driver builds default to mcmodel=medium, so that GHC
passes this default parameter to CC without having to add it to the
compiled project.

Commit e70d4140 does not ensure that all
GHC-built object files have a default model of medium, and will raise an
R_LARCH_B26 overflow error.

Ben Gamari authored 1 month ago and Marge Bot committed 1 month ago

As noted in #25838, previously `selectIOManager` failed to set
`rts_IOManagerIsWin32Native` in its `IO_MNGR_FLAG_AUTO`. This meant
that the MIO path was taken when WinIO was supposedly selected,
resulting in chaos.

Fixes #25838.

Ben Gamari authored 2 months ago and Marge Bot committed 1 month ago

CentOS 7 is EoL and moreover we cannot even build images for it.

See #25061.

Sjoerd Visscher authored 4 months ago and Marge Bot committed 1 month ago

Needing to store multiplicity annotations on records triggered a refactoring of AST of data declarations:

Moved HsBangTy and HsRecTy from HsType to HsTypeGhcPsExt, the extension of HsType during parsing, since they are only needed during parsing.

New HsConDeclField that stores all source data shared by all constructor declaration fields: unpackedness, strictness, multiplicity, documentation and the type of the field.

Merged HsMultAnn and HsArrowOf, so all multiplicity annotations share the same data type.

HsBang was no longer needed as a separate type, and was inlined into HsSrcBang.

Andreas Klebinger authored 1 month ago and Marge Bot committed 1 month ago

Fixes #25867. (Ben-raytrace being broken by library changes)