|
|
This page is a working document about the JavaScript backend in GHC and is regularly updated. The JavaScript backend has been implemented (cc25d52e0f65d54c052908c7d91d5946342ab88a) and is included as a code generator as of GHC 9.6.
|
|
|
This page is a working document about the JavaScript backend in GHC and is regularly updated. The JavaScript backend has been implemented (cc25d52e0f65d54c052908c7d91d5946342ab88a) and is included as a code generator as of GHC 9.6.
|
|
|
|
|
|
## Current status and supported features
|
|
|
- [X] (cc25d52e0f65d54c052908c7d91d5946342ab88a) Boot libraries build
|
|
|
- [X] (394b91ce859653231813fb9af77c26664063c1b6) gitlab CI tests the backend
|
|
|
- [ ] (In progress: !9779) Template Haskell
|
|
|
- [ ] FFI: foreign imports
|
|
|
- [ ] FFI: foreign exports
|
|
|
- [ ] Dom integration
|
|
|
- [ ] Perf: compactor is implemented
|
|
|
- [ ] Perf: optimizer is implemented
|
|
|
|
|
|
## Roadmap
|
|
|
TODO
|
|
|
|
|
|
## Build Requirements
|
|
|
The following have to be on your `$PATH`.
|
|
|
- emscripten version 3.14 or better
|
|
|
- llvm 15, or an llvm with a series of specific patches, see the FAQ for errors to expect if you don't have these patches.
|
|
|
|
|
|
## Build Instructions
|
|
|
The JS backend uses Hadrian. The build is similar to vanilla GHC, just slower and with some changes to the configure step.
|
|
|
|
|
|
### Boot and configure
|
|
|
Run, configure should take longer than normal because its being run through `emscripten`:
|
|
|
```
|
|
|
[nix-shell:~/programming/ghc]$ ./boot && emconfigure ./configure --target=js-unknown-ghcjs
|
|
|
```
|
|
|
|
|
|
Notice that `emconfigure` wraps the `configure` script _and_ its arguments. This was not the case in early versions of the JS backend (we didn't use `emconfigure` at all). But after a lot of pain we agreed it was the most portable way to set emscripten binaries for autoconf. A successful configure should point to `emscripten` wrapped binaries for `ar` `nm` `clang` and friends, like so:
|
|
|
```
|
|
|
----------------------------------------------------------------------
|
|
|
Configure completed successfully.
|
|
|
|
|
|
Building GHC version : 9.5.20220819
|
|
|
Git commit id : 08c3c4783c72d3173d79ccda2ac282e2d3e04e34
|
|
|
|
|
|
Build platform : x86_64-unknown-linux
|
|
|
Host platform : x86_64-unknown-linux
|
|
|
Target platform : js-unknown-ghcjs
|
|
|
|
|
|
Bootstrapping using : /nix/store/4bkmkc7c98m4qyszsshnw9iclzzmdn4n-ghc-9.2.3-with-packages/bin/ghc
|
|
|
which is version : 9.2.3
|
|
|
with threaded RTS? : YES
|
|
|
|
|
|
Using (for bootstrapping) : /nix/store/yzs8390walgk2rwl6i5li2g672hdn0kv-gcc-wrapper-11.3.0/bin/cc
|
|
|
Using clang : /nix/store/p894nlicv53firllwgrfxfi51jzckh5l-emscripten-3.1.15/bin/emcc
|
|
|
which is version : 15.0.0
|
|
|
linker options :
|
|
|
Building a cross compiler : YES
|
|
|
Unregisterised : NO
|
|
|
TablesNextToCode : YES
|
|
|
Build GMP in tree : NO
|
|
|
hs-cpp : /nix/store/p894nlicv53firllwgrfxfi51jzckh5l-emscripten-3.1.15/bin/emcc
|
|
|
hs-cpp-flags : -E -undef -traditional -Wno-invalid-pp-token -Wno-unicode -Wno-trigraphs
|
|
|
ar : /nix/store/p894nlicv53firllwgrfxfi51jzckh5l-emscripten-3.1.15/bin/emar
|
|
|
ld : /nix/store/p894nlicv53firllwgrfxfi51jzckh5l-emscripten-3.1.15/bin/emcc
|
|
|
nm : /nix/store/0dp0bfg9sncg7bjy389zwyg2gskknm6b-emscripten-llvm-3.1.15/bin/llvm-nm
|
|
|
objdump : /nix/store/zgvxnf9047rdd8g8kq2zxxm9k6kfqf8b-binutils-2.38/bin/objdump
|
|
|
ranlib : /nix/store/p894nlicv53firllwgrfxfi51jzckh5l-emscripten-3.1.15/bin/emranlib
|
|
|
otool : otool
|
|
|
install_name_tool : install_name_tool
|
|
|
windres :
|
|
|
dllwrap :
|
|
|
genlib :
|
|
|
Happy : /nix/store/ijdmyaj6i6hgx5ll0lxxgcm9b0xn8nma-happy-1.20.0/bin/happy (1.20.0)
|
|
|
Alex : /nix/store/qzgm2m7p7xc0fnyj4vy3jcmz8pvbg9p7-alex-3.2.6/bin/alex (3.2.6)
|
|
|
sphinx-build : /nix/store/27dk5i52465a4azjr2dqmrhyc0m4lpf2-python3.9-sphinx-4.5.0/bin/sphinx-build
|
|
|
xelatex : /nix/store/8jc2258h4nqzqjy303zzkssd3ip675pf-texlive-combined-2021/bin/xelatex
|
|
|
makeinfo : /run/current-system/sw/bin/makeinfo
|
|
|
git : /nix/store/vsr2cn15h7cbwd5vqsam2ab2jzwfbyf9-git-2.36.0/bin/git
|
|
|
cabal-install : /nix/store/cjmd2qv1b5pdw4lxh1aw4xwwy4ibnb2p-cabal-install-3.6.2.0/bin/cabal
|
|
|
|
|
|
Using LLVM tools
|
|
|
clang : clang
|
|
|
llc : llc
|
|
|
opt : opt
|
|
|
|
|
|
HsColour was not found; documentation will not contain source links
|
|
|
|
|
|
Tools to build Sphinx HTML documentation available: YES
|
|
|
Tools to build Sphinx PDF documentation available: YES
|
|
|
Tools to build Sphinx INFO documentation available: YES
|
|
|
----------------------------------------------------------------------
|
|
|
```
|
|
|
|
|
|
You can also double check this output in `ghc/hadrian/cfg/system.config`:
|
|
|
```
|
|
|
# This file is processed by the configure script.
|
|
|
# See hadrian/src/UserSettings.hs for user-defined settings.
|
|
|
#===========================================================
|
|
|
|
|
|
# Paths to builders:
|
|
|
#===================
|
|
|
|
|
|
alex = /nix/store/qzgm2m7p7xc0fnyj4vy3jcmz8pvbg9p7-alex-3.2.6/bin/alex
|
|
|
ar = /nix/store/p894nlicv53firllwgrfxfi51jzckh5l-emscripten-3.1.15/bin/emar
|
|
|
autoreconf = /nix/store/yfnhm2b6plv48i8sgl64sd148b48hcly-autoconf-2.71/bin/autoreconf
|
|
|
cc = /nix/store/p894nlicv53firllwgrfxfi51jzckh5l-emscripten-3.1.15/bin/emcc
|
|
|
...
|
|
|
```
|
|
|
|
|
|
Notice that each of the binaries very obviously point to the `emscripten` wrapped versions. If they aren't then it means `emconfigure` isn't setting the environment variables correctly for some reason (cf #22814). You can try setting them explicitly as follows:
|
|
|
|
|
|
```bash
|
|
|
./configure CC=$(which emcc) LD=$(which emcc) --target=js-unknown-ghcjs
|
|
|
```
|
|
|
|
|
|
We're now ready to start building.
|
|
|
* [x] (cc25d52e0f65d54c052908c7d91d5946342ab88a) Boot libraries build
|
|
|
* [x] (394b91ce859653231813fb9af77c26664063c1b6) gitlab CI tests the backend
|
|
|
* [ ] (In progress: !9779) Template Haskell
|
|
|
* [ ] FFI: foreign imports
|
|
|
* [ ] FFI: foreign exports
|
|
|
* [ ] Dom integration
|
|
|
* [ ] Perf: compactor is implemented
|
|
|
* [ ] Perf: optimizer is implemented
|
|
|
|
|
|
### Build the compiler
|
|
|
Build using Hadrian, no surprises here. Most of the existing flavours have been modified to be JS aware:
|
|
|
```
|
|
|
[nix-shell:~/programming/ghc]$ hadrian/build -j12 --flavour=quick --bignum=native --docs=none
|
|
|
```
|
|
|
## Roadmap
|
|
|
|
|
|
That should take around 30 minutes to complete. `hsc2hs` should be very slow (we're sorry) because its running `emcc` several times. Similarly, expect the `CCS.hs` module itself to take minutes (again, we're sorry! I (Jeff) promise we have a ticket for it). Now you should have a working Haskell to JavaScript compiler. Good luck!
|
|
|
TODO
|
|
|
|
|
|
## Running the test suite
|
|
|
You can run the test suite like so:
|
|
|
```
|
|
|
[nix-shell:~/programming/ghc]$ hadrian/build --bignum=native --docs=none -j test
|
|
|
```
|
|
|
## Build instructions
|
|
|
|
|
|
## Shell Scripts
|
|
|
We use this script to wrap the call to `hadrian` and `configure`. It will also build the compiler for you if you do not have one in `_build`, so be warned that it may delete a previous build:
|
|
|
```
|
|
|
#!/usr/bin/env bash
|
|
|
|
|
|
## Sane defaults
|
|
|
set -euo pipefail
|
|
|
|
|
|
## flags to make emscripten slightly faster, these speeds up compilation
|
|
|
export EMCC_SKIP_SANITY_CHECK=1
|
|
|
export EM_IGNORE_SANITY=1
|
|
|
|
|
|
## configure using emconfigure, targeting js-backend
|
|
|
emconfigure ./configure --target=js-unknown-ghcjs
|
|
|
|
|
|
## build and run the test suite
|
|
|
./hadrian/build-stack \
|
|
|
--flavour=perf+omit_pragmas \
|
|
|
--bignum=native \
|
|
|
--docs=none \
|
|
|
"stage1.*.ghc.hs.opts += -ddump-stg-cg -ddump-js -ddump-to-file" \
|
|
|
-j \
|
|
|
"$@"
|
|
|
```
|
|
|
it sets the appropriate variables for you, configures using `emconfigure`, and passes the right flags to `hadrian`. Notice that it calls `build-stack`, which you may want to change to simply `build` if you are not using `stack`.
|
|
|
Build instructions can be found on [javascript-backend/building](https://gitlab.haskell.org/ghc/ghc/-/wikis/javascript-backend/building).
|
|
|
|
|
|
## Further Reading and Demos
|
|
|
|
|
|
- [Say Hello World to JS](https://engineering.iog.io/2022-12-13-ghc-js-backend-merged)
|
|
|
- [Run Haskell to JS code in the browser](https://engineering.iog.io/2023-01-24-javascript-browser-tutorial)
|
|
|
|
... | ... | @@ -210,7 +82,7 @@ We do plan to support `--make` mode. It's just that one task to get there is fir |
|
|
|
|
|
At first it doesn't need to because users (or cabal/stack) can use `-pgmi` and `-opti` command line flags to setup the external interpreter to use.
|
|
|
|
|
|
In the longer term, it would probably be better for the external interpreter to be configured per target in a settings file. Probably something to discuss in the context of [#19877](https://gitlab.haskell.org/ghc/ghc/-/issues/19877).
|
|
|
In the longer term, it would probably be better for the external interpreter to be configured per target in a settings file. Probably something to discuss in the context of [#19877](https://gitlab.haskell.org/ghc/ghc/-/issues/19877 "Add support for several toolchains").
|
|
|
|
|
|
### What needs to happen to put proper JS backend support in Cabal? Is upstream aware of this plan? Presumably `c-sources` support will require emscripten support in both Cabal and GHC?
|
|
|
|
... | ... | @@ -224,20 +96,26 @@ Need input from Luite about c-sources support with emscripten. We should also en |
|
|
|
|
|
The current Template Haskell support in the standalone GHCJS has two issues
|
|
|
|
|
|
* It uses a non-standard Template Haskell runnner, GHC gained its own `iserv` external interpreter in the meantime
|
|
|
* The GHCJS TH runner links JavaScript in the compiler, then sends it to the Template Haskell runner (`thrunner.js`) as a message. This occasionally runs into size limitations in node.js
|
|
|
```
|
|
|
* It uses a non-standard Template Haskell runnner, GHC gained its own `iserv` external interpreter in the meantime
|
|
|
* The GHCJS TH runner links JavaScript in the compiler, then sends it to the Template Haskell runner (`thrunner.js`) as a message. This occasionally runs into size limitations in node.js
|
|
|
```
|
|
|
|
|
|
A good solution fixes both problems. It has to integrate with iserv, but we still have some room for decisions:
|
|
|
|
|
|
* We could have a full bytecode interpreter running in JavaScript, running most of the code in bytecode.
|
|
|
* We could reuse most of the `iserv` infrastructure, but run compiled JS code (like `-fobject-code`). We'd have to fix the linker to get around the size issue.
|
|
|
```
|
|
|
* We could have a full bytecode interpreter running in JavaScript, running most of the code in bytecode.
|
|
|
* We could reuse most of the `iserv` infrastructure, but run compiled JS code (like `-fobject-code`). We'd have to fix the linker to get around the size issue.
|
|
|
```
|
|
|
|
|
|
## Building Issues and how to fix them
|
|
|
|
|
|
### Configure fails with: sub-word sized atomic operations not available
|
|
|
|
|
|
#### The error output
|
|
|
|
|
|
The error looks like this:
|
|
|
|
|
|
```
|
|
|
[nix-shell:~/programming/haskell/ghc]$ ./boot && ./configure --target=js-unknown-ghcjs
|
|
|
...
|
... | ... | @@ -250,7 +128,8 @@ configure: error: sub-word-sized atomic operations are not available. |
|
|
```
|
|
|
|
|
|
#### The cause and fix
|
|
|
If you got this error you are most likely using `nix` and `ghc.nix`. The error is caused by `emscripten` trying to write to the `emscripten_cache`, but it cannot write because the `nix store` is read-only.
|
|
|
|
|
|
If you got this error you are most likely using `nix` and `ghc.nix`. The error is caused by `emscripten` trying to write to the `emscripten_cache`, but it cannot write because the `nix store` is read-only.
|
|
|
|
|
|
The fix is hacky, we need to provide the `emscripten_cache` to emscripten so that it is mutable. Similarly, setting `EM_CACHE` to the cache in the nix store does not work. The workaround, in nix, is to copy the cache, make it writable and export `EM_CACHE` to point to the cache. This is done in the following lines in `ghc.nix`:
|
|
|
|
... | ... | @@ -262,12 +141,15 @@ cp -Lr ${emscripten}/share/emscripten/cache .emscripten_cache |
|
|
chmod u+rwX -R .emscripten_cache
|
|
|
export EM_CACHE=.emscripten_cache
|
|
|
```
|
|
|
|
|
|
See [this](https://discourse.nixos.org/t/emscripten-tries-to-write-to-nix/15263) post for more.
|
|
|
|
|
|
### JS Backend fails during stage1 with error: symbol memset missing .functype
|
|
|
|
|
|
#### The error output
|
|
|
|
|
|
This will fail during stage1 with:
|
|
|
|
|
|
```
|
|
|
/run/user/1729/ghc3035129_0/ghc_1.s:528:2: error:
|
|
|
error: symbol memset missing .functype
|
... | ... | @@ -278,6 +160,7 @@ This will fail during stage1 with: |
|
|
```
|
|
|
|
|
|
#### The cause and fix
|
|
|
|
|
|
This error is a bug in `llvm` which `emscripten` uses to generate JS through wasm (via `wasm2js`). The fix is to use a more recent version of `llvm` or a patched version, and a more recent version of `emscripten` (at least `3.1.x`). Whichever you choose the llvm you use must include [this](https://github.com/llvm/llvm-project/commit/2368f18eb305ae9d5a4f2110c048e5daf5007992) patch, and that llvm must be the llvm that `emscripten` uses. See also [this](https://github.com/llvm/llvm-project/issues/53712) issue. Thus far this has only been problematic for nix users. If that is you please see [this](https://github.com/NixOS/nixpkgs/pull/182840) PR on nixpkgs. Other distros, such as Ubuntu have worked out of the box due to a patched llvm included in their package manager.
|
|
|
|
|
|
## References
|
... | ... | |