Implementers: Matthew Pickering, Cheng Shao, Rodrigo Mesquita ## Motivation GHC’s bytecode interpreter is widely used in GHCi and for Template Haskell evaluation with (`-fprefer-bytecode`), but until now it has only ever generated transient, in-memory bytecode. Every time you load a module in GHCi, the bytecode is regenerated from source. The primary advantage of making bytecode into a persistable artifact is allowing caching of bytecode for REPL and tooling use. This is not intended to replace native code for production executables, native code remains faster for runtime performance, but rather to extend GHC’s capabilities where fast build times, portability, and tooling integration are more important than peak execution speed. ## Implementation Plan This change introduces a new module build way that allows you to compile Haskell code into persistent bytecode artifacts instead of native object code. - In GHC: new `-fwrite-bytecode` flag: - When the bytecode backend runs (`-fbyte-code` or `-fbyte-code-and-object-code`), also write `.gbc` files (bytecode objects) to disk. - Disabled in GHCi, so interactive sessions remain transient (by default). - Works with both bytecode-only and bytecode+object builds. - Has no effect if no bytecode backend is active. - New option to create a bytecode library format for bundling together bytecode for packages. - In `ghc-pkg`: - New field `bytecode-library-dirs` to register and discover bytecode artifacts. - Package databases can store both native and bytecode artifacts for the same package. - In Cabal: - New `--enable-bytecode` build option. - Installed bytecode libraries are registered with `ghc-pkg` using the new fields. This extends GHC’s existing bytecode generator (previously used only in GHCi) into a first-class, persistent build mode, enabling packaging and reuse of bytecode across builds and environments. ### Bytecode Object format * The Bytecode object format is a serialised version of `CompiledByteCode`, the format is created using GHC's binary serialisation framework. * Bytecode objects are specific to one version of GHC. ### Bytecode Library Format * The `-bytecodelib` option instructs GHC to create a bytecode library. * Like for other library parts, you can either create one from in-memory bytecode objects (as the final part of --make mode), or more commonly, by passing a collection of .gbc files on the command line. * The bytecode library format is the concatentation of the bytecode objects for each module in a package which provides them. * The compiler can load the bytecode for an external package and use bytecode in order to evaluate TH splices if `-fprefer-bytecode` is being used. * TODO: What is a module doesn't produce a .gbc file? ``` ghc -bytecodelib A.gdc B.gbc C.gbc -o lib.bytecode > Produces a bytecode library called lib.bytecode containing A B and C. ``` #### Bytecode Library Naming Convention * `ghc` expects the bytecode library to follow the same naming convention as a static archive but using the `bytecode` suffix. ("lib<pkg-id>.bytecode") * `ghc` looks in `bytecode-library-dirs` to find `*.bytecode` libraries for package dependencies. # Interaction with existing flags | Combination | Result | |-------------|--------| | `-fbyte-code` | Bytecode backend only; transient bytecode. | | `-fbyte-code -fwrite-bytecode` | Bytecode backend only; persistent `.gbc` files. | | `-fbyte-code-and-object-code` | Native + bytecode backends; Implies `-fwrite-byte-code`. | | `-fobject-code` | Native backend only; no `.gbc` files even with `-fwrite-bytecode`. | | In GHCi + `-fwrite-bytecode` | Write bytecode artifacts whilst using GHCi (like `-fwrite-interface`). | ## Interaction with “Interface Files with Core Definitions” GHC’s `-fwrite-if-simplified-core` feature (introduced in GHC 9.6) allows storing simplified Core in `.hi` files. This is described in blog post [Interface Files with Core](https://well-typed.com/blog/2023/02/interface-files-with-core/). Historically, the loading time of bytecode has been reduced by the ability to use these core definitions as the starting point rather than starting the whole compilation pipeline again. Introducing a proper bytecode artifact means that we don't have to rely on this step anymore.