Skip to content
Snippets Groups Projects

Add head.hackage tutorial

Open Add head.hackage tutorial
head-hackage-tutorial into master
Open Ben Gamari requested to merge head-hackage-tutorial into master
Compare
and
1 file
+ 221
− 0
Compare changes
  • Side-by-side
  • Inline
blog/20230215-head-hackage-tutorial.mkd 0 → 100644
+ 221
− 0
---
title: "Using GHC pre-releases with head.hackage: A tutorial"
author: Ben Gamari
date: "2023-02-15"
---
The user testing period for GHC pre-releases is an important part of the GHC
release process, allowing users to see how changes in the compiler will affect
their codebases and catch compiler bugs before they make it into a final
release. However, it has historically been difficult for users to
meaningfully test pre-release compilers against real-world projects due to
interface changes in `base` and related libraries.
The [head.hackage](https://gitlab.haskell.org/ghc/head.hackage) project attempts to
ease this testing process by providing infrastructure and a common place for users
to provide patches for pre-release compilers. This automation includes:
* tools for preparing patches against Hackage packages
* continuous integration testing of the patch-set against GHC prereleases
* automated preparation of a [Hackage overlay
repository](http://ghc.gitlab.haskell.org/head.hackage/) providing patched
packages for convenient use by users
This post will walk through how an end-user can use a GHC pre-release (either
an alpha release or CI snapshot) to build their project using `cabal build`.
# Installing the pre-release
Let's start by installing a GHC pre-release binary distribution. There are
several ways in which one could get such a compiler:
* a proper alpha or release candidate (e.g., via [GHCup](https://www.haskell.org/ghcup/))
* a [self-built](https://gitlab.haskell.org/ghc/ghc/-/wikis/building/hadrian) compiler
* a binary distribution from [CI](https://gitlab.haskell.org/ghc/ghc/commits/master)
* a binary distribution from CI prepared via [ghc-artefact-nix](https://github.com/mpickering/ghc-artefact-nix)
* a compiler from [ghcs.nix](https://gitlab.haskell.org/bgamari/ghcs-nix/)
For the sake of demonstration, we will show usage of a binary distribution from
CI in this post. For this we will use
[ghcs.nix](https://gitlab.haskell.org/bgamari/ghcs-nix/), which is a
convenient option as it is agnostic to the host operating system:
```bash
$ nix run -f https://gitlab.haskell.org/bgamari/ghcs-nix/-/archive/master/ghcs-nix-master.tar.gz ghc-9_6_1-alpha3
$ GHC=$(realpath result/bin/ghc)
$ $GHC --version
The Glorious Glasgow Haskell Compilation System, version 9.1.20210317
$ echo 'main = putStrLn "Hello world!"' > Hello.hs
$ $GHC Hello.hs
$ ./Hello
Hello world!
```
# Building a package against head.hackage
To demonstrate use of `head.hackage`, we will use it to build the handy
`ghc-events-analyze` utility.
First we can start by just checking out `ghc-events-analyze`:
```bash
$ git clone https://github.com/well-typed/ghc-events-analyze
```
We can now configure `cabal` to use the `head.hackage` overlay by adding it to
our project configuration:
```bash
$ echo "with-compiler: $GHC" > cabal.project.local
$ curl https://ghc.gitlab.haskell.org/head.hackage/cabal.project >> cabal.project.local
```
This configuration includes:
* a `repository` stanza pointing `cabal` at the patched packages
* an `allow-newer` field allowing packages to build with the boot libraries provided by the pre-release compiler
* a set of version constraints to ensure that only the patched packages are used
We can then ask `cabal` to fetch overlay repository and build `ghc-events-analyze`:
```bash
$ cabal update
$ cabal build ghc-events-analyze
```
Ideally this would be the end of the story. However, after a bit of
compilation, I encountered the following error when compiling the
`ghc-events` package:
```
Building library for ghc-events-0.13.0..
[ 1 of 10] Compiling GHC.RTS.EventTypes ( src/GHC/RTS/EventTypes.hs, dist/build/GHC/RTS/EventTypes.o, dist/build/GHC/RTS/EventTypes.dyn_o )
src/GHC/RTS/EventTypes.hs:183:59: error:
Operator applied to too few arguments: !
|
183 | sparksRemaining :: {-# UNPACK #-}! Word64
| ^
cabal: Failed to build ghc-events-0.13.0 (which is required by
exe:ghc-events-analyze from ghc-events-analyze-0.2.7). See the build log above
for details.
```
This error arises from the [GHC Proposal
229][proposal-229] introduced in GHC 9.0. While I have since fixed it, this
serves as a great opportunity to walk through the process for submitting a new
patch to `head.hackage`.
[proposal-229]: https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0229-whitespace-bang-patterns.rst)
[head.hackage]: https://gitlab.haskell.org/ghc/head.hackage
# Contributing a patch
Currently the `head.hackage` patch-set includes patches for a relatively small
fraction of Hackage. Consequently, when building a package with a large
transitive dependency closure it is likely that you will need to contribute a
patch of your own. However, we have tried to make this process as painless as
possible in the hope that with low friction will come many helping hands.
To patch `ghc-events` we will start by cloning the `head.hackage` repository:
```bash
$ git clone git@gitlab.haskell.org:ghc/head.hackage
$ cd head.hackage
$ git checkout -b fix-ghc-events
```
Next we fetch and unpack `ghc-events` using the handy `patch-tool` script:
```bash
$ scripts/patch-tool unpack-patch ghc-events
Unpacking to ghc-events-0.13.0/
Initialized empty Git repository in /home/ben/ghc-events-analyze/head.hackage/packages/ghc-events-0.13.0/.git/
Added packages/ghc-events-0.13.0 to cabal.project.local
```
`patch-tool` will fetch the latest version of `ghc-events` from Hackage,
extract it to the `packages/ghc-events` directory and initialize a fresh
`git` repository in that working directory to ensure that we can later easily
generate a patch with our changes.
We can now go ahead and make the necessary changes to `ghc-events` to address
the build errors we previously encountered.
```bash
$ cd packages/ghc-events
$ vim src/GHC/RTS/EventTypes.hs
$ git diff
diff --git a/src/GHC/RTS/EventTypes.hs b/src/GHC/RTS/EventTypes.hs
index f2b1508..541f15d 100644
--- a/src/GHC/RTS/EventTypes.hs
+++ b/src/GHC/RTS/EventTypes.hs
@@ -180,7 +180,7 @@ data EventInfo
}
| SparkCounters { sparksCreated, sparksDud, sparksOverflowed,
sparksConverted, sparksFizzled, sparksGCd,
- sparksRemaining :: {-# UNPACK #-}! Word64
+ sparksRemaining :: {-# UNPACK #-}!Word64
}
| SparkCreate { }
| SparkDud { }
$ git commit -a -m "Fix whitespace before bang"
$ cd ../..
```
We can then use `patch-tool` to add our patch to `head.hackage`:
```bash
$ scripts/patch-tool update-patches
On branch fix-ghc-events
Changes to be committed:
(use "git restore --staged <file>..." to unstage)
new file: ghc-events-0.13.0.patch
$ git commit -m "ghc-events: Fix whitespace before bang"
```
Finally we can push our branch to a personal `head.hackage` fork and open a merge
request with our changes:
```bash
$ git push git@gitlab.haskell.org:bgamari/head.hackage
```
# Continuing our build
Having fixed `ghc-events`, we can return to compiling `ghc-events-analyse`.
While our `head.hackage` merge request is under review we can manually add our
patched tree to the `cabal.project.local` file in our `ghc-events-analyse` tree
and restart the build:
```bash
$ echo 'packages: head.hackage/packages/ghc-events-0.13.0' >> cabal.project.local
$ cabal build ghc-events-analyze
```
We will now find that that the build succeeds.
# Adding a package to CI
In addition to serving as a tool for end-users, `head.hackage` also serves as a
valuable source of programs against which GHC can be tested. We use this, for
instance, in GHC's CI pipeline to assess the extent breakage caused by GHC
merge requests marked with the `user-facing` GitLab label.
In addition to all of the packages having `.patch` files, `head.hackage` CI
also builds a number of "extra" packages defined in
[`ci/config.sh`][config.sh]. These extra packages are intended to include
widely-used packages from across the Haskell ecosystem.
If you would like to see your package in this set by maintaining its presence
in `head.hackage`, feel free to [open a
ticket][open ticket].
[config.sh]: https://gitlab.haskell.org/ghc/head.hackage/-/blob/83ad8179adf601845c406a547c0e585b99996064/ci/config.sh#L99
[open ticket]: https://gitlab.haskell.org/ghc/head.hackage/-/issues/new
# Acknowledgements
Thanks to the `cabal-install` developers for all of their working in putting in
place the pieces that make `head.hackage` possible. Additionally, thanks to
Herbert Valerio Riedel for kick-starting the patch-set and its infrastructure,
Moritz Angermann for his contributions to the repository generation
infrastructure, and Ryan G.L. Scott for his tireless effort in
maintaining the patchset thusfar.