Overlay Hackage Package Index for GHC HEAD

How to contribute

Submit PRs with patch(es) relative to the source tarball(s) of existing Hackage package(s).

• The patches MUST apply cleanly by patch -p1 when inside the original unpacked source-tarball. (CI will verify this when you submit a PR).

• The patches SHOULD work with at least GHC HEAD and a set of recent stable released GHC versions (currently this means with GHC 9.4, 9.6, 9.8, and 9.9).

• The patches SHOULD ideally result in the same code being compiled, as one of the main purposes of these patches is to make regression testing possible. I.e. try to avoid conditional compilation.

How this works

This repo contains <pkg-id>.patch files in the patches/ folder (where <pkg-id> refers to a specific release of a package, e.g. lens-4.15.3).

Once merged to master, all package releases whose <pkg-id> is mentioned will enter the HEAD.hackage package index; if there is a .patch file, the respective releases tarballs are patched (i.e. mutated!).

If this operation succeeds, the HEAD.hackage package index at http://HEAD.hackage.haskell.org/ is updated to contain the new index state.

HEAD.hackage contains only a small subset of package releases, and needs to be combined with the main Hackage repository.

Cabal's new nix-style local build facility makes sure that the modified packages don't contaminate your package databases, while allowing to maximise sharing via the nix-style package-db cache store.

How to use

If you know what you are looking for, here it is:

repository head.hackage.ghc.haskell.org
   url: https://ghc.gitlab.haskell.org/head.hackage/
   secure: True
   key-threshold: 3
   root-keys:
       f76d08be13e9a61a377a85e2fb63f4c5435d40f8feb3e12eb05905edb8cdea89
       26021a13b401500c8eb2761ca95c61f2d625bfef951b939a8124ed12ecf07329
       7541f32a4ccca4f97aea3b22f5e593ba2c0267546016b992dfadcd2fe944e55d

active-repositories: hackage.haskell.org, head.hackage.ghc.haskell.org:override

The use of :override forces cabal's constraint solver to pick versions of libraries that have corresponding patches in head.hackage whenever possible. This may or may not be what you want depending on your use case. If you wish to permit cabal to choose build plans that include different versions of libraries than what are patched in head.hackage, skip the :override:

active-repositories: hackage.haskell.org, head.hackage.ghc.haskell.org

Also see https://cabal.readthedocs.io/en/latest/cabal-project.html#cfg-field-active-repositories.

HEAD.hackage doesn't bump the bounds of boot packages + certain other packages to avoid the busywork of bumping them. When using HEAD.hackage, you should use --allow-newer for these packages. The full list is here.

As an add-on remote repository

It is not recommended to add the HEAD.hackage repository index to your global cabal configuration.

Instead, you should mix in the HEAD.hackage repository on a per-project level. Then the packages in the HEAD.hackage will overlay those from the main package index, by adding the repository stanza (as shown on https://ghc.gitlab.haskell.org/head.hackage/) to the cabal.project(.local) file or use head.hackage.sh init (see below).

To workaround some current issues in cabal and make it more convenient, the script scripts/head.hackage.sh is provided, which facilitates common tasks.

It's been tested on Linux so far. Other operating systems may require tweaks (patches welcome!).

The main operations provided are

• head.hackage.sh update: Resets & syncs the local package download cache for the HEAD.hackage repo.

• head.hackage.sh init: generates a new cabal.project file with a repository stanza enabling the HEAD.hackage repo locally. This command also takes an optional list of arguments which are included as optional-packages: declarations in the resulting cabal.project file.

• head.hackage.sh init-local: generate a cabal.project.local file instead.

• head.hackage.sh dump-repo: print repository stanza to stdout

As an add-on local repository

The HEAD.hackage package repo can also be generated as a file-based local repository. The handling is similiar to using HEAD.hackage via a remote repo.

TODO: provide scripting

As locally patched packages

The process of applying patches can be used in a cabal project with local packages.

You can add something like optional-packages: */*.cabal to your cabal.project file, and then for each package-id with a .patch or .cabal file you want to provide as a locally patched package do

$ cabal unpack --pristine $PKGID
$ cd $PKGID/
$ patch -p1 -i ${WhereThisGitHubRepoIsCloned}/patches/$PKGID.patch
$ cp ${WhereThisGitHubRepoIsCloned}/patches/$PKGID.cabal ./*.cabal
$ cd ..

Alternatively, you can use the handy patch-tool utility:

$ scripts/patch-tool unpack-patch patches/$PKGID.patch

This will extract the given package into the packages/$PKGID directory, initialize it as a git repository, and the patch.

Adding a patch

The scripts/patch-tool script is a tool for conveniently authoring and updating patches. For example, to patch the doctest package, you can run the following steps:

1. scripts/patch-tool unpack doctest
2. Modify files in packages/doctest-$version/ as necessary
3. Build/test as normal, e.g. cabal build doctest
4. scripts/patch-tool update-patches
5. Commit the patch

GitLab CI

GHC's GitLab instance uses GitLab CI and the head-hackage-ci tool (contained in the ci/ directory) to test the head.hackage patchset against GHC releases and snapshots. It can also compile head.hackage using a patch to GHC; just add the user-facing label to a GHC MR, and the existing CI infrastructure will invoke head.hackage.

To run a similar build locally start by downloading and installing a binary distribution appropriate for your distribution and then call the run-ci script:

$ export GHC=/path/to/my/ghc
  # enable Core Linting for extra correctness assurance...
$ export EXTRA_HC_OPTS=-dcore-lint
$ ./run-ci

This will build all packages having patches and produce a textual summary, as well as a JSON file (result.json) describing the outcome.

If you are using nix you can run:

nix-shell ci/ --command run-ci

Note that we currently rely on IOG's Hydra instance for caching of flake outputs to ensure that they aren't rebuilt with every job.

Hackage repository

GHC's GitLab instance uses GitLab CI to deploy a Hackage repository with the patches provided by head.hackage. See the repository for usage instructions.

Travis CI

The Travis CI script generator has recently added support for enabling the HEAD.hackage repository automatically for jobs using unreleased GHC versions.

Nix(Os)

The patches maintained for the head.hackage project can also be used with Nix (not to be confused with Cabal's Nix-style local builds). See the README in the script/ folder and/or the Using a development version of GHC with nix blogpost for more information.

Other Package Index Overlays

• Experimental mobile haskell hackage overlay
• GHCJS package overlay for Hackage
• Hackage Package Candidate Overlay