1. 18 Oct, 2019 5 commits
    • Ben Gamari's avatar
      ceeaf488
    • Ben Gamari's avatar
      gitlab-ci: Use cabal's build parallelism · f82bb0bc
      Ben Gamari authored
      f82bb0bc
    • Ben Gamari's avatar
      ci: Introduce --cabal-option flag · f2b9b3f8
      Ben Gamari authored
      f2b9b3f8
    • Ben Gamari's avatar
      gitlab-ci: Clean products of previous runs · 2e98fcb4
      Ben Gamari authored
      2e98fcb4
    • Ben Gamari's avatar
      A new CI story · 1b616c6c
      Ben Gamari authored
      The Problem
      -----------
      
      When I started looking at the problem of providing CI for `head.hackage`
      I considered two possible designs:
      
       1. Build upon `cabal-install`
       2. Build upon Nix's Haskell infrastructure
      
      While I preferred (1), I found that integrating with `cabal-install` was
      quite difficult:
      
       * it [does not produce
         logs](https://github.com/haskell/cabal/issues/5901) for local packages,
         which was the obvious way to incorporate patched packages into the
         build plan
      
       * it is difficult to reconstruct why a package build failed (e.g. due
         to a planning failure, dependency failing to build, or an error in
         the package itself)
      
      For these reasons it so happened that (2) ended up being a tad easier to
      implement. However, it suffers from a number of problems:
      
       * Nix's Haskell infrastructure doesn't handle multiple versions of a
         single package at all, yet we now have patches for multiple package
         versions in `head.hackage`
      
       * Nix's Haskell infrastructure doesn't handle flags, which can
         complicate building some packages
      
       * The Nix expressions ended up being rather difficult to maintain
      
      The Solution
      ------------
      
      This MR moves the CI infrastructure back in the direction of (1),
      facilitated by workarounds that I found for the two issues described
      above.
      
      The infrastructure revolves around the `head-hackage-ci` executable
      which provides a `test-patches` mode which `gitlab-ci.yml` invokes
      thusly:
      ```
      head-hackage-ci test-patches --patches=./patches --with-compiler=$GHC
      ```
      This mode does several things:
      
       1. Builds a local package repository (using the same script used to
          build `https://ghc.gitlab.haskell.org/head.hackage/`). (N.B. by
          pulling patched packages from a proper repository instead of using
          local packages we side-step the fact that `cabal-install` doesn't
          produce logs for local packages)
       2. Generate a `cabal.project` file containing:
      
          * a `remote-repository` stanza referring to this repository
          * constraints to ensure that we only choose patched package versions
          * some additional `package` stanzas to ensure that `Cabal` can find
            native library dependencies (these are defined in
            `ci/build-deps.nix`)
      
       3. Run `cabal new-update` (as well as perform a dummy build of the
          `acme-box` package to ensure that the package index cache is built,
          otherwise parallel builds can randomly fail)
       4. For each patched create a new working directory containing:
      
          * the previously generated `cabal.project` file
          * a `test-$PKGNAME.cabal` file defining a dummy package depending
            upon the library
      
          and perform the build. We use some heuristics depending upon:
      
          * the `plan.json` file
          * which log files exist
          * the contents of said log files
      
          to sort out what happened.
       6. After all the packages have been built produce a final report of the
          result.
      
      While this is admittedly pretty hacky, in truth it's no worse than the
      somersaults which we had to perform in the Nix infrastructure. Reliably
      introspecting on failed builds seems to be messy business no matter
      which build system you use.
      1b616c6c
  2. 17 Oct, 2019 5 commits
  3. 16 Oct, 2019 1 commit
  4. 08 Oct, 2019 1 commit
  5. 02 Oct, 2019 2 commits
  6. 30 Sep, 2019 1 commit
    • Ryan Scott's avatar
      Old patch cleanup, migration · f7d0cf4c
      Ryan Scott authored
      * The patches for `asn1-encoding`, `control-monad-omega`,
        `equivalence`, `hedgehog`, `language-c`, `socks`, `streaming`,
        `turtle`, and `vector-th-unbox` are no longer needed, as the latest
        Hackage versions support GHC 8.8 and HEAD.
      * The patches for `regex-pcre-builtin`, `safecopy`,
        `skylighting-core`, and `texmath` have been migrated to the latest
        Hackage versions.
      * A patch for `vinyl-0.12.0` has been added. This is mostly the same
        as the patch for `vinyl-0.11.0`, except that fewer changes are
        needed, since `vinyl-0.12.0` supports GHC 8.8.
      f7d0cf4c
  7. 26 Sep, 2019 1 commit
  8. 25 Sep, 2019 1 commit
  9. 13 Sep, 2019 2 commits
  10. 10 Sep, 2019 1 commit
  11. 08 Sep, 2019 2 commits
  12. 07 Sep, 2019 1 commit
  13. 06 Sep, 2019 2 commits
  14. 01 Sep, 2019 1 commit
  15. 28 Aug, 2019 2 commits
    • Ryan Scott's avatar
      GHC 8.9-related patches for barbies, inspection-testing, shakespeare, yesod-core · 1530228b
      Ryan Scott authored
      * `barbies` must adapt to GHC proposal 103
        (https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0103-no-kind-vars.rst),
        which has been implemented in 8.9.
      * `inspection-testing` must adapt to changes in the GHC API.
      * `shakespeare` and `yesod-core` must adapt to the type of `TupE`
        changing in `template-haskell-2.16.0.0`.
      1530228b
    • Ryan Scott's avatar
      Migrate/remove old patches · 673821b9
      Ryan Scott authored
      * Migrate the patches for `doctest`, `free`, `persistent`, and `yesod-core`
        to the latest versions on Hackage.
      * Remove the patches for `Cabal`, `bytes`, `cabal-doctest`, `ghc-paths`,
        `hsc2hs`, `polyparse`, `singleton-nats`, `tar`, `yaml`, and `zlib`, as
        more recent versions of these packages (that support GHC 8.8 and HEAD)
        are now available on Hackage.
      
      I opted not to remove the `th-desugar-1.9`/`singletons-2.5.1` patches for
      now, even though they are not the latest Hackage releases, since upgrading
      them can be a nontrivial process (similar to the `primitive-0.6`/`-0.7`
      situation).
      673821b9
  16. 08 Aug, 2019 3 commits
  17. 02 Aug, 2019 3 commits
  18. 30 Jul, 2019 1 commit
    • Ryan Scott's avatar
      Fix #2 · 758d23c1
      Ryan Scott authored
      This fixes #2 by tweaking `scripts/head.hackage` in two ways:
      
      1. This changes the script to use
         `cabal new-update head.hackage.ghc.haskell.org`, not
         `cabal new-update head.hackage`, since the former is now the name
         of the `repository`. Since `head.hackage.ghc.haskell.org` is
         referred to in quite a few places in the script, I factored this
         out into its own variable.
      2. This changes the `url` to use `http://`, not `https://`, so that
         `http-transport: plain-http` works properly. An alternative would
         be to keep the use of `https://` and use `http-transport: curl`
         instead, but that assumes the existence of a `curl` binary on
         one's machine, making it a less portable solution. In any case,
         the use of `http://` in this script shouldn't pose any security
         issues, since `hackage-security` already introduces a security
         layer independent of CAs.
      758d23c1
  19. 29 Jul, 2019 1 commit
  20. 27 Jul, 2019 2 commits
  21. 24 Jul, 2019 2 commits