GitLab

Convert hadrian buildscripts to unsuffixed, dashed form

final cleanups

This should fix #17861.

The fact that `exec` isn't POSIX compliant means that things can break
in arbitrarily bad ways. Sometimes things happen to work correctly but
sadly this isn't always the case.

Due to the resistance of #17736 to resolution.

This moves nearly all of the CI logic to .gitlab/ci.sh. This improves
things in a number of ways:

 * it's harder for inconsistencies to arise between architectures
 * it's easier to share logic between architectures
 * on Windows, it's easier to ensure that all CI steps are executed from
   within a properly initialized mingw session.

While in town I also add a FreeBSD build job and update the Windows job
to use the gitlab-runner PowerShell executor, since cmd.exe will be
deprecated soon (fixing #17699).

This commit implements a few Windows-specific fixes which get us from a CI
job that can't even get as far as starting the testsuite driver, to a state
where we can run the entire testssuite (but have test failures to fix).

- Don't forget about a potential extension for the haddock program, when
  preparing the bindist.
- Build the timeout program, used by the testsuite driver on Windows in place
  of the Python script used elsewhere, using the boot compiler. We could
  alternatively build it with the compiler that we're going to test but this
  would be a lot more tedious to write.
- Implement a wrapper-script less installation procedure for Windows, in
  `hadrian/bindist/Makefile.
- Make dependencies a bit more accurate in the aforementioned Makefile.
- Update Windows/Hadrian CI job accordingly.

This patch fixes #17486.

Due to T7702 and the process issues described in #17777.

Currently CI is inexplicably failing with
```
$ git submodule foreach git clean -xdf
fatal: not a git repository: libffi-tarballs/../.git/modules/libffi-tarballs
```
I have no idea how this working tree got into such a state but we do
need to fail more gracefully when it happens. Consequently, we allow the
cleaning step to fail.

Not all runners have symlink permissions enabled.

Previously we would keep them for two weeks. However, on the stable
branches two weeks can easily elapse with no pushes.

Otherwise the testsuite driver may not have an up-to-date
baseline.

In addition, we prefer the Mingw64 Python distribution on Windows due
to #17483.

Also eliminate some redundancy.

The python release shipped with deb8 (3.3) is too old for our testsuite
driver.

Variable interpolation in gitlab-ci.yml apparently doesn't work. Sigh.

This sacrifices some precision in favor of improving parallelism.

Allowing it to be easily used locally.

(cherry picked from commit 67b5de88ef923971f1980335137e3c7193213abd)

(cherry picked from commit cbedb3c4a90649f474cb716842ba53afc5a642ca)

Apparently variable interpolation in the `artifacts.paths` key of
`gitlab-ci.yml` doesn't work on Windows as it does on WIndows.

(cherry picked from commit 100cc756)

It otherwise seems to default to ascii

Fall back to xz

Installs pxz on Centos7

(cherry picked from commit 86960e69)

The docutils version available appears to be too old to support the
`table` directive's `:widths:` options.

(cherry picked from commit 75764487)