Test Plan: Observe CircleCI

Reviewers: bgamari

Reviewed By: bgamari

Subscribers: rwbarton, carter

Differential Revision: https://phabricator.haskell.org/D5464

Might as well make sure it works before we retire it.

As discussed on ghc-devops, CircleCI is changing their billing policy to require
users of instances larger than `medium` to use a paid account. Consequently
these instances will no longer be available to us starting tomorrow.

These are taken from our previous arcanist linters as well as the
gitolite hooks but with some heavy refactoring.

Our previous attempt ended up breaking the CircleCI config, but
this one has been tested, see:

  https://circleci.com/gh/ghc/ghc/tree/alp%2Fcircleci%2Fignore-gitlab-branches

which shows the builds getting triggered correctly. I also pushed the same
branch under 'gitlab/alp/test', and that didn't trigger any build, as desired.

Reviewers: bgamari

Subscribers: rwbarton, carter

Differential Revision: https://phabricator.haskell.org/D5429

Fixes #15915.

It's currently too out-of-date to build current hadrian and is arguably
completely broken anyways (see #15794).

This was added (for good reasons) in b9260e92
but it seems this field isn't allowed in this context. Sadly there doesn't
appear to be an easy way to accomplish what we are after with CircleCI's rather
restrictive configuration language. Sigh.

Alp Mestanogullari authored 6 years ago and Ben Gamari committed 6 years ago

And put it to use for running i386 and x86_64 linux validate,
x86_64 darwin validate and building with hadrian. For all the validate
jobs, we save the artifacts (bindist + test results for now) in Circle CI
and subsequently grab them in Gitlab. The nice side effect is that merge
requests now show the change in test results from the patches and let you
download or browse the artifacts.

Dario Bertini authored 6 years ago and Ben Gamari committed 6 years ago

Fixes #15404.

This patch makes the following improvement:
  - Automatically records test metrics (per test environment) so that
    the programmer need not supply nor update expected values in *.T
    files.
    - On expected metric changes, the programmer need only indicate the
      direction of change in the git commit message.
  - Provides a simple python tool "perf_notes.py" to compare metrics
    over time.

Issues:
  - Using just the previous commit allows performance to drift with each
    commit.
    - Currently we allow drift as we have a preference for minimizing
      false positives.
    - Some possible alternatives include:
      - Use metrics from a fixed commit per test: the last commit that
        allowed a change in performance (else the oldest metric)
      - Or use some sort of aggregate since the last commit that allowed
        a change in performance (else all available metrics)
      - These alternatives may result in a performance issue (with the
        test driver) having to heavily search git commits/notes.
  - Run locally, performance tests will trivially pass unless the tests
    were run locally on the previous commit. This is often not the case
    e.g.  after pulling recent changes.

Previously, *.T files contain statements such as:
```
stats_num_field('peak_megabytes_allocated', (2, 1))
compiler_stats_num_field('bytes allocated',
                         [(wordsize(64), 165890392, 10)])
```
This required the programmer to give the expected values and a tolerance
deviation (percentage). With this patch, the above statements are
replaced with:
```
collect_stats('peak_megabytes_allocated', 5)
collect_compiler_stats('bytes allocated', 10)
```
So that programmer must only enter which metrics to test and a tolerance
deviation. No expected value is required. CircleCI will then run the
tests per test environment and record the metrics to a git note for that
commit and push them to the git.haskell.org ghc repo. Metrics will be
compared to the previous commit. If they are different by the tolerance
deviation from the *.T file, then the corresponding test will fail. By
adding to the git commit message e.g.
```
 # Metric (In|De)crease <metric(s)> <options>: <tests>
Metric Increase ['bytes allocated', 'peak_megabytes_allocated'] \
         (test_env='linux_x86', way='default'):
    Test012, Test345
Metric Decrease 'bytes allocated':
    Test678
Metric Increase:
    Test711
```
This will allow the noted changes (letting the test pass). Note that by
omitting metrics or options, the change will apply to all possible
metrics/options (i.e. in the above, an increase for all metrics in all
test environments is allowed for Test711)

phabricator will use the message in the description

Reviewers: bgamari, hvr

Reviewed By: bgamari

Subscribers: rwbarton, carter

GHC Trac Issues: #12758

Differential Revision: https://phabricator.haskell.org/D5059

This reverts commit 406978c4.

Fixes #15404.

(cherry picked from commit 578012be)

Alp Mestanogullari authored 6 years ago and Krzysztof Gogolewski committed 6 years ago

Summary:
Since the LLVM backend is slow enough that *slowtest can't run
in the allocated time on Circle CI, let's just use *test. If that still
doesn't fit in the allocated 5 hours, we can try running just the
compiler tests.

Test Plan: Circle CI validation for LLVM

Reviewers: bgamari

Reviewed By: bgamari

Subscribers: rwbarton, thomie, carter

Differential Revision: https://phabricator.haskell.org/D5033

Ben Gamari authored 6 years ago and Krzysztof Gogolewski committed 6 years ago

Summary: This was a cut-and-paste error.

Reviewers: alpmestan

Reviewed By: alpmestan

Subscribers: alpmestan, rwbarton, thomie, carter

GHC Trac Issues: #15466

Differential Revision: https://phabricator.haskell.org/D5037

Ben Gamari authored 6 years ago and Ben Gamari committed 6 years ago

Test Plan: Try `./validate`, CircleCI build; make sure core count
detection works in both cases.

Reviewers: alpmestan

Reviewed By: alpmestan

Subscribers: rwbarton, thomie, carter

GHC Trac Issues: #14470

Differential Revision: https://phabricator.haskell.org/D4897

This seems to fix a number of segmentation faults.

Lest we end up with a non-functional user/group lookup, resulting in #15230.

The ‘haskell’ images are not being updated and there is no image with GHC
8.4.2, so we probably should not use them anymore. I adapted Dockerfiles
used by those images so the end result is almost the same except we get
newer GHC and cabal-install.

Reviewers: mrkkrp, bgamari

Reviewed By: mrkkrp, bgamari

Subscribers: thomie, carter

GHC Trac Issues: #15093

Differential Revision: https://phabricator.haskell.org/D4646

Since for every job there are failing tests, if we try to upload artifacts
after tests, we won't get any. As a temporary measure I suggest uploading
artifacts before running tests. Once tests are fixed, we should peform
artifact uploading after tests again.