GHC issues
https://gitlab.haskell.org/ghc/ghc/-/issues
2024-02-13T15:23:02Z
https://gitlab.haskell.org/ghc/ghc/-/issues/24420
CI: Intermittent "Command failed with error code: -4" on Darwin.
2024-02-13T15:23:02Z
Bryan R
bryan@haskell.foundation
CI: Intermittent "Command failed with error code: -4" on Darwin.
### Recent occurrences:
| json->>'web_url' | json->>'status' | json->>'ref' | json->>'name' | json->>'runner.description' | date(json->>'created_at') |
|------...
### Recent occurrences:
| json->>'web_url' | json->>'status' | json->>'ref' | json->>'name' | json->>'runner.description' | date(json->>'created_at') |
|---------------------------------------------------|-----------------|--------------------------------|------------------------|----------------------------------------------|---------------------------|
| https://gitlab.haskell.org/ghc/ghc/-/jobs/1772620 | failed | refs/merge-requests/11968/head | x86_64-darwin-validate | m1-delta_Mini18-Delta_f87b8a204711 | 2024-02-07 |
| https://gitlab.haskell.org/ghc/ghc/-/jobs/1767573 | failed | refs/merge-requests/11576/head | x86_64-darwin-validate | aarch64-darwin-1_Mini18-Beta_f87b8a204711 | 2024-02-01 |
| https://gitlab.haskell.org/ghc/ghc/-/jobs/1767305 | failed | refs/merge-requests/11976/head | x86_64-darwin-validate | aarch64-darwin-1_Mini18-Epsilon_004e0928d81c | 2024-02-01 |
| https://gitlab.haskell.org/ghc/ghc/-/jobs/1764568 | failed | refs/merge-requests/11989/head | x86_64-darwin-validate | aarch64-darwin-1_Mini18-Beta_f87b8a204711 | 2024-01-29 |
| https://gitlab.haskell.org/ghc/ghc/-/jobs/1753166 | failed | refs/merge-requests/11923/head | x86_64-darwin-validate | aarch64-darwin-1_Mini18-Beta_f87b8a204711 | 2024-01-16 |
### Analysis
* Signal 4 is "Illegal instruction", so maybe Rosetta is involved.
* The (truncated) command lines that failed in these occurences are:
* `/nix/store/sbfhs3rv53vgh3cn46q900fmgsl5q4x4-ghc/bin/ghc-pkg --no-user-package-db field filepath depends`
* `_build/stage0/bin/ghc -Wall -Wcompat -fdiagnostics-color=never -dynamic-too -hisuf hi -osuf o -hcsuf <SNIP>`
* `_build/stage0/bin/ghc -Wall -Wcompat -hisuf dyn_hi -osuf dyn_o -hcsuf dyn_hc -fPIC -dynamic <SNIP>`
* `/nix/store/sbfhs3rv53vgh3cn46q900fmgsl5q4x4-ghc/bin/ghc-pkg --expand-pkgroot --no-user-package-db describe stm`
* `/usr/bin/gcc --target=x86_64-apple-darwin -Qunused-arguments -E -MM -MG -MF _build/stage1/rts/build/c/sm/NonMovingScav.thr_debug_o.d <SNIP>`
So if it managed to affect nix-packaged software, our own built software, *and* `/usr/bin/gcc`, I feel it has to be a system issue. Unfortunately it also seems to affect multiple machines.
### Tracking link
https://grafana.gitlab.haskell.org/d/167r9v6nk/ci-spurious-failures?orgId=2&refresh=15m&from=now-30d&to=now&var-types=code_-4&var-runners=All&var-jobs=All
https://gitlab.haskell.org/ghc/ghc/-/issues/24417
Intermittent "wasm32-wasi-ghc-pkg: createProcess: exec: invalid argument (Bad...
2024-02-13T14:29:10Z
Bryan R
bryan@haskell.foundation
Intermittent "wasm32-wasi-ghc-pkg: createProcess: exec: invalid argument (Bad file descriptor)"
The error seems to happen when running `wasm32-wasi-ghc-pkg recache`.
Occurrences in the last 3-4 weeks:
| json->>'web_url' | json->>'status' | json->>'ref' | ...
The error seems to happen when running `wasm32-wasi-ghc-pkg recache`.
Occurrences in the last 3-4 weeks:
| json->>'web_url' | json->>'status' | json->>'ref' | json->>'name' | json->>'runner.description' | date(json->>'created_at') |
|---------------------------------------------------|-----------------|--------------------------------|----------------------------------------------------------------------------------------|--------------------------------------------------|---------------------------|
| https://gitlab.haskell.org/ghc/ghc/-/jobs/1772088 | failed | master | x86_64-linux-alpine3_17-wasm-cross_wasm32-wasi-release+fully_static | x86_64-linux-5.zw3rk_x86_64-linux-5_db038e4e1623 | 2024-02-07 |
| https://gitlab.haskell.org/ghc/ghc/-/jobs/1771720 | failed | refs/merge-requests/11400/head | x86_64-linux-alpine3_17-wasm-cross_wasm32-wasi-release+fully_static | x86_64-linux-7.zw3rk_x86_64-linux-7_db038e4e1623 | 2024-02-06 |
| https://gitlab.haskell.org/ghc/ghc/-/jobs/1771185 | failed | refs/merge-requests/12028/head | x86_64-linux-alpine3_17-wasm-cross_wasm32-wasi-release+fully_static | x86_64-linux-7.zw3rk_x86_64-linux-7_db038e4e1623 | 2024-02-06 |
| https://gitlab.haskell.org/ghc/ghc/-/jobs/1767896 | failed | refs/merge-requests/11551/head | x86_64-linux-alpine3_17-wasm-cross_wasm32-wasi-release+fully_static | x86_64-linux-7.zw3rk_x86_64-linux-7_db038e4e1623 | 2024-02-02 |
| https://gitlab.haskell.org/ghc/ghc/-/jobs/1767533 | failed | master | x86_64-linux-alpine3_17-wasm-cross_wasm32-wasi-release+fully_static | x86_64-linux-6.zw3rk_x86_64-linux-6_db038e4e1623 | 2024-02-01 |
| https://gitlab.haskell.org/ghc/ghc/-/jobs/1767370 | failed | refs/merge-requests/11551/head | x86_64-linux-alpine3_17-wasm-cross_wasm32-wasi-release+fully_static | x86_64-linux-5.zw3rk_x86_64-linux-5_db038e4e1623 | 2024-02-01 |
| https://gitlab.haskell.org/ghc/ghc/-/jobs/1767029 | failed | master | x86_64-linux-alpine3_17-wasm-cross_wasm32-wasi-release+fully_static | x86_64-linux-2.zw3rk_x86_64-linux-2_d292a225d2eb | 2024-02-01 |
| https://gitlab.haskell.org/ghc/ghc/-/jobs/1766313 | failed | refs/merge-requests/11976/head | x86_64-linux-alpine3_17-wasm-cross_wasm32-wasi-release+fully_static | x86_64-linux-7.zw3rk_x86_64-linux-7_db038e4e1623 | 2024-01-31 |
| https://gitlab.haskell.org/ghc/ghc/-/jobs/1765681 | failed | refs/merge-requests/11551/head | x86_64-linux-alpine3_17-wasm-cross_wasm32-wasi-release+fully_static | x86_64-linux-3.zw3rk_x86_64-linux-3_db038e4e1623 | 2024-01-30 |
| https://gitlab.haskell.org/ghc/ghc/-/jobs/1765496 | failed | refs/merge-requests/10417/head | x86_64-linux-alpine3_17-wasm-cross_wasm32-wasi-release+fully_static | x86_64-linux-3.zw3rk_x86_64-linux-3_db038e4e1623 | 2024-01-30 |
| https://gitlab.haskell.org/ghc/ghc/-/jobs/1764875 | failed | master | x86_64-linux-alpine3_17-wasm-cross_wasm32-wasi-release+fully_static | x86_64-linux-7.zw3rk_x86_64-linux-7_db038e4e1623 | 2024-01-29 |
| https://gitlab.haskell.org/ghc/ghc/-/jobs/1762230 | failed | master | nightly-x86_64-linux-alpine3_17-wasm-cross_wasm32-wasi-release+fully_static | x86_64-linux-3.zw3rk_x86_64-linux-3_db038e4e1623 | 2024-01-26 |
| https://gitlab.haskell.org/ghc/ghc/-/jobs/1761309 | failed | master | x86_64-linux-alpine3_17-wasm-cross_wasm32-wasi-release+fully_static | x86_64-linux-2.zw3rk_x86_64-linux-2_d292a225d2eb | 2024-01-25 |
| https://gitlab.haskell.org/ghc/ghc/-/jobs/1761061 | failed | refs/merge-requests/11978/head | x86_64-linux-alpine3_17-wasm-cross_wasm32-wasi-release+fully_static | x86_64-linux-7.zw3rk_x86_64-linux-7_db038e4e1623 | 2024-01-24 |
| https://gitlab.haskell.org/ghc/ghc/-/jobs/1756116 | failed | master | nightly-x86_64-linux-alpine3_17-wasm-int_native-cross_wasm32-wasi-release+fully_static | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-19 |
| https://gitlab.haskell.org/ghc/ghc/-/jobs/1755330 | failed | master | nightly-x86_64-linux-alpine3_17-wasm-int_native-cross_wasm32-wasi-release+fully_static | x86_64-linux-2.zw3rk_x86_64-linux-2_d292a225d2eb | 2024-01-18 |
| https://gitlab.haskell.org/ghc/ghc/-/jobs/1754127 | failed | refs/merge-requests/11930/head | x86_64-linux-alpine3_17-wasm-cross_wasm32-wasi-release+fully_static | x86_64-linux-7.zw3rk_x86_64-linux-7_db038e4e1623 | 2024-01-17 |
| https://gitlab.haskell.org/ghc/ghc/-/jobs/1753168 | failed | refs/merge-requests/11923/head | x86_64-linux-alpine3_17-wasm-cross_wasm32-wasi-release+fully_static | x86_64-linux-2.zw3rk_x86_64-linux-2_d292a225d2eb | 2024-01-16 |
| https://gitlab.haskell.org/ghc/ghc/-/jobs/1752167 | failed | master | nightly-x86_64-linux-alpine3_17-wasm-cross_wasm32-wasi-release+fully_static | x86_64-linux-7.zw3rk_x86_64-linux-7_db038e4e1623 | 2024-01-15 |
| https://gitlab.haskell.org/ghc/ghc/-/jobs/1751802 | failed | refs/merge-requests/11910/head | x86_64-linux-alpine3_17-wasm-cross_wasm32-wasi-release+fully_static | x86_64-linux-6.zw3rk_x86_64-linux-6_db038e4e1623 | 2024-01-14 |
| https://gitlab.haskell.org/ghc/ghc/-/jobs/1750663 | failed | master | nightly-x86_64-linux-alpine3_17-wasm-int_native-cross_wasm32-wasi-release+fully_static | x86_64-linux-2.zw3rk_x86_64-linux-2_d292a225d2eb | 2024-01-12 |
| https://gitlab.haskell.org/ghc/ghc/-/jobs/1750390 | failed | master | nightly-x86_64-linux-alpine3_17-wasm-unreg-cross_wasm32-wasi-release+fully_static | x86_64-linux-2.zw3rk_x86_64-linux-2_d292a225d2eb | 2024-01-12 |
| https://gitlab.haskell.org/ghc/ghc/-/jobs/1748067 | failed | refs/merge-requests/11879/head | x86_64-linux-alpine3_17-wasm-cross_wasm32-wasi-release+fully_static | x86_64-linux-2.zw3rk_x86_64-linux-2_d292a225d2eb | 2024-01-09 |
Cheng Shao
Cheng Shao
https://gitlab.haskell.org/ghc/ghc/-/issues/24407
head.hackage: ghc-debugger's test:system-test hangs on test-downstream job
2024-02-06T15:33:58Z
Bryan R
bryan@haskell.foundation
head.hackage: ghc-debugger's test:system-test hangs on test-downstream job
The failure affects both x86-64 and arm64.
Example 1:
https://gitlab.haskell.org/ghc/head.hackage/-/jobs/1769505
Full list:
https://gitlab.haskell.org/ghc/head.hackage/-/jobs?statuses=RUNNING
![image](/uploads/bc181b40cb8e8753603fb7...
The failure affects both x86-64 and arm64.
Example 1:
https://gitlab.haskell.org/ghc/head.hackage/-/jobs/1769505
Full list:
https://gitlab.haskell.org/ghc/head.hackage/-/jobs?statuses=RUNNING
![image](/uploads/bc181b40cb8e8753603fb7344a9732cb/image.png)
(and many more)
Since a test timeout is treated as a spurious failure, these get restarted. I think there's around 10 restarts, which results in pipelines taking 80 hours to fail entirely. The result is that many head.hackage pipelines are still running.
https://gitlab.haskell.org/ghc/head.hackage/-/pipelines?page=1&scope=all&status=running
![image](/uploads/cd6673132a4e42c70488ae23434c1242/image.png)
Luckily, the test is merely zombied and doesn't take any system resources. Therefore I'll keep them running for now, so people can investigate. Example process tree:
```
489457 ? S 0:00 \_ bash /nix/store/36qkrsgvqb0scf7hcdr45vz37nbi904x-run-ci
489516 ? Sl 0:00 \_ /nix/store/jx4pn482brdqi9fp41m6faawwcibq354-head-hackage-ci-0.1.0.0/bin/head-hackage-ci test-patches --extra-cabal-fragment=/builds/ghc/head.hackage/ci/run/deps.cabal.project --patches=../../patches --with-compiler=/builds/ghc/head.hackage/ghc/bin/ghc --extra-cabal-fragment=/builds/ghc/head.hackage/ci/config.cabal.project --expect-broken=charsetdetect --expect-broken=packman --extra-package=lens==5.2.2 --extra-package=optics==0.4.2.1 --extra-package=aeson==2.2.0.0 --extra-package=criterion==1.6.2.0 --extra-package=scotty==0.12.1 --extra-package=generic-lens==2.2.2.0 --extra-package=microstache==1.0.2.3 --extra-package=singletons-base==3.1.1 --extra-package=servant==0.20 --extra-package=hgmp==0.1.2.1 --extra-package=Agda==2.6.3 --extra-package=mmark==0.0.7.6 --extra-package=doctest==0.22.0 --extra-package=tasty==1.4.3 --extra-package=pandoc==3.1.5 --extra-package=servant-conduit==0.16 --extra-package=servant-machines==0.16 --extra-package=linear-generics==0.2.2 --extra-package=futhark==0.25.2 --extra-package=generic-random==1.5.0.1 --extra-package=lame==0.2.1 --extra-package=inspection-testing==0.5.0.2 --extra-package=ghcide==2.0.0.1 --extra-package=vector-space==0.16 --build-tool-package=alex --build-tool-package=happy --build-tool-package=c2hs --only=tasty --test-package=system-test=/builds/ghc/head.hackage/ci/../tests/ghc-debug/test/ --test-package=ghc-tests=/builds/ghc/head.hackage/ci/../tests/ghc-tests --test-package=all=/builds/ghc/head.hackage/ci/../tests/text --test-package=bytestring-tests=/builds/ghc/head.hackage/ci/../tests/bytestring --test-package=all=/builds/ghc/head.hackage/ci/../tests/containers/containers-tests --cabal-option=-j5 --ghc-option=-dlint
548483 ? Sl 0:00 \_ /nix/store/gwlb8xmagw5a8wam56ja1l0xjpzd139a-cabal-install-3.10.1.0/bin/cabal new-test system-test --enable-tests -j5 -w /builds/ghc/head.hackage/ghc/bin/ghc
548511 ? Sl 0:00 \_ /nix/store/gwlb8xmagw5a8wam56ja1l0xjpzd139a-cabal-install-3.10.1.0/bin/.cabal-wrapped act-as-setup --build-type=Simple -- test --verbose=1 --builddir=/builds/ghc/head.hackage/ci/run/test-system-test/dist-newstyle/build/aarch64-linux/ghc-9.9.20240202/ghc-debugger-0.5.0.0/t/system-test --log=$pkgid-$test-suite.log --machine-log=$pkgid.log --show-details=failures system-test
548522 ? Sl 0:00 \_ /builds/ghc/head.hackage/ci/run/test-system-test/dist-newstyle/build/aarch64-linux/ghc-9.9.20240202/ghc-debugger-0.5.0.0/t/system-test/build/system-test/system-test
548717 ? Z 0:00 \_ [clock] <defunct>
```
9.10.1
Ben Gamari
Ben Gamari
https://gitlab.haskell.org/ghc/ghc/-/issues/24366
Intermittent stat failures for MultiLayerModulesRecomp
2024-01-23T16:46:16Z
Bryan R
bryan@haskell.foundation
Intermittent stat failures for MultiLayerModulesRecomp
Perhaps it just needs to have its acceptable wobble room increased.
Incidents in the last 2 weeks:
| json->>'id' | json->>'web_url' | json->>'runner.description' | date(json->>'cr...
Perhaps it just needs to have its acceptable wobble room increased.
Incidents in the last 2 weeks:
| json->>'id' | json->>'web_url' | json->>'runner.description' | date(json->>'created_at') |
|-------------|---------------------------------------------------|----------------------------------------------------|---------------------------|
| 1757884 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1757884 | aarch64-darwin-3_aarch64-darwin-3_cf5fc9331e7a | 2024-01-22 |
| 1757422 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1757422 | x86-64-win-1.zw3rk.com | 2024-01-20 |
| 1757421 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1757421 | x86_64-linux-5.zw3rk_x86_64-linux-5_db038e4e1623 | 2024-01-20 |
| 1757419 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1757419 | aarch64-linux_ghc-arm-5_6b65c6b684af | 2024-01-20 |
| 1757418 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1757418 | aarch64-darwin-1_aarch64-darwin-1_cf5fc9331e7a | 2024-01-20 |
| 1750154 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1750154 | x86-64-win-1.zw3rk.com | 2024-01-11 |
| 1750153 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1750153 | x86_64-linux-3.zw3rk_x86_64-linux-3_db038e4e1623 | 2024-01-11 |
| 1750151 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1750151 | aarch64-linux-2.zw3rk_aarch64-linux-2_379242e21d29 | 2024-01-11 |
| 1750150 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1750150 | aarch64-darwin-2_aarch64-darwin-2_cf5fc9331e7a | 2024-01-11 |
| 1748950 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1748950 | x86-64-win-2.zw3rk.com | 2024-01-10 |
| 1748947 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1748947 | aarch64-linux-2.zw3rk_aarch64-linux-2_379242e21d29 | 2024-01-10 |
| 1748946 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1748946 | aarch64-darwin-3_aarch64-darwin-3_cf5fc9331e7a | 2024-01-10 |
https://gitlab.haskell.org/ghc/ghc/-/issues/24365
Intermittent hadrian exception: withFile: resource busy (file is locked)
2024-01-29T07:09:22Z
Bryan R
bryan@haskell.foundation
Intermittent hadrian exception: withFile: resource busy (file is locked)
Not many in the last two weeks, though I think I've seen it before:
| json->>'id' | json->>'web_url' | json->>'runner.description' | date(json->>'created_at') |
|-------------|----...
Not many in the last two weeks, though I think I've seen it before:
| json->>'id' | json->>'web_url' | json->>'runner.description' | date(json->>'created_at') |
|-------------|---------------------------------------------------|----------------------------------------------------|---------------------------|
| 1758509 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1758509 | azure-ci-1__9d97fee6dfe8 | 2024-01-23 |
| 1757881 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1757881 | aarch64-darwin-2_aarch64-darwin-2_cf5fc9331e7a | 2024-01-22 |
| 1748231 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1748231 | aarch64-linux-2.zw3rk_aarch64-linux-2_379242e21d29 | 2024-01-09 |
Two of the occasions mention `stamp-ghc-bignum-1.3-inplace_dyn`; one of them mentions `stamp-ghc-9.9-inplace_dyn`. But see #20886: the filename may be a red herring. Or maybe not!
```
/---------------------------------------------------------------------------------\
| Successfully built library 'ghc-bignum' (Stage1, way v). |
| Library: _build/stage1/libraries/ghc-bignum/build/libHSghc-bignum-1.3-inplace.a |
| Library synopsis: GHC BigNum library. |
\---------------------------------------------------------------------------------/
Error when running Shake build system:
at want, called at src/Main.hs:126:44 in main:Main
* Depends on: test:all_deps
at apply1, called at src/Development/Shake/Internal/Rules/Rerun.hs:41:5 in shake-0.19.7-cd9c3e55acea68cf885a8c15ce0927a9c158686ecdfdc3cb2cfc3905a7237499:Development.Shake.Internal.Rules.Rerun
* Depends on: test:check-exact
at apply1, called at src/Development/Shake/Internal/Rules/Rerun.hs:41:5 in shake-0.19.7-cd9c3e55acea68cf885a8c15ce0927a9c158686ecdfdc3cb2cfc3905a7237499:Development.Shake.Internal.Rules.Rerun
* Depends on: _build/test/bin/check-exact
at need, called at src/Rules/Test.hs:163:17 in main:Rules.Test
* Depends on: _build/stage1/bin/check-exact
at need, called at src/Rules/Register.hs:86:5 in main:Rules.Register
* Depends on: _build/stage1/lib/i386-linux-ghc-9.9.20240123/libHSghc-bignum-1.3-inplace-ghc9.9.20240123.so
at need, called at src/Rules/Library.hs:83:5 in main:Rules.Library
* Depends on: _build/stage1/lib/package.conf.d/ghc-bignum-1.3-inplace.conf
at need, called at src/Rules/Register.hs:140:5 in main:Rules.Register
* Depends on: _build/stage1/libraries/ghc-bignum/build/stamp-ghc-bignum-1.3-inplace_dyn
* Raised the exception:
_build/stage1/libraries/ghc-bignum/build/stamp-ghc-bignum-1.3-inplace_dyn: withFile: resource busy (file is locked)
```
https://gitlab.haskell.org/ghc/ghc/-/issues/24364
Intermittent cabal solver failing building (I think) hadrian
2024-02-08T15:05:49Z
Bryan R
bryan@haskell.foundation
Intermittent cabal solver failing building (I think) hadrian
Possibly a duplicate of #24363, though the symptoms are different.
Cabal says, "After searching the rest of the dependency tree exhaustively, these were the goals I've had most trouble fulfilling: base, QuickCheck, splitmix, hadrian, ha...
Possibly a duplicate of #24363, though the symptoms are different.
Cabal says, "After searching the rest of the dependency tree exhaustively, these were the goals I've had most trouble fulfilling: base, QuickCheck, splitmix, hadrian, hadrian:selftest, ghc-platform"
Incidents in the last two weeks:
| json->>'id' | json->>'name' | json->>'web_url' | json->>'runner.description' | date(json->>'created_at') |
|-------------|--------------------------------------------------------------|---------------------------------------------------|--------------------------------------------------|---------------------------|
| 1758443 | nightly-x86_64-linux-alpine3_17-wasm-cross_wasm32-wasi-relea | https://gitlab.haskell.org/ghc/ghc/-/jobs/1758443 | x86_64-linux-5.zw3rk_x86_64-linux-5_db038e4e1623 | 2024-01-23 |
| | se+fully_static | | | |
| 1758141 | x86_64-linux-alpine3_12-validate+fully_static | https://gitlab.haskell.org/ghc/ghc/-/jobs/1758141 | x86_64-linux-5.zw3rk_x86_64-linux-5_db038e4e1623 | 2024-01-22 |
| 1757730 | x86_64-linux-alpine3_12-validate+fully_static | https://gitlab.haskell.org/ghc/ghc/-/jobs/1757730 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-21 |
| 1757460 | nightly-x86_64-linux-alpine3_12-validate+fully_static | https://gitlab.haskell.org/ghc/ghc/-/jobs/1757460 | x86_64-linux-7.zw3rk_x86_64-linux-7_db038e4e1623 | 2024-01-21 |
| 1756114 | nightly-x86_64-linux-alpine3_12-validate+fully_static | https://gitlab.haskell.org/ghc/ghc/-/jobs/1756114 | x86_64-linux-5.zw3rk_x86_64-linux-5_db038e4e1623 | 2024-01-19 |
| 1755231 | x86_64-linux-alpine3_17-wasm-cross_wasm32-wasi-release+fully | https://gitlab.haskell.org/ghc/ghc/-/jobs/1755231 | x86_64-linux-6.zw3rk_x86_64-linux-6_db038e4e1623 | 2024-01-17 |
| | _static | | | |
| 1754565 | x86_64-linux-alpine3_12-validate+fully_static | https://gitlab.haskell.org/ghc/ghc/-/jobs/1754565 | x86_64-linux-2.zw3rk_x86_64-linux-2_d292a225d2eb | 2024-01-17 |
| 1753962 | x86_64-linux-alpine3_17-wasm-cross_wasm32-wasi-release+fully | https://gitlab.haskell.org/ghc/ghc/-/jobs/1753962 | x86_64-linux-2.zw3rk_x86_64-linux-2_d292a225d2eb | 2024-01-16 |
| | _static | | | |
| 1753102 | nightly-x86_64-linux-alpine3_12-validate+fully_static | https://gitlab.haskell.org/ghc/ghc/-/jobs/1753102 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-16 |
| 1751753 | x86_64-linux-alpine3_12-validate+fully_static | https://gitlab.haskell.org/ghc/ghc/-/jobs/1751753 | x86_64-linux-5.zw3rk_x86_64-linux-5_db038e4e1623 | 2024-01-14 |
| 1751699 | x86_64-linux-alpine3_17-wasm-cross_wasm32-wasi-release+fully | https://gitlab.haskell.org/ghc/ghc/-/jobs/1751699 | x86_64-linux-2.zw3rk_x86_64-linux-2_d292a225d2eb | 2024-01-14 |
| | _static | | | |
| 1751550 | x86_64-linux-alpine3_17-wasm-cross_wasm32-wasi-release+fully | https://gitlab.haskell.org/ghc/ghc/-/jobs/1751550 | x86_64-linux-6.zw3rk_x86_64-linux-6_db038e4e1623 | 2024-01-13 |
| | _static | | | |
| 1750389 | nightly-x86_64-linux-alpine3_17-wasm-int_native-cross_wasm32 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1750389 | x86_64-linux-2.zw3rk_x86_64-linux-2_d292a225d2eb | 2024-01-12 |
| | -wasi-release+fully_static | | | |
| 1749579 | nightly-x86_64-linux-alpine3_17-wasm-unreg-cross_wasm32-wasi | https://gitlab.haskell.org/ghc/ghc/-/jobs/1749579 | x86_64-linux-7.zw3rk_x86_64-linux-7_db038e4e1623 | 2024-01-11 |
| | -release+fully_static | | | |
| 1749033 | x86_64-linux-alpine3_12-validate+fully_static | https://gitlab.haskell.org/ghc/ghc/-/jobs/1749033 | x86_64-linux-5.zw3rk_x86_64-linux-5_db038e4e1623 | 2024-01-10 |
| 1748590 | x86_64-linux-alpine3_12-validate+fully_static | https://gitlab.haskell.org/ghc/ghc/-/jobs/1748590 | x86_64-linux-5.zw3rk_x86_64-linux-5_db038e4e1623 | 2024-01-10 |
```
Running hadrian/build-cabal --flavour=validate+fully_static -j8 --broken-test=ghcilink002 linker_unload_native encoding004 T10458 --bignum=gmp --docs=no-sphinx test:all_deps bin
ary-dist -V...
Warning: Unknown/unsupported 'ghc' version detected (Cabal 3.6.2.0 supports
'ghc' version < 9.4): /opt/ghc/9.4.3/bin/ghc is version 9.4.3
Warning: Requested index-state 2023-10-05T11:38:51Z is newer than
'hackage.haskell.org'! Falling back to older state (2022-06-09T19:15:55Z).
Resolving dependencies...
cabal: Could not resolve dependencies:
[__0] trying: ghc-platform-0.1.0.0 (user goal)
[__1] trying: base-4.17.0.0/installed-4.17.0.0 (dependency of ghc-platform)
[__2] trying: hadrian-0.1.0.0 (user goal)
[__3] trying: hadrian:+selftest
[__4] trying: QuickCheck-2.14.2 (dependency of hadrian +selftest)
[__5] next goal: splitmix (dependency of QuickCheck)
[__5] rejecting: splitmix-0.1.0.4 (conflict:
base==4.17.0.0/installed-4.17.0.0, splitmix => base>=4.3 && <4.17)
[__5] skipping: splitmix-0.1.0.3, splitmix-0.1.0.2, splitmix-0.1.0.1,
splitmix-0.1, splitmix-0.0.5, splitmix-0.0.4, splitmix-0.0.3, splitmix-0.0.2,
splitmix-0.0.1, splitmix-0 (has the same characteristics that caused the
previous version to fail: excludes 'base' version 4.17.0.0)
[__5] fail (backjumping, conflict set: QuickCheck, base, splitmix)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: base, QuickCheck, splitmix, hadrian,
hadrian:selftest, ghc-platform
Try running with --minimize-conflict-set to improve the error message.
```
https://gitlab.haskell.org/ghc/ghc/-/issues/24363
Intermittent cabal solver failure building (I think) "binary"
2024-02-08T13:47:13Z
Bryan R
bryan@haskell.foundation
Intermittent cabal solver failure building (I think) "binary"
Cabal says "After searching the rest of the dependency tree exhaustively, these were the goals I've had most trouble fulfilling: ghc-boot:setup.Cabal, ghc-boot:setup.pretty, pretty, ghc-boot, Cabal".
Incidents in the last two weeks:
| ...
Cabal says "After searching the rest of the dependency tree exhaustively, these were the goals I've had most trouble fulfilling: ghc-boot:setup.Cabal, ghc-boot:setup.pretty, pretty, ghc-boot, Cabal".
Incidents in the last two weeks:
| json->>'id' | json->>'name' | json->>'web_url' | json->>'runner.description' | date(json->>'created_at') |
|-------------|-----------------------------------------|---------------------------------------------------|--------------------------------------------------|---------------------------|
| 1758471 | test-cabal-reinstall-x86_64-linux-deb10 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1758471 | azure-ghc-ci-linux-1 | 2024-01-23 |
| 1757837 | test-cabal-reinstall-x86_64-linux-deb10 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1757837 | x86_64-linux-6.zw3rk_x86_64-linux-6_db038e4e1623 | 2024-01-22 |
| 1757489 | test-cabal-reinstall-x86_64-linux-deb10 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1757489 | azure-ghc-ci-linux-1 | 2024-01-21 |
| 1756611 | test-cabal-reinstall-x86_64-linux-deb10 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1756611 | x86_64-linux-6.zw3rk_x86_64-linux-6_db038e4e1623 | 2024-01-20 |
| 1756143 | test-cabal-reinstall-x86_64-linux-deb10 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1756143 | azure-ghc-ci-linux-2 | 2024-01-19 |
| 1755357 | test-cabal-reinstall-x86_64-linux-deb10 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1755357 | x86_64-linux-5.zw3rk_x86_64-linux-5_db038e4e1623 | 2024-01-18 |
| 1754037 | test-cabal-reinstall-x86_64-linux-deb10 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1754037 | x86_64-linux-2.zw3rk_x86_64-linux-2_d292a225d2eb | 2024-01-17 |
| 1753131 | test-cabal-reinstall-x86_64-linux-deb10 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1753131 | azure-ghc-ci-linux-2 | 2024-01-16 |
| 1752195 | test-cabal-reinstall-x86_64-linux-deb10 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1752195 | x86_64-linux-5.zw3rk_x86_64-linux-5_db038e4e1623 | 2024-01-15 |
| 1751676 | test-cabal-reinstall-x86_64-linux-deb10 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1751676 | azure-ghc-ci-linux-1 | 2024-01-14 |
| 1751211 | test-cabal-reinstall-x86_64-linux-deb10 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1751211 | x86_64-linux-3.zw3rk_x86_64-linux-3_db038e4e1623 | 2024-01-13 |
| 1750686 | test-cabal-reinstall-x86_64-linux-deb10 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1750686 | x86_64-linux-6.zw3rk_x86_64-linux-6_db038e4e1623 | 2024-01-12 |
| 1750635 | test-cabal-reinstall-x86_64-linux-deb10 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1750635 | x86_64-linux-6.zw3rk_x86_64-linux-6_db038e4e1623 | 2024-01-12 |
| 1749605 | test-cabal-reinstall-x86_64-linux-deb10 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1749605 | x86_64-linux-3.zw3rk_x86_64-linux-3_db038e4e1623 | 2024-01-11 |
| 1748472 | test-cabal-reinstall-x86_64-linux-deb10 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1748472 | x86_64-linux-7.zw3rk_x86_64-linux-7_db038e4e1623 | 2024-01-10 |
```
| Create directory /builds/ghc/ghc/_build/stage-cabal/bin
# _build/stage-cabal/cabal-packages
# /usr/local/bin/cabal
| Run Cabal Install Stage2: none => none
/usr/local/bin/cabal --store-dir=/builds/ghc/ghc/_build/stage-cabal/cabal-store build binary --project-file /builds/ghc/ghc/cabal.project-reinstall --distdir _build/stage-cabal/
dist-newstyle --with-compiler=/builds/ghc/ghc/_build/stage1/bin/ghc --enable-executable-dynamic --enable-library-vanilla
Command line: /usr/local/bin/cabal --store-dir=/builds/ghc/ghc/_build/stage-cabal/cabal-store build binary --project-file /builds/ghc/ghc/cabal.project-reinstall --distdir _buil
d/stage-cabal/dist-newstyle --with-compiler=/builds/ghc/ghc/_build/stage1/bin/ghc --enable-executable-dynamic --enable-library-vanilla
===> Command failed with error code: 1
Warning: Unknown/unsupported 'ghc' version detected (Cabal 3.6.2.0 supports
'ghc' version < 9.4): /builds/ghc/ghc/_build/stage1/bin/ghc is version
9.9.20240120
cabal: Could not resolve dependencies:
[__0] trying: Cabal-3.11.0.0 (user goal)
[__1] trying: pretty-1.1.3.6/installed-inplace (dependency of Cabal)
[__2] trying: ghc-boot-9.9 (user goal)
[__3] next goal: ghc-boot:setup.Cabal (dependency of ghc-boot)
[__3] rejecting: ghc-boot:setup.Cabal~>Cabal-3.11.0.0,
ghc-boot:setup.Cabal-3.11.0.0, ghc-boot:setup.Cabal-3.10.1.0,
ghc-boot:setup.Cabal-3.8.1.0 (constraint from maximum version of Cabal used by
Setup.hs requires <3.8)
[__3] trying: ghc-boot:setup.Cabal-3.6.3.0
[__4] next goal: ghc-boot:setup.pretty (dependency of ghc-boot:setup.Cabal)
[__4] rejecting: ghc-boot:setup.pretty~>pretty-1.1.3.6/installed-inplace
(conflict: ghc-boot:setup.Cabal => ghc-boot:setup.deepseq>=1.3.0.1 && <1.5,
ghc-boot:setup.pretty => ghc-boot:setup.deepseq==1.5.0.0/installed-inplace)
[__4] rejecting: ghc-boot:setup.pretty-1.1.3.6/installed-inplace (multiple
instances)
[__4] rejecting: ghc-boot:setup.pretty-1.1.3.6, ghc-boot:setup.pretty-1.1.3.5,
ghc-boot:setup.pretty-1.1.3.4, ghc-boot:setup.pretty-1.1.3.3,
ghc-boot:setup.pretty-1.1.3.2, ghc-boot:setup.pretty-1.1.3.1,
ghc-boot:setup.pretty-1.1.2.1, ghc-boot:setup.pretty-1.1.2.0,
ghc-boot:setup.pretty-1.1.1.3, ghc-boot:setup.pretty-1.1.1.2,
ghc-boot:setup.pretty-1.1.1.1, ghc-boot:setup.pretty-1.1.1.0,
ghc-boot:setup.pretty-1.1.0.0, ghc-boot:setup.pretty-1.0.1.2,
ghc-boot:setup.pretty-1.0.1.1, ghc-boot:setup.pretty-1.0.1.0,
ghc-boot:setup.pretty-1.0.0.0 (constraint from project config
/builds/ghc/ghc/cabal.project-reinstall requires installed instance)
[__4] fail (backjumping, conflict set: pretty, ghc-boot:setup.Cabal,
ghc-boot:setup.pretty)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: ghc-boot:setup.Cabal,
ghc-boot:setup.pretty, pretty, ghc-boot, Cabal
Try running with --minimize-conflict-set to improve the error message.
```
https://gitlab.haskell.org/ghc/ghc/-/issues/24362
Many unexpected test timeouts
2024-01-23T15:25:15Z
Bryan R
bryan@haskell.foundation
Many unexpected test timeouts
The overwhelming culprit is zw3rk linux-4, so I have paused that runner and alerted @angerman.
Incidents in the last two weeks:
| json->>'id' | json->>'web_url' | json->>'runner.description' ...
The overwhelming culprit is zw3rk linux-4, so I have paused that runner and alerted @angerman.
Incidents in the last two weeks:
| json->>'id' | json->>'web_url' | json->>'runner.description' | date(json->>'created_at') |
|-------------|---------------------------------------------------|--------------------------------------------------|---------------------------|
| 1758393 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1758393 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-22 |
| 1758303 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1758303 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-22 |
| 1758294 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1758294 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-22 |
| 1758160 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1758160 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-22 |
| 1757990 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1757990 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-22 |
| 1757824 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1757824 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-22 |
| 1757808 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1757808 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-22 |
| 1757741 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1757741 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-21 |
| 1757731 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1757731 | x86_64-linux-6.zw3rk_x86_64-linux-6_db038e4e1623 | 2024-01-21 |
| 1757668 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1757668 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-21 |
| 1757655 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1757655 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-21 |
| 1757477 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1757477 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-21 |
| 1757474 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1757474 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-21 |
| 1757469 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1757469 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-21 |
| 1756592 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1756592 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-20 |
| 1756583 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1756583 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-20 |
| 1756128 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1756128 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-19 |
| 1756117 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1756117 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-19 |
| 1755896 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1755896 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-18 |
| 1755698 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1755698 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-18 |
| 1755343 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1755343 | x86_64-linux-7.zw3rk_x86_64-linux-7_db038e4e1623 | 2024-01-18 |
| 1755328 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1755328 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-18 |
| 1755237 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1755237 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-17 |
| 1754569 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1754569 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-17 |
| 1754212 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1754212 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-17 |
| 1754209 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1754209 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-17 |
| 1754206 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1754206 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-17 |
| 1754033 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1754033 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-17 |
| 1753889 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1753889 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-16 |
| 1753888 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1753888 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-16 |
| 1753875 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1753875 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-16 |
| 1753559 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1753559 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-16 |
| 1753534 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1753534 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-16 |
| 1753483 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1753483 | azure-ghc-ci-linux-2 | 2024-01-16 |
| 1753289 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1753289 | x86_64-linux-7.zw3rk_x86_64-linux-7_db038e4e1623 | 2024-01-16 |
| 1753248 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1753248 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-16 |
| 1753239 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1753239 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-16 |
| 1753234 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1753234 | x86_64-linux-5.zw3rk_x86_64-linux-5_db038e4e1623 | 2024-01-16 |
| 1753228 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1753228 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-16 |
| 1753118 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1753118 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-16 |
| 1753112 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1753112 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-16 |
| 1752820 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1752820 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-15 |
| 1752817 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1752817 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-15 |
| 1752718 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1752718 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-15 |
| 1752420 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1752420 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-15 |
| 1752416 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1752416 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-15 |
| 1751858 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1751858 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-14 |
| 1751852 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1751852 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-14 |
| 1751646 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1751646 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-14 |
| 1751642 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1751642 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-14 |
| 1751181 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1751181 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-13 |
| 1751177 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1751177 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-13 |
| 1750977 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1750977 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-12 |
| 1750644 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1750644 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-12 |
| 1750643 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1750643 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-12 |
| 1750346 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1750346 | x86_64-linux-7.zw3rk_x86_64-linux-7_db038e4e1623 | 2024-01-12 |
| 1750264 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1750264 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-11 |
| 1750238 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1750238 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-11 |
| 1749516 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1749516 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-10 |
| 1749513 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1749513 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-10 |
| 1749001 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1749001 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-10 |
| 1748835 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1748835 | x86_64-linux-6.zw3rk_x86_64-linux-6_db038e4e1623 | 2024-01-10 |
| 1748634 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1748634 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-10 |
| 1748455 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1748455 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-10 |
| 1748087 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1748087 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-09 |
| 1747940 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1747940 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-09 |
| 1747936 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1747936 | x86_64-linux-4.zw3rk_x86_64-linux-4_db038e4e1623 | 2024-01-09 |
https://gitlab.haskell.org/ghc/ghc/-/issues/24361
Darwin: T8089 failing after receiving signal 9
2024-02-23T07:01:31Z
Bryan R
bryan@haskell.foundation
Darwin: T8089 failing after receiving signal 9
Incidents in the last two weeks:
| json->>'id' | json->>'web_url' | json->>'runner.description' | date(json->>'created_at') |
|-------------|-------------------------------------------...
Incidents in the last two weeks:
| json->>'id' | json->>'web_url' | json->>'runner.description' | date(json->>'created_at') |
|-------------|---------------------------------------------------|------------------------------------------------|---------------------------|
| 1757642 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1757642 | aarch64-darwin-1_aarch64-darwin-1_cf5fc9331e7a | 2024-01-21 |
| 1756702 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1756702 | aarch64-darwin-1_aarch64-darwin-1_cf5fc9331e7a | 2024-01-20 |
| 1756051 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1756051 | aarch64-darwin-1_aarch64-darwin-1_cf5fc9331e7a | 2024-01-18 |
| 1755685 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1755685 | aarch64-darwin-3_aarch64-darwin-3_cf5fc9331e7a | 2024-01-18 |
| 1753481 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1753481 | aarch64-darwin-1_aarch64-darwin-1_cf5fc9331e7a | 2024-01-16 |
| 1749572 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1749572 | aarch64-darwin-3_aarch64-darwin-3_cf5fc9331e7a | 2024-01-11 |
| 1747995 | https://gitlab.haskell.org/ghc/ghc/-/jobs/1747995 | aarch64-darwin-1_aarch64-darwin-1_cf5fc9331e7a | 2024-01-09 |
Ben Gamari
Ben Gamari
https://gitlab.haskell.org/ghc/ghc/-/issues/24303
"MultiLayerModulesDefsGhciReload" timeouts on master
2024-01-09T16:17:25Z
Bryan R
bryan@haskell.foundation
"MultiLayerModulesDefsGhciReload" timeouts on master
Seen on https://grafana.gitlab.haskell.org/d/phJc-UyVz/ci-success-stats?orgId=2&refresh=30m&viewPanel=8
* https://gitlab.haskell.org/ghc/ghc/-/jobs/1741961
* https://gitlab.haskell.org/ghc/ghc/-/jobs/1740330
* https://gitlab.haskell.org...
Seen on https://grafana.gitlab.haskell.org/d/phJc-UyVz/ci-success-stats?orgId=2&refresh=30m&viewPanel=8
* https://gitlab.haskell.org/ghc/ghc/-/jobs/1741961
* https://gitlab.haskell.org/ghc/ghc/-/jobs/1740330
* https://gitlab.haskell.org/ghc/ghc/-/jobs/1741973
Zubin
Zubin
https://gitlab.haskell.org/ghc/ghc/-/issues/24274
Spurious failures due to intermittent stack concurrency bug
2024-01-19T12:53:04Z
Bryan R
bryan@haskell.foundation
Spurious failures due to intermittent stack concurrency bug
We have spurious failures because of https://github.com/commercialhaskell/stack/issues/6388.
What's not clear to me is why a package that required a call to `updateHackageIndex` in one run did *not* require the call in a different run. ...
We have spurious failures because of https://github.com/commercialhaskell/stack/issues/6388.
What's not clear to me is why a package that required a call to `updateHackageIndex` in one run did *not* require the call in a different run. I would expect the set of packages that trigger the call to be fixed. Are we caching the Pantry database?
https://gitlab.haskell.org/ghc/ghc/-/issues/24268
Bug in ghcup metadata generation?
2024-02-05T17:53:11Z
Bryan R
bryan@haskell.foundation
Bug in ghcup metadata generation?
Both 9.6.3 and 9.8.1 ghcup metadata have the same defect:
* https://github.com/haskell/ghcup-metadata/pull/140#pullrequestreview-1750607568
* https://github.com/haskell/ghcup-metadata/pull/124#pullrequestreview-1750600916
**Expected**
...
Both 9.6.3 and 9.8.1 ghcup metadata have the same defect:
* https://github.com/haskell/ghcup-metadata/pull/140#pullrequestreview-1750607568
* https://github.com/haskell/ghcup-metadata/pull/124#pullrequestreview-1750600916
**Expected**
`vlTestDL.dlUri` should be the url for the testsuite tarball. Its sha256 hash should match `vlTestDL.dlHash`.
**Actual**
`vlTestDL.dlUri` is the url for the *source* tarball.
`VlTestDL.dlHash` is the sha256 hash of the *testsuite* tarball, so it doesn't correspond.
Zubin
Zubin
https://gitlab.haskell.org/ghc/ghc/-/issues/24259
JS: Spurious failures in CI
2023-12-21T21:18:04Z
Luite Stegeman
JS: Spurious failures in CI
The JS backend experiences occasional failures in CI, which is holding up marge batches.
This is likely due to a combination of real bugs and poorly adjusted timeouts for the tests.
I'm going to mark the tests as `js_fragile` with a re...
The JS backend experiences occasional failures in CI, which is holding up marge batches.
This is likely due to a combination of real bugs and poorly adjusted timeouts for the tests.
I'm going to mark the tests as `js_fragile` with a reference to this ticket number. In the future we can track the underlying cause here, and if necessary open new tickets for them.
Hopefully in a few months we can check back on this ticket, review the CI logs and remove the `js_fragile` designation if tests aren't failing anymore.
Luite Stegeman
Luite Stegeman
https://gitlab.haskell.org/ghc/ghc/-/issues/24209
Alpine-based CI jobs fail with out-of-date Hackage index
2023-11-28T15:24:20Z
Ben Gamari
Alpine-based CI jobs fail with out-of-date Hackage index
Nightly Alpine jobs are failing with:
```
Running hadrian/build-cabal --flavour=release+fully_static -j8 --broken-test= --bignum=gmp --docs=none test:all_deps binary-dist -V...
Warning: Unknown/unsupported 'ghc' version detected (Cabal 3...
Nightly Alpine jobs are failing with:
```
Running hadrian/build-cabal --flavour=release+fully_static -j8 --broken-test= --bignum=gmp --docs=none test:all_deps binary-dist -V...
Warning: Unknown/unsupported 'ghc' version detected (Cabal 3.6.2.0 supports
'ghc' version < 9.4): /opt/ghc/9.4.3/bin/ghc is version 9.4.3
Warning: Requested index-state 2023-10-05T11:38:51Z is newer than
'hackage.haskell.org'! Falling back to older state (2022-06-09T19:15:55Z).
Resolving dependencies...
cabal: Could not resolve dependencies:
...
```
For instance, see Job [#1717907](https://gitlab.haskell.org/ghc/ghc/-/jobs/1717907), Job [#1717906](https://gitlab.haskell.org/ghc/ghc/-/jobs/1717906), and Job [#1717910](https://gitlab.haskell.org/ghc/ghc/-/jobs/1717910).
This is strange since we earlier had run:
```
Running /usr/local/bin/cabal update -w /opt/ghc/9.4.3/bin/ghc hackage.haskell.org,2023-10-05T11:38:51Z...
Config file path source is default config file.
Config file /builds/ghc/ghc/cabal/config not found.
Writing default configuration to /builds/ghc/ghc/cabal/config
Downloading the latest package list from hackage.haskell.org
Updated package list of hackage.haskell.org to the index-state 2023-10-05T11:38:51Z
```
My initial suspicion was that we are using a different `cabal-install` between these two invocations. However, looking at `ci.sh` it appears that we use `$CABAL` consistently. Very odd.
9.10.1
Ben Gamari
Ben Gamari
https://gitlab.haskell.org/ghc/ghc/-/issues/24167
ld: warning: ignoring duplicate libraries: '-lm'
2024-02-09T10:35:36Z
Rodrigo Mesquita
ld: warning: ignoring duplicate libraries: '-lm'
## Summary
I've been getting warnings when linking Haskell executables:
AA.hs:
```haskell
module Main where
main = pure()
```
results in:
```
> ghc AA.hs
[1 of 2] Compiling Main ( AA.hs, AA.o ) [Source file changed]
[2 of 2...
## Summary
I've been getting warnings when linking Haskell executables:
AA.hs:
```haskell
module Main where
main = pure()
```
results in:
```
> ghc AA.hs
[1 of 2] Compiling Main ( AA.hs, AA.o ) [Source file changed]
[2 of 2] Linking AA [Objects changed]
ld: warning: ignoring duplicate libraries: '-lm'
```
This also seems to have broken CI for aarch64-darwin
## Expected behavior
We should fix the warning
## Environment
* GHC version used: 9.6
* Operating System: macOS
* System Architecture: aarch64
Rodrigo Mesquita
Rodrigo Mesquita
https://gitlab.haskell.org/ghc/ghc/-/issues/24061
wasm-cross failures: Cabal complains about splitmix
2023-10-10T07:40:08Z
Bryan R
bryan@haskell.foundation
wasm-cross failures: Cabal complains about splitmix
Cabal finds itself unable to build a plan. But only sometimes. :thinking:
Examples I've seen recently:
* https://gitlab.haskell.org/ghc/ghc/-/jobs/1682833
* https://gitlab.haskell.org/ghc/ghc/-/jobs/1678274
Log looks like
<details><...
Cabal finds itself unable to build a plan. But only sometimes. :thinking:
Examples I've seen recently:
* https://gitlab.haskell.org/ghc/ghc/-/jobs/1682833
* https://gitlab.haskell.org/ghc/ghc/-/jobs/1678274
Log looks like
<details><summary>cabal error log</summary>
```
Resolving dependencies...
cabal: Could not resolve dependencies:
[__0] trying: ghc-platform-0.1.0.0 (user goal)
[__1] trying: base-4.17.0.0/installed-4.17.0.0 (dependency of ghc-platform)
[__2] trying: hadrian-0.1.0.0 (user goal)
[__3] trying: hadrian:+selftest
[__4] trying: QuickCheck-2.14.2 (dependency of hadrian +selftest)
[__5] next goal: splitmix (dependency of QuickCheck)
[__5] rejecting: splitmix-0.1.0.4 (conflict:
base==4.17.0.0/installed-4.17.0.0, splitmix => base>=4.3 && <4.17)
[__5] skipping: splitmix-0.1.0.3, splitmix-0.1.0.2, splitmix-0.1.0.1,
splitmix-0.1, splitmix-0.0.5, splitmix-0.0.4, splitmix-0.0.3, splitmix-0.0.2,
splitmix-0.0.1, splitmix-0 (has the same characteristics that caused the
previous version to fail: excludes 'base' version 4.17.0.0)
[__5] fail (backjumping, conflict set: QuickCheck, base, splitmix)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: base, QuickCheck, splitmix, hadrian,
hadrian:selftest, ghc-platform
```
</details>
Giving it high priority because it keeps blocking Marge. This job is not run on the quick validate so doesn't get tested that often.
Cheng Shao
Cheng Shao
https://gitlab.haskell.org/ghc/ghc/-/issues/24056
aarch64-deb10 image has too old LLVM
2023-11-01T00:05:01Z
Ben Gamari
aarch64-deb10 image has too old LLVM
Currently the nightly `nightly-aarch64-linux-deb10-validate+llvm` job fails as the image has too old an LLVM toolchain (`llvm-12`)
See https://gitlab.haskell.org/ghc/ghc/-/jobs/1674986.
Currently the nightly `nightly-aarch64-linux-deb10-validate+llvm` job fails as the image has too old an LLVM toolchain (`llvm-12`)
See https://gitlab.haskell.org/ghc/ghc/-/jobs/1674986.
Ben Gamari
Ben Gamari
https://gitlab.haskell.org/ghc/ghc/-/issues/24047
Darwin CI seems to fail because of missing xcode installation
2023-10-03T13:19:53Z
Andreas Klebinger
Darwin CI seems to fail because of missing xcode installation
## Summary
I get these sort of errors:
```
warning: Nix search path entry '/nix/var/nix/profiles/per-user/root/channels' does not exist, ignoring
/nix/store/ck1slah4x89kplmi4701nwgm152075l0-toolchain
...
Using 9374f116c489443bb61df4f...
## Summary
I get these sort of errors:
```
warning: Nix search path entry '/nix/var/nix/profiles/per-user/root/channels' does not exist, ignoring
/nix/store/ck1slah4x89kplmi4701nwgm152075l0-toolchain
...
Using 9374f116c489443bb61df4ff148edb9c641104dd for performance metric baseline...
xcode-select: error: no developer tools were found at '/Applications/Xcode.app', and no install could be requested (perhaps no UI is present), please install manually from 'developer.apple.com'.
```
## Expected behavior
This happened on this CI job: https://gitlab.haskell.org/ghc/ghc/-/jobs/1680663
Seems like a missconfiguration issue?
Moritz Angermann
Bryan R
bryan@haskell.foundation
Moritz Angermann
https://gitlab.haskell.org/ghc/ghc/-/issues/24012
Nightly failures
2023-10-30T10:50:16Z
Ben Gamari
Nightly failures
Currently there are a few failures in the nightly pipeline which block #24000:
* [ ] `test-cabal-reinstall` [fails](https://gitlab.haskell.org/ghc/ghc/-/jobs/1675022) due to `text` bounds in the linters (fixed in !11335)
* [x] `nightly...
Currently there are a few failures in the nightly pipeline which block #24000:
* [ ] `test-cabal-reinstall` [fails](https://gitlab.haskell.org/ghc/ghc/-/jobs/1675022) due to `text` bounds in the linters (fixed in !11335)
* [x] `nightly-x86_64-linux-ubuntu18_04-validate` [fails](https://gitlab.haskell.org/ghc/ghc/-/jobs/1675018) due to #23843 (fixed by !11079)
* [ ] `nightly-aarch64-linux-deb10-validate+llvm` [fails](https://gitlab.haskell.org/ghc/ghc/-/jobs/1674986) due to an apparently missing LLVM toolchain
* [x] `nightly-aarch64-linux-alpine3_18-validate` [fails](https://gitlab.haskell.org/ghc/ghc/-/jobs/1674984) due to #24011
* [ ] `nightly-x86_64-linux-centos7-validate` and `nightly-x86_64-linux-deb9-validate` [fail](https://gitlab.haskell.org/ghc/ghc/-/jobs/1674997) due to `ghc-toolchain` inconsistencies (fixed by !11135 )
https://gitlab.haskell.org/ghc/ghc/-/issues/23978
95.5% fail rate for Validate pipelines on master in the last month
2023-10-24T08:19:33Z
Bryan R
bryan@haskell.foundation
95.5% fail rate for Validate pipelines on master in the last month
[![image](/uploads/d7445438005c220886d5d74478668c42/image.png)](https://grafana.gitlab.haskell.org/d/kv1oojs4z/ci-health?orgId=2&refresh=1m&viewPanel=8)
| job | pipeline | created_at |
|----|--------|-----------|
| [aarch64-darwin-vali...
[![image](/uploads/d7445438005c220886d5d74478668c42/image.png)](https://grafana.gitlab.haskell.org/d/kv1oojs4z/ci-health?orgId=2&refresh=1m&viewPanel=8)
| job | pipeline | created_at |
|----|--------|-----------|
| [aarch64-darwin-validate](https://gitlab.haskell.org/ghc/ghc/-/jobs/1675871) | [Pipeline #84634](https://gitlab.haskell.org/ghc/ghc/-/pipelines/84634) | 2023-09-26 19:26:02.182335 |
| [aarch64-linux-deb10-validate](https://gitlab.haskell.org/ghc/ghc/-/jobs/1675872) | [Pipeline #84634](https://gitlab.haskell.org/ghc/ghc/-/pipelines/84634) | 2023-09-26 19:26:02.182335 |
| [i386-linux-deb10-validate](https://gitlab.haskell.org/ghc/ghc/-/jobs/1689335) | [Pipeline #85255](https://gitlab.haskell.org/ghc/ghc/-/pipelines/85255) | 2023-10-10 23:06:03.398766 |
| [i386-linux-deb10-validate](https://gitlab.haskell.org/ghc/ghc/-/jobs/1695092) | [Pipeline #85560](https://gitlab.haskell.org/ghc/ghc/-/pipelines/85560) | 2023-10-18 23:41:53.60398 |
| [lint-ci-config](https://gitlab.haskell.org/ghc/ghc/-/jobs/1675089) | [Pipeline #84587](https://gitlab.haskell.org/ghc/ghc/-/pipelines/84587) | 2023-09-26 01:57:12.916866 |
| [lint-ci-config](https://gitlab.haskell.org/ghc/ghc/-/jobs/1697082) | [Pipeline #85656](https://gitlab.haskell.org/ghc/ghc/-/pipelines/85656) | 2023-10-22 06:50:42.367846 |
| [lint-submods-branch](https://gitlab.haskell.org/ghc/ghc/-/jobs/1690820) | [Pipeline #85348](https://gitlab.haskell.org/ghc/ghc/-/pipelines/85348) | 2023-10-12 14:32:05.387576 |
| [stack-hadrian-build](https://gitlab.haskell.org/ghc/ghc/-/jobs/1675870) | [Pipeline #84634](https://gitlab.haskell.org/ghc/ghc/-/pipelines/84634) | 2023-09-26 19:26:02.182335 |
| [test-bootstrap: [9.6.2, registry.gitlab.haskell.org/ghc/ci-images/x86_64-linux-deb10-ghc9_6:$DOCKER_REV]](https://gitlab.haskell.org/ghc/ghc/-/jobs/1687643) | [Pipeline #85186](https://gitlab.haskell.org/ghc/ghc/-/pipelines/85186) | 2023-10-10 01:36:49.329917 |
| [typecheck-testsuite](https://gitlab.haskell.org/ghc/ghc/-/jobs/1677545) | [Pipeline #84717](https://gitlab.haskell.org/ghc/ghc/-/pipelines/84717) | 2023-09-28 07:27:41.812681 |
| [typecheck-testsuite](https://gitlab.haskell.org/ghc/ghc/-/jobs/1690817) | [Pipeline #85348](https://gitlab.haskell.org/ghc/ghc/-/pipelines/85348) | 2023-10-12 14:32:05.387576 |
| [x86_64-darwin-validate](https://gitlab.haskell.org/ghc/ghc/-/jobs/1679545) | [Pipeline #84812](https://gitlab.haskell.org/ghc/ghc/-/pipelines/84812) | 2023-09-30 20:11:34.546562 |
| [x86_64-darwin-validate](https://gitlab.haskell.org/ghc/ghc/-/jobs/1683714) | [Pipeline #84981](https://gitlab.haskell.org/ghc/ghc/-/pipelines/84981) | 2023-10-05 09:38:38.353623 |
| [x86_64-darwin-validate](https://gitlab.haskell.org/ghc/ghc/-/jobs/1696173) | [Pipeline #85610](https://gitlab.haskell.org/ghc/ghc/-/pipelines/85610) | 2023-10-20 08:03:44.795575 |
| [x86_64-linux-alpine3_12-validate+fully_static](https://gitlab.haskell.org/ghc/ghc/-/jobs/1675875) | [Pipeline #84634](https://gitlab.haskell.org/ghc/ghc/-/pipelines/84634) | 2023-09-26 19:26:02.182335 |
| [x86_64-linux-alpine3_12-validate+fully_static](https://gitlab.haskell.org/ghc/ghc/-/jobs/1692921) | [Pipeline #85454](https://gitlab.haskell.org/ghc/ghc/-/pipelines/85454) | 2023-10-14 23:20:02.15024 |
| [x86_64-linux-alpine3_12-validate+fully_static](https://gitlab.haskell.org/ghc/ghc/-/jobs/1697091) | [Pipeline #85656](https://gitlab.haskell.org/ghc/ghc/-/pipelines/85656) | 2023-10-22 06:50:42.367846 |
| [x86_64-linux-alpine3_17-wasm-cross_wasm32-wasi-release+fully_static](https://gitlab.haskell.org/ghc/ghc/-/jobs/1677207) | [Pipeline #84703](https://gitlab.haskell.org/ghc/ghc/-/pipelines/84703) | 2023-09-27 21:34:27.758873 |
| [x86_64-linux-alpine3_17-wasm-cross_wasm32-wasi-release+fully_static](https://gitlab.haskell.org/ghc/ghc/-/jobs/1686166) | [Pipeline #85110](https://gitlab.haskell.org/ghc/ghc/-/pipelines/85110) | 2023-10-07 14:01:21.41664 |
| [x86_64-linux-alpine3_17-wasm-cross_wasm32-wasi-release+fully_static](https://gitlab.haskell.org/ghc/ghc/-/jobs/1686556) | [Pipeline #85129](https://gitlab.haskell.org/ghc/ghc/-/pipelines/85129) | 2023-10-08 19:07:09.149642 |
| [x86_64-linux-alpine3_17-wasm-cross_wasm32-wasi-release+fully_static](https://gitlab.haskell.org/ghc/ghc/-/jobs/1695095) | [Pipeline #85560](https://gitlab.haskell.org/ghc/ghc/-/pipelines/85560) | 2023-10-18 23:41:53.60398 |
| [x86_64-linux-alpine3_17-wasm-cross_wasm32-wasi-release+fully_static](https://gitlab.haskell.org/ghc/ghc/-/jobs/1695322) | [Pipeline #85560](https://gitlab.haskell.org/ghc/ghc/-/pipelines/85560) | 2023-10-18 23:41:53.60398 |
| [x86_64-linux-alpine3_17-wasm-cross_wasm32-wasi-release+fully_static](https://gitlab.haskell.org/ghc/ghc/-/jobs/1696175) | [Pipeline #85610](https://gitlab.haskell.org/ghc/ghc/-/pipelines/85610) | 2023-10-20 08:03:44.795575 |
| [x86_64-linux-deb10-int_native-validate](https://gitlab.haskell.org/ghc/ghc/-/jobs/1676342) | [Pipeline #84653](https://gitlab.haskell.org/ghc/ghc/-/pipelines/84653) | 2023-09-27 05:19:55.819081 |
| [x86_64-linux-deb10-numa-slow-validate](https://gitlab.haskell.org/ghc/ghc/-/jobs/1682650) | [Pipeline #84946](https://gitlab.haskell.org/ghc/ghc/-/pipelines/84946) | 2023-10-04 09:46:12.331567 |
| [x86_64-linux-deb10-numa-slow-validate](https://gitlab.haskell.org/ghc/ghc/-/jobs/1695100) | [Pipeline #85560](https://gitlab.haskell.org/ghc/ghc/-/pipelines/85560) | 2023-10-18 23:41:53.60398 |
| [x86_64-linux-deb10-unreg-validate](https://gitlab.haskell.org/ghc/ghc/-/jobs/1690042) | [Pipeline #85309](https://gitlab.haskell.org/ghc/ghc/-/pipelines/85309) | 2023-10-11 20:04:22.784955 |
| [x86_64-linux-deb10-validate+debug_info](https://gitlab.haskell.org/ghc/ghc/-/jobs/1690835) | [Pipeline #85348](https://gitlab.haskell.org/ghc/ghc/-/pipelines/85348) | 2023-10-12 14:32:05.387576 |
| [x86_64-linux-deb11-int_native-cross_javascript-unknown-ghcjs-validate](https://gitlab.haskell.org/ghc/ghc/-/jobs/1675886) | [Pipeline #84634](https://gitlab.haskell.org/ghc/ghc/-/pipelines/84634) | 2023-09-26 19:26:02.182335 |
| [x86_64-linux-deb11-int_native-cross_javascript-unknown-ghcjs-validate](https://gitlab.haskell.org/ghc/ghc/-/jobs/1677566) | [Pipeline #84717](https://gitlab.haskell.org/ghc/ghc/-/pipelines/84717) | 2023-09-28 07:27:41.812681 |
| [x86_64-linux-deb11-int_native-cross_javascript-unknown-ghcjs-validate](https://gitlab.haskell.org/ghc/ghc/-/jobs/1679557) | [Pipeline #84812](https://gitlab.haskell.org/ghc/ghc/-/pipelines/84812) | 2023-09-30 20:11:34.546562 |
| [x86_64-linux-deb11-int_native-cross_javascript-unknown-ghcjs-validate](https://gitlab.haskell.org/ghc/ghc/-/jobs/1683726) | [Pipeline #84981](https://gitlab.haskell.org/ghc/ghc/-/pipelines/84981) | 2023-10-05 09:38:38.353623 |
| [x86_64-linux-deb11-int_native-cross_javascript-unknown-ghcjs-validate](https://gitlab.haskell.org/ghc/ghc/-/jobs/1689348) | [Pipeline #85255](https://gitlab.haskell.org/ghc/ghc/-/pipelines/85255) | 2023-10-10 23:06:03.398766 |
| [x86_64-linux-deb11-int_native-cross_javascript-unknown-ghcjs-validate](https://gitlab.haskell.org/ghc/ghc/-/jobs/1690515) | [Pipeline #85332](https://gitlab.haskell.org/ghc/ghc/-/pipelines/85332) | 2023-10-12 10:50:42.9644 |
| [x86_64-linux-deb11-int_native-cross_javascript-unknown-ghcjs-validate](https://gitlab.haskell.org/ghc/ghc/-/jobs/1692932) | [Pipeline #85454](https://gitlab.haskell.org/ghc/ghc/-/pipelines/85454) | 2023-10-14 23:20:02.15024 |
| [x86_64-linux-deb11-int_native-cross_javascript-unknown-ghcjs-validate](https://gitlab.haskell.org/ghc/ghc/-/jobs/1695105) | [Pipeline #85560](https://gitlab.haskell.org/ghc/ghc/-/pipelines/85560) | 2023-10-18 23:41:53.60398 |
| [x86_64-linux-deb11-int_native-cross_javascript-unknown-ghcjs-validate](https://gitlab.haskell.org/ghc/ghc/-/jobs/1695587) | [Pipeline #85585](https://gitlab.haskell.org/ghc/ghc/-/pipelines/85585) | 2023-10-19 15:38:16.332596 |
| [x86_64-linux-deb11-int_native-cross_javascript-unknown-ghcjs-validate](https://gitlab.haskell.org/ghc/ghc/-/jobs/1697718) | [Pipeline #85689](https://gitlab.haskell.org/ghc/ghc/-/pipelines/85689) | 2023-10-23 13:10:40.502862 |
| [x86_64-linux-fedora33-release](https://gitlab.haskell.org/ghc/ghc/-/jobs/1677567) | [Pipeline #84717](https://gitlab.haskell.org/ghc/ghc/-/pipelines/84717) | 2023-09-28 07:27:41.812681 |
```sql
with pipe as (select *
from ci_pipelines p
where p.project_id = 1
and p.status in ('success', 'failed')
and p.created_at > now() - interval '1 month'
and p.pipeline_schedule_id is null
and p.ref = 'master')
select format('[%s](%s)', j.name, 'https://gitlab.haskell.org/ghc/ghc/-/jobs/' || j.id :: text) "job",
format('[Pipeline #%s}(%s)', pipe.id, 'https://gitlab.haskell.org/ghc/ghc/-/pipelines/' || pipe.id) :: text "pipeline",
pipe.created_at
from ci_builds j
join ci_stages s
on j.stage_id = s.id
join pipe
on s.pipeline_id = pipe.id
where j.status = 'failed'
and not j.allow_failure
order by j.name, pipe.created_at
```