GHC issueshttps://gitlab.haskell.org/ghc/ghc/-/issues2020-01-05T18:32:18Zhttps://gitlab.haskell.org/ghc/ghc/-/issues/16926Hadrian: Implement cloud build artifact caching2020-01-05T18:32:18ZDavid EichmannHadrian: Implement cloud build artifact cachingShake supports cloud cache builds with it's `--share` option which caches build artifacts. Hadrian already exposes this feature, but it is unclear if it will produce correct results. Much work has already been done, but the expected bene...Shake supports cloud cache builds with it's `--share` option which caches build artifacts. Hadrian already exposes this feature, but it is unclear if it will produce correct results. Much work has already been done, but the expected benefit decreased as the cost continued to rise. Hence **this feature has been put on hold**.
If you are thinking about picking up this issue:
1. Read [developing hadrian](https://gitlab.haskell.org/ghc/ghc/wikis/Developing-Hadrian). It outlines what is needed to achieve correctness, and the lessons learned the first time around.
2. See #16400 Most remaining work is centred around fixing fsatrace lint errors. That issue roughly summarizes the remaining issues and explains how to collect such results yourself.
3. Fix #16956 to support caching independent of build and source directory location (i.e. relocatable build artifacts).
Feel free to contact me, @DavidEichmann, with any questions.⊥https://gitlab.haskell.org/ghc/ghc/-/issues/16399Error when running --share2020-01-05T18:31:39ZOleg GrenrusError when running --shareOn a fresh checkout, db039a4a10fc8fa9e03e6781d1c0dc33151beda6
```
# succeeds
./hadrian/build.sh --progress-colour=Never --share=_share -j3
...
# fails
./hadrian/build.sh --progress-colour=Never --share=_share -j3 --build-root=_build-e...On a fresh checkout, db039a4a10fc8fa9e03e6781d1c0dc33151beda6
```
# succeeds
./hadrian/build.sh --progress-colour=Never --share=_share -j3
...
# fails
./hadrian/build.sh --progress-colour=Never --share=_share -j3 --build-root=_build-experiment
...
# this succeeds ok
./hadrian/build.sh --progress-colour=Never --share=_share -j3
...
Build completed in 6.97s
# Trying without share: still fails with the same error
./hadrian/build.sh --progress-colour=Never -j3 --build-root=_build-experiment
...
| Run Ghc CompileHs Stage1: libraries/ghc-prim/GHC/Classes.hs => _build-baseline/stage1/libraries/ghc-prim/build/GHC/Classes.p_o
Error when running Shake build system:
at action, called at src/Rules.hs:35:19 in main:Rules
at need, called at src/Rules.hs:52:5 in main:Rules
* Depends on: _build-baseline/stage1/bin/ghc-pkg
at need, called at src/Utilities.hs:71:18 in main:Utilities
* Depends on: _build-baseline/stage1/libraries/integer-gmp/build/libHSinteger-gmp-1.0.2.0-ghc8.9.20190306.so
at need, called at src/Rules/Gmp.hs:14:5 in main:Rules.Gmp
* Depends on: _build-baseline/stage1/gmp/include/ghc-gmp.h
at need, called at src/Development/Shake/Internal/Derived.hs:118:15 in shake-0.17.6-ef5d6f7a92b6a347b5077f2591e0d47990d974d264275e5e4048829d4b393706:Development.Shake.Internal.Derived
* Depends on: _build-baseline/stage1/libraries/integer-gmp/build/config.mk
at error, called at src/Development/Shake/Internal/Rules/File.hs:180:58 in shake-0.17.6-ef5d6f7a92b6a347b5077f2591e0d47990d974d264275e5e4048829d4b393706:Development.Shake.Internal.Rules.File
* Raised the exception:
Error, file does not exist and no rule available:
_build-baseline/stage1/libraries/integer-gmp/build/config.mk
```
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | ---------------------- |
| Version | |
| Type | Bug |
| TypeOfFailure | OtherFailure |
| Priority | normal |
| Resolution | Unresolved |
| Component | Build System (Hadrian) |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"Error when running --share","status":"New","operating_system":"","component":"Build System (Hadrian)","related":[],"milestone":"","resolution":"Unresolved","owner":{"tag":"Unowned"},"version":"","keywords":[],"differentials":[],"test_case":"","architecture":"","cc":[""],"type":"Bug","description":"On a fresh checkout, db039a4a10fc8fa9e03e6781d1c0dc33151beda6\r\n\r\n{{{\r\n# succeeds\r\n./hadrian/build.sh --progress-colour=Never --share=_share -j3\r\n...\r\n\r\n# fails\r\n./hadrian/build.sh --progress-colour=Never --share=_share -j3 --build-root=_build-experiment\r\n...\r\n\r\n# this succeeds ok\r\n./hadrian/build.sh --progress-colour=Never --share=_share -j3\r\n...\r\nBuild completed in 6.97s\r\n\r\n# Trying without share: still fails with the same error\r\n./hadrian/build.sh --progress-colour=Never -j3 --build-root=_build-experiment\r\n...\r\n| Run Ghc CompileHs Stage1: libraries/ghc-prim/GHC/Classes.hs => _build-baseline/stage1/libraries/ghc-prim/build/GHC/Classes.p_o\r\nError when running Shake build system:\r\n at action, called at src/Rules.hs:35:19 in main:Rules\r\n at need, called at src/Rules.hs:52:5 in main:Rules\r\n* Depends on: _build-baseline/stage1/bin/ghc-pkg\r\n at need, called at src/Utilities.hs:71:18 in main:Utilities\r\n* Depends on: _build-baseline/stage1/libraries/integer-gmp/build/libHSinteger-gmp-1.0.2.0-ghc8.9.20190306.so\r\n at need, called at src/Rules/Gmp.hs:14:5 in main:Rules.Gmp\r\n* Depends on: _build-baseline/stage1/gmp/include/ghc-gmp.h\r\n at need, called at src/Development/Shake/Internal/Derived.hs:118:15 in shake-0.17.6-ef5d6f7a92b6a347b5077f2591e0d47990d974d264275e5e4048829d4b393706:Development.Shake.Internal.Derived\r\n* Depends on: _build-baseline/stage1/libraries/integer-gmp/build/config.mk\r\n at error, called at src/Development/Shake/Internal/Rules/File.hs:180:58 in shake-0.17.6-ef5d6f7a92b6a347b5077f2591e0d47990d974d264275e5e4048829d4b393706:Development.Shake.Internal.Rules.File\r\n* Raised the exception:\r\nError, file does not exist and no rule available:\r\n _build-baseline/stage1/libraries/integer-gmp/build/config.mk\r\n\r\n}}}\r\n","type_of_failure":"OtherFailure","blocking":[]} -->