GHC issueshttps://gitlab.haskell.org/ghc/ghc/-/issues2020-12-20T18:22:52Zhttps://gitlab.haskell.org/ghc/ghc/-/issues/11587Place shared objects in LIBDIR2020-12-20T18:22:52ZBen GamariPlace shared objects in LIBDIRIf one compiles a program with `-dynamic`, the resulting executable includes in its `rpath` the library directory of every Haskell package that the program links against. This causes a significant number of excess system calls at program...If one compiles a program with `-dynamic`, the resulting executable includes in its `rpath` the library directory of every Haskell package that the program links against. This causes a significant number of excess system calls at program start-up. For instance, in the case of a dynamically linked `ghc` executable on Debian 8, compiling a trivial "hello world" application produces over 800 `open` calls, the majority of which originate from the dynamic linker. e.g.,
```
$ strace -f -e open ghc-7.10.3 -c -fforce-recomp Test.hs 2>&1 | grep open
...
open("/usr/lib/ghc/bin/../haske_GGvi737nHHfG6zm2y7Rimi/tls/x86_64/libtinfo.so.5", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/usr/lib/ghc/bin/../haske_GGvi737nHHfG6zm2y7Rimi/tls/libtinfo.so.5", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/usr/lib/ghc/bin/../haske_GGvi737nHHfG6zm2y7Rimi/x86_64/libtinfo.so.5", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/usr/lib/ghc/bin/../haske_GGvi737nHHfG6zm2y7Rimi/libtinfo.so.5", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/usr/lib/ghc/bin/../termi_6iVf4EBnOgfIaaOCLRs8jl/tls/x86_64/libtinfo.so.5", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/usr/lib/ghc/bin/../termi_6iVf4EBnOgfIaaOCLRs8jl/tls/libtinfo.so.5", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/usr/lib/ghc/bin/../termi_6iVf4EBnOgfIaaOCLRs8jl/x86_64/libtinfo.so.5", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/usr/lib/ghc/bin/../termi_6iVf4EBnOgfIaaOCLRs8jl/libtinfo.so.5", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/usr/lib/ghc/bin/../ghc_0AG9TOjDEtx4Ji3wSwHOBe/tls/x86_64/libtinfo.so.5", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/usr/lib/ghc/bin/../ghc_0AG9TOjDEtx4Ji3wSwHOBe/tls/libtinfo.so.5", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/usr/lib/ghc/bin/../ghc_0AG9TOjDEtx4Ji3wSwHOBe/x86_64/libtinfo.so.5", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
...
```
The dynamic linker must look in nearly 25 Haskell library directories to locate every system library! This is madness.
Instead of placing each shared library in its own directory, `$LIBDIR/$PKG_KEY/lib$PKG_KEY.so` as we do currently, why not just place them in `$LIBDIR`, e.g. `$LIBDIR/lib$PKG_KEY.so`. This would mean that we need to include only one directory, `$LIBDIR`, in `rpath`.8.10.1https://gitlab.haskell.org/ghc/ghc/-/issues/11558Using Cabal 1.22 against GHC 8.0 results in unhelpful errors2019-07-07T18:29:49ZBen GamariUsing Cabal 1.22 against GHC 8.0 results in unhelpful errorsGHC 8.0 will require Cabal \>1.22 due to a variety of interface changes (see #10714, [D1780](https://phabricator.haskell.org/D1780), https://mail.haskell.org/pipermail/ghc-devs/2016-January/010978.html). Unfortunately things currently bl...GHC 8.0 will require Cabal \>1.22 due to a variety of interface changes (see #10714, [D1780](https://phabricator.haskell.org/D1780), https://mail.haskell.org/pipermail/ghc-devs/2016-January/010978.html). Unfortunately things currently blow up late in compilation with a seemingly unrelated error when this requirement isn't met. Namely, we complain about bad interface files for dependencies. For instance,
```
Text/Parsec/Prim.hs:85:1: error:
Bad interface file: /root/.cabal/lib/x86_64-linux-ghc-8.0.0.20160204/mtl-2.2.1-9wMhzcfsAXi7cNtl9n7svO/Control/Monad/Cont/Class.hi
Something is amiss; requested module mtl-2.2.1@mtl-2.2.1-1af5dabd2d7abb688f2145aec87badc4:Control.Monad.Cont.Class differs from name found in the interface file mtl_9wMhzcfsAXi7cNtl9n7svO:Control.Monad.Cont.Class
```
# For users seeing this error
The solution here is to install `Cabal` 1.23 or later and a `cabal-install` linking against it. Currently this requires installing `Cabal` and `cabal-install` from git. This is most easily done using GHC 7.10. Finally, you need to clear your package database.
This can be done with,
```
$ git clone git://github.com/haskell/cabal
$ cd cabal
$ cabal update
$ cabal install Cabal/ cabal-install/
$ rm -R $HOME/.ghc/*-8.0.0*
```8.0.1https://gitlab.haskell.org/ghc/ghc/-/issues/11504Can't install haskell-platform in OpenBSD2019-07-07T18:30:09ZAchifaifaCan't install haskell-platform in OpenBSDI'm trying to install the haskell platform in openbsd. This is the output I get
` Can't install freeglut-2.8.0 because of libraries
|library GL.13.0 not found
| not found anywhere
|library X11.15.1 not found
| not found anywhere
|librar...I'm trying to install the haskell platform in openbsd. This is the output I get
` Can't install freeglut-2.8.0 because of libraries
|library GL.13.0 not found
| not found anywhere
|library X11.15.1 not found
| not found anywhere
|library Xdamage.3.1 not found
| not found anywhere
|library Xext.12.0 not found
| not found anywhere
|library Xfixes.5.1 not found
| not found anywhere
|library Xi.11.1 not found
| not found anywhere
|library Xrandr.6.1 not found
| not found anywhere
|library Xrender.5.0 not found
| not found anywhere
|library Xxf86vm.5.0 not found
| not found anywhere
|library drm.3.1 not found
| not found anywhere
|library xcb.2.4 not found
| not found anywhere
Can't install hs-GLUT-2.1.2.1p11: can't resolve freeglut-2.8.0
Can't install haskell-platform-2012.4.0.0p0: can't resolve hs-GLUT-2.1.2.1p11 `
I'm installing it on a laptop without a desktop environment or X server, so I'm not sure if the problem is that I don't have a X/GL installation or if the installer can't locate some packages in the source.
I'm a total newbie to both OpenBSD and Haskell, so I know nothing that could help with this. Let me know if you need any extra information about the issue.https://gitlab.haskell.org/ghc/ghc/-/issues/11455GHC-8.0.0.20160109 runs into "Bad interface file: ... Something is amiss"2019-07-07T18:30:21ZLemmingGHC-8.0.0.20160109 runs into "Bad interface file: ... Something is amiss"When I first started testing GHC-8.0.0.20160109 I could compile a lot of packages. However, now I get the following error more and more frequently:
```
$ cabal install --with-ghc=ghc-8.0.0.20160109 --with-haddock=haddock-ghc-8.0.0.20160...When I first started testing GHC-8.0.0.20160109 I could compile a lot of packages. However, now I get the following error more and more frequently:
```
$ cabal install --with-ghc=ghc-8.0.0.20160109 --with-haddock=haddock-ghc-8.0.0.20160109 http://hackage.haskell.org/package/set-cover-0.0.8/set-cover-0.0.8.tar.gz
Downloading
http://hackage.haskell.org/package/set-cover-0.0.8/set-cover-0.0.8.tar.gz
Resolving dependencies...
Configuring enummapset-0.5.2.1...
Building enummapset-0.5.2.1...
Preprocessing library enummapset-0.5.2.1...
[1 of 5] Compiling Data.EnumSet ( Data/EnumSet.hs, dist/build/Data/EnumSet.o )
...
[4 of 5] Compiling Data.EnumMap.Lazy ( Data/EnumMap/Lazy.hs, dist/build/Data/EnumMap/Lazy.p_o )
[5 of 5] Compiling Data.EnumMap ( Data/EnumMap.hs, dist/build/Data/EnumMap.p_o )
In-place registering enummapset-0.5.2.1...
Running Haddock for enummapset-0.5.2.1...
Preprocessing library enummapset-0.5.2.1...
...
Registering enummapset-0.5.2.1...
Installed enummapset-0.5.2.1
Configuring set-cover-0.0.8...
Building set-cover-0.0.8...
Preprocessing library set-cover-0.0.8...
[ 1 of 15] Compiling Math.SetCover.Cuboid ( src/Math/SetCover/Cuboid.hs, dist/build/Math/SetCover/Cuboid.o )
[ 2 of 15] Compiling Math.SetCover.Queue ( src/Math/SetCover/Queue.hs, dist/build/Math/SetCover/Queue.o )
src/Math/SetCover/Queue.hs:3:1: error:
Bad interface file: /home/cabal/lib/x86_64-linux-ghc-8.0.0.20160109/enummapset-0.5.2.1-KxKeewRMTNa4DSa3VnUuRQ/Data/EnumMap.hi
Something is amiss; requested module enummapset-0.5.2.1@enummapset-0.5.2.1-64948c41f2d75b38829fcb75933a73f0:Data.EnumMap differs from name found in the interface file enumm_KxKeewRMTNa4DSa3VnUuRQ:Data.EnumMap
src/Math/SetCover/Queue.hs:4:1: error:
Bad interface file: /home/cabal/lib/x86_64-linux-ghc-8.0.0.20160109/enummapset-0.5.2.1-KxKeewRMTNa4DSa3VnUuRQ/Data/EnumSet.hi
Something is amiss; requested module enummapset-0.5.2.1@enummapset-0.5.2.1-64948c41f2d75b38829fcb75933a73f0:Data.EnumSet differs from name found in the interface file enumm_KxKeewRMTNa4DSa3VnUuRQ:Data.EnumSet
Failed to install set-cover-0.0.8
```
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | -------------- |
| Version | 8.0.1-rc1 |
| Type | Bug |
| TypeOfFailure | OtherFailure |
| Priority | normal |
| Resolution | Unresolved |
| Component | Package system |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"GHC-8.0.0.20160109 runs into \"Bad interface file: ... Something is amiss\"","status":"New","operating_system":"","component":"Package system","related":[],"milestone":"","resolution":"Unresolved","owner":{"tag":"Unowned"},"version":"8.0.1-rc1","keywords":[],"differentials":[],"test_case":"","architecture":"","cc":[""],"type":"Bug","description":"When I first started testing GHC-8.0.0.20160109 I could compile a lot of packages. However, now I get the following error more and more frequently:\r\n{{{\r\n$ cabal install --with-ghc=ghc-8.0.0.20160109 --with-haddock=haddock-ghc-8.0.0.20160109 http://hackage.haskell.org/package/set-cover-0.0.8/set-cover-0.0.8.tar.gz\r\nDownloading\r\nhttp://hackage.haskell.org/package/set-cover-0.0.8/set-cover-0.0.8.tar.gz\r\nResolving dependencies...\r\nConfiguring enummapset-0.5.2.1...\r\nBuilding enummapset-0.5.2.1...\r\nPreprocessing library enummapset-0.5.2.1...\r\n[1 of 5] Compiling Data.EnumSet ( Data/EnumSet.hs, dist/build/Data/EnumSet.o )\r\n...\r\n[4 of 5] Compiling Data.EnumMap.Lazy ( Data/EnumMap/Lazy.hs, dist/build/Data/EnumMap/Lazy.p_o )\r\n[5 of 5] Compiling Data.EnumMap ( Data/EnumMap.hs, dist/build/Data/EnumMap.p_o )\r\nIn-place registering enummapset-0.5.2.1...\r\nRunning Haddock for enummapset-0.5.2.1...\r\nPreprocessing library enummapset-0.5.2.1...\r\n...\r\nRegistering enummapset-0.5.2.1...\r\nInstalled enummapset-0.5.2.1\r\nConfiguring set-cover-0.0.8...\r\nBuilding set-cover-0.0.8...\r\nPreprocessing library set-cover-0.0.8...\r\n[ 1 of 15] Compiling Math.SetCover.Cuboid ( src/Math/SetCover/Cuboid.hs, dist/build/Math/SetCover/Cuboid.o )\r\n[ 2 of 15] Compiling Math.SetCover.Queue ( src/Math/SetCover/Queue.hs, dist/build/Math/SetCover/Queue.o )\r\n\r\nsrc/Math/SetCover/Queue.hs:3:1: error:\r\n Bad interface file: /home/cabal/lib/x86_64-linux-ghc-8.0.0.20160109/enummapset-0.5.2.1-KxKeewRMTNa4DSa3VnUuRQ/Data/EnumMap.hi\r\n Something is amiss; requested module enummapset-0.5.2.1@enummapset-0.5.2.1-64948c41f2d75b38829fcb75933a73f0:Data.EnumMap differs from name found in the interface file enumm_KxKeewRMTNa4DSa3VnUuRQ:Data.EnumMap\r\n\r\nsrc/Math/SetCover/Queue.hs:4:1: error:\r\n Bad interface file: /home/cabal/lib/x86_64-linux-ghc-8.0.0.20160109/enummapset-0.5.2.1-KxKeewRMTNa4DSa3VnUuRQ/Data/EnumSet.hi\r\n Something is amiss; requested module enummapset-0.5.2.1@enummapset-0.5.2.1-64948c41f2d75b38829fcb75933a73f0:Data.EnumSet differs from name found in the interface file enumm_KxKeewRMTNa4DSa3VnUuRQ:Data.EnumSet\r\nFailed to install set-cover-0.0.8\r\n}}}","type_of_failure":"OtherFailure","blocking":[]} -->https://gitlab.haskell.org/ghc/ghc/-/issues/11286ghc-pkg library2023-03-13T17:14:21ZEdward Z. Yangghc-pkg libraryOn Twitter, Anthony Cowley was wondering why there was no GHC API functions for taking Cabal InstalledPackageInfos and turning them into GHC InstalledPackageInfos. https://twitter.com/a_cowley/status/680158885953564672
No such interface...On Twitter, Anthony Cowley was wondering why there was no GHC API functions for taking Cabal InstalledPackageInfos and turning them into GHC InstalledPackageInfos. https://twitter.com/a_cowley/status/680158885953564672
No such interface exists: to keep GHC decoupled from Cabal, the GHC package representation is mediated solely by `ghc-pkg`, so the "correct" way to create a package config file, run `ghc-pkg` to load it into the GHC binary representation, and go from there.
It seems like it would be convenient if `ghc-pkg` was libified, and so people who linked against GHC and Cabal could just directly do the conversion without going through `ghc-pkg`. It's unclear what the right interface is; ghc-pkg does a number of sanity checks and it's unclear if those should be libified too. Any such library also must be uploaded to Hackage, because if you want to link against a newer version of Cabal you have to rebuild the ghc-pkg library against the newest version of Cabal.
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | -------------- |
| Version | 7.11 |
| Type | FeatureRequest |
| TypeOfFailure | OtherFailure |
| Priority | normal |
| Resolution | Unresolved |
| Component | Package system |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | acowley |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"ghc-pkg library","status":"New","operating_system":"","component":"Package system","related":[],"milestone":"","resolution":"Unresolved","owner":{"tag":"Unowned"},"version":"7.11","keywords":[],"differentials":[],"test_case":"","architecture":"","cc":["acowley"],"type":"FeatureRequest","description":"On Twitter, Anthony Cowley was wondering why there was no GHC API functions for taking Cabal InstalledPackageInfos and turning them into GHC InstalledPackageInfos. https://twitter.com/a_cowley/status/680158885953564672\r\n\r\nNo such interface exists: to keep GHC decoupled from Cabal, the GHC package representation is mediated solely by `ghc-pkg`, so the \"correct\" way to create a package config file, run `ghc-pkg` to load it into the GHC binary representation, and go from there.\r\n\r\nIt seems like it would be convenient if `ghc-pkg` was libified, and so people who linked against GHC and Cabal could just directly do the conversion without going through `ghc-pkg`. It's unclear what the right interface is; ghc-pkg does a number of sanity checks and it's unclear if those should be libified too. Any such library also must be uploaded to Hackage, because if you want to link against a newer version of Cabal you have to rebuild the ghc-pkg library against the newest version of Cabal.","type_of_failure":"OtherFailure","blocking":[]} -->https://gitlab.haskell.org/ghc/ghc/-/issues/11124GHC does not shadow -package-name/-this-package-key2019-07-07T18:31:51ZEdward Z. YangGHC does not shadow -package-name/-this-package-keySteps to reproduce:
1. Install a package named `a` with these contents:
\*\*a.cabal\*\*
```
name: a
version: 0.1.0.0
cabal-version: >=1.10
library
exposed-modules: A
build-depends: bas...Steps to reproduce:
1. Install a package named `a` with these contents:
\*\*a.cabal\*\*
```
name: a
version: 0.1.0.0
cabal-version: >=1.10
library
exposed-modules: A
build-depends: base
```
- \*A.hs\*\*
```
module A where
data A = A String
```
> Note what unit ID/package key/package name the package installs as. On GHC 7.8 it will be something like `a-0.1.0.0`, on GHC 7.10 it will be something like `a_LKCPrTJwOTOLk4OU37YmeN`, on GHC 8.0 it will be something like `a-0.1.0.0-LKCPrTJwOTOLk4OU37Yme`.
1. Install a package named `b` with these contents:
\*\*b.cabal\*\*
```
name: b
version: 0.1.0.0
cabal-version: >=1.10
library
exposed-modules: B
build-depends: base, a
```
- \*b.hs\*\*
```
module B where
import A
b = A "foo"
```
1. Go back to `a`, and load `A.hs` in GHCi with `ghci A.hs -package-name a-0.1.0.0 -package-db ../db`. Notice that we can `import B` from this GHCi session:
```
ezyang@sabre:~/Dev/labs/ii/a$ ghci A.hs -package-name a-0.1.0.0 -package-db ../db
GHCi, version 7.6.3: http://www.haskell.org/ghc/ :? for help
Loading package ghc-prim ... linking ... done.
Loading package integer-gmp ... linking ... done.
Loading package base ... linking ... done.
[1 of 1] Compiling A ( A.hs, interpreted )
Ok, modules loaded: A.
*A> import B
*A B> b
Loading package a-0.1.0.0 ... linking ... done.
Loading package b-0.1.0.0 ... linking ... done.
A "asdfsdf"
*A B>
Leaving GHCi.
```
1. Edit `A.hs` so that now contains:
```
module A where
data A = A Int
```
1. Load in GHCi again:
```
ezyang@sabre:~/Dev/labs/ii/a$ ghci A.hs -package-name a-0.1.0.0 -package-db ../db
GHCi, version 7.6.3: http://www.haskell.org/ghc/ :? for help
Loading package ghc-prim ... linking ... done.
Loading package integer-gmp ... linking ... done.
Loading package base ... linking ... done.
[1 of 1] Compiling A ( A.hs, interpreted )
Ok, modules loaded: A.
*A> import B
*A B> b
Loading package a-0.1.0.0 ... linking ... done.
Loading package b-0.1.0.0 ... linking ... done.
A 3458765062984598387
```
Disaster!
Actually, this bug is relatively harmless:
1. If you try to actually reinstall package a, that's a "dangerous" reinstall and it's known that reinstalls can break things.
1. It's hard to convince Cabal to trigger this, since importing `B` requires `b` in the build-depends, and that would constitute a circular package dependency. (You can't separate it out because of (1))
1. If you don't specify `-package-name` (or similar) the local `A` and the remote `A` will have different identities, no problem.
1. When you do specify `-package-name`, you can't build a main executable (so the only way to run is by loading into GHCi, as shown above.)
1. Sometimes, GHCi will notice that something is amiss if it tries to load the object file for both the remote A and the local A.
But the fix seems simple: if a user specifies `-package-name`, it should be AS IF they also specified `-ignore-package`.
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | -------------- |
| Version | 7.11 |
| Type | Bug |
| TypeOfFailure | OtherFailure |
| Priority | low |
| Resolution | Unresolved |
| Component | Package system |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"GHC does not shadow -package-name/-this-package-key","status":"New","operating_system":"","component":"Package system","related":[],"milestone":"","resolution":"Unresolved","owner":{"tag":"Unowned"},"version":"7.11","keywords":[],"differentials":[],"test_case":"","architecture":"","cc":[""],"type":"Bug","description":"Steps to reproduce:\r\n\r\n1. Install a package named `a` with these contents:\r\n\r\n **a.cabal**\r\n{{{\r\nname: a\r\nversion: 0.1.0.0\r\ncabal-version: >=1.10\r\n\r\nlibrary\r\n exposed-modules: A\r\n build-depends: base\r\n}}}\r\n\r\n **A.hs**\r\n{{{\r\nmodule A where\r\ndata A = A String\r\n}}}\r\n\r\n Note what unit ID/package key/package name the package installs as. On GHC 7.8 it will be something like `a-0.1.0.0`, on GHC 7.10 it will be something like `a_LKCPrTJwOTOLk4OU37YmeN`, on GHC 8.0 it will be something like `a-0.1.0.0-LKCPrTJwOTOLk4OU37Yme`.\r\n\r\n2. Install a package named `b` with these contents:\r\n\r\n **b.cabal**\r\n{{{\r\nname: b\r\nversion: 0.1.0.0\r\ncabal-version: >=1.10\r\n\r\nlibrary\r\n exposed-modules: B\r\n build-depends: base, a\r\n}}}\r\n\r\n **b.hs**\r\n{{{\r\nmodule B where\r\nimport A\r\nb = A \"foo\"\r\n}}}\r\n\r\n3. Go back to `a`, and load `A.hs` in GHCi with `ghci A.hs -package-name a-0.1.0.0 -package-db ../db`. Notice that we can `import B` from this GHCi session:\r\n\r\n{{{\r\nezyang@sabre:~/Dev/labs/ii/a$ ghci A.hs -package-name a-0.1.0.0 -package-db ../db\r\nGHCi, version 7.6.3: http://www.haskell.org/ghc/ :? for help\r\nLoading package ghc-prim ... linking ... done.\r\nLoading package integer-gmp ... linking ... done.\r\nLoading package base ... linking ... done.\r\n[1 of 1] Compiling A ( A.hs, interpreted )\r\nOk, modules loaded: A.\r\n*A> import B\r\n*A B> b\r\nLoading package a-0.1.0.0 ... linking ... done.\r\nLoading package b-0.1.0.0 ... linking ... done.\r\nA \"asdfsdf\"\r\n*A B> \r\nLeaving GHCi.\r\n}}}\r\n\r\n3. Edit `A.hs` so that now contains:\r\n\r\n{{{\r\nmodule A where\r\ndata A = A Int\r\n}}}\r\n\r\n4. Load in GHCi again:\r\n\r\n{{{\r\nezyang@sabre:~/Dev/labs/ii/a$ ghci A.hs -package-name a-0.1.0.0 -package-db ../db\r\nGHCi, version 7.6.3: http://www.haskell.org/ghc/ :? for help\r\nLoading package ghc-prim ... linking ... done.\r\nLoading package integer-gmp ... linking ... done.\r\nLoading package base ... linking ... done.\r\n[1 of 1] Compiling A ( A.hs, interpreted )\r\nOk, modules loaded: A.\r\n*A> import B\r\n*A B> b\r\nLoading package a-0.1.0.0 ... linking ... done.\r\nLoading package b-0.1.0.0 ... linking ... done.\r\nA 3458765062984598387\r\n}}}\r\n\r\nDisaster!\r\n\r\nActually, this bug is relatively harmless:\r\n\r\n1. If you try to actually reinstall package a, that's a \"dangerous\" reinstall and it's known that reinstalls can break things.\r\n2. It's hard to convince Cabal to trigger this, since importing `B` requires `b` in the build-depends, and that would constitute a circular package dependency. (You can't separate it out because of (1))\r\n3. If you don't specify `-package-name` (or similar) the local `A` and the remote `A` will have different identities, no problem.\r\n4. When you do specify `-package-name`, you can't build a main executable (so the only way to run is by loading into GHCi, as shown above.)\r\n5. Sometimes, GHCi will notice that something is amiss if it tries to load the object file for both the remote A and the local A.\r\n\r\nBut the fix seems simple: if a user specifies `-package-name`, it should be AS IF they also specified `-ignore-package`.","type_of_failure":"OtherFailure","blocking":[]} -->https://gitlab.haskell.org/ghc/ghc/-/issues/11025Per-database shadowing2019-07-07T18:32:34ZEdward Z. YangPer-database shadowingIn `5b0191f74ab05b187f81ea037623338a615b1619`, I eliminated shadowing, on the justification that we should never pick the same component ID (previously installed package ID) for two distinct packages unless the ABIs of the packages were ...In `5b0191f74ab05b187f81ea037623338a615b1619`, I eliminated shadowing, on the justification that we should never pick the same component ID (previously installed package ID) for two distinct packages unless the ABIs of the packages were the same, so we could just globally assume that component IDs are unique.
However, I've recently come across some cases where we can't easily actually make this be the case. Here are a few of them:
1. GHC's build system assumes that the ghc package will always be a library name of the form HSghc-7.11.a (i.e. that its component ID is ghc-7.11). It's a bit nontrivial to change this assumption, because there are stage1 shenanigans which are rewriting the version number to one without the date. But obviously if you are bootstrapping GHC with itself, the component ID of the GHC you are building will conflict with the component ID of the host GHC.
1. A similar situation exist for bootstrapping packages. When compiling GHC, we want to pick fairly deterministic component IDs for it in order to avoid needless rebuilding. But then if you turn around and use one of these builds to bootstrap GHC, if you use the same deterministic scheme you will end up with conflicting component IDs.
1. More generally, if you are building a package which is already in the global database (and thus not easily removable), it is sometimes VERY CONVENIENT to be able to pick whatever component ID you want, even if it's in the global database. (1) and (2) are cases of this.
So I think what we want to do is bring back shadowing, but have it operate on a PER-DATABASE basis. For example, if we have the following databases:
```
pkg.conf.1
foo-0.1-XXX
bar-0.2-YYY (depends on foo-0.1-XXX)
pkg.conf.2
foo-0.1-XXX
baz-0.3-ZZZ (depends on foo-0.1-XXX)
```
If we stack pkg.conf.2 on top of pkg.conf.1, we invalidate foo-0.1-XXX FROM pkg.conf.1 (which simply means any packages from pkg.conf.1 which refer to it.)
It also occurs to me that we should also be recording the ABIs of packages we depend on, to ensure consistency. Consider:
```
pkg.conf.1
foo-0.1-XXX (abi is AAA)
pkg.conf.2
foo-0.1-XXX (abi is BBB)
pkg.conf.3
bar-0.1-YYY (depends on foo-0.1-XXX)
```
Which is the correct database to use `pkg.conf.3` with? We have no way of telling today! Sure, `pkg.conf.3` is incomplete, but this is a supported mode of operation.
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | -------------- |
| Version | 7.11 |
| Type | Bug |
| TypeOfFailure | OtherFailure |
| Priority | high |
| Resolution | Unresolved |
| Component | Package system |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"Per-database shadowing","status":"New","operating_system":"","component":"Package system","related":[],"milestone":"8.0.1","resolution":"Unresolved","owner":{"tag":"OwnedBy","contents":"ezyang"},"version":"7.11","keywords":[],"differentials":[],"test_case":"","architecture":"","cc":[""],"type":"Bug","description":"In `5b0191f74ab05b187f81ea037623338a615b1619`, I eliminated shadowing, on the justification that we should never pick the same component ID (previously installed package ID) for two distinct packages unless the ABIs of the packages were the same, so we could just globally assume that component IDs are unique.\r\n\r\nHowever, I've recently come across some cases where we can't easily actually make this be the case. Here are a few of them:\r\n\r\n1. GHC's build system assumes that the ghc package will always be a library name of the form HSghc-7.11.a (i.e. that its component ID is ghc-7.11). It's a bit nontrivial to change this assumption, because there are stage1 shenanigans which are rewriting the version number to one without the date. But obviously if you are bootstrapping GHC with itself, the component ID of the GHC you are building will conflict with the component ID of the host GHC.\r\n\r\n2. A similar situation exist for bootstrapping packages. When compiling GHC, we want to pick fairly deterministic component IDs for it in order to avoid needless rebuilding. But then if you turn around and use one of these builds to bootstrap GHC, if you use the same deterministic scheme you will end up with conflicting component IDs.\r\n\r\n3. More generally, if you are building a package which is already in the global database (and thus not easily removable), it is sometimes VERY CONVENIENT to be able to pick whatever component ID you want, even if it's in the global database. (1) and (2) are cases of this.\r\n\r\nSo I think what we want to do is bring back shadowing, but have it operate on a PER-DATABASE basis. For example, if we have the following databases:\r\n\r\n{{{\r\npkg.conf.1\r\n foo-0.1-XXX\r\n bar-0.2-YYY (depends on foo-0.1-XXX)\r\npkg.conf.2\r\n foo-0.1-XXX\r\n baz-0.3-ZZZ (depends on foo-0.1-XXX)\r\n}}}\r\n\r\nIf we stack pkg.conf.2 on top of pkg.conf.1, we invalidate foo-0.1-XXX FROM pkg.conf.1 (which simply means any packages from pkg.conf.1 which refer to it.)\r\n\r\nIt also occurs to me that we should also be recording the ABIs of packages we depend on, to ensure consistency. Consider:\r\n\r\n{{{\r\npkg.conf.1\r\n foo-0.1-XXX (abi is AAA)\r\npkg.conf.2\r\n foo-0.1-XXX (abi is BBB)\r\npkg.conf.3\r\n bar-0.1-YYY (depends on foo-0.1-XXX)\r\n}}}\r\n\r\nWhich is the correct database to use `pkg.conf.3` with? We have no way of telling today! Sure, `pkg.conf.3` is incomplete, but this is a supported mode of operation.","type_of_failure":"OtherFailure","blocking":[]} -->8.0.1Edward Z. YangEdward Z. Yanghttps://gitlab.haskell.org/ghc/ghc/-/issues/11023ghci and ghc-pkg disagree about what's exposed2019-07-07T18:32:34Zdmwitghci and ghc-pkg disagree about what's exposedI have installed vector-0.10.12.3 and vector-0.11.0.0. At some point, I had hidden both from the package database; however, I later used ghc-pkg expose to mark one as visible again. Now I seem to be in a strange state where ghci and ghc-...I have installed vector-0.10.12.3 and vector-0.11.0.0. At some point, I had hidden both from the package database; however, I later used ghc-pkg expose to mark one as visible again. Now I seem to be in a strange state where ghci and ghc-pkg disagree about what is hidden:
```
% ghci-7.10.2
GHCi, version 7.10.2: http://www.haskell.org/ghc/ :? for help
Prelude> :m +Data.Vector.Unboxed
<no location info>:
Could not find module ‘Data.Vector.Unboxed’
It is a member of the hidden package ‘vector-0.11.0.0@vecto_3jMaUrldidp1bqsrn0qsS2’.
It is a member of the hidden package ‘vector-0.10.12.3@vecto_1COyUuV1LrA1IjYnWfJnbs’.
Prelude>
Leaving GHCi.
% ghc-pkg-7.10.2 list vector | cat
/usr/local/lib/ghc-7.10.2/package.conf.d:
(no packages)
/home/dmwit/.ghc/x86_64-linux-7.10.2/package.conf.d:
vector-0.10.12.3
(vector-0.11.0.0)
% ghc-pkg-7.10.2 describe vector-0.10.12.3 | grep 'key:\|exposed:'
key: vecto_1COyUuV1LrA1IjYnWfJnbs
exposed: True
```
I have tried reproducing this with other packages in a handful of ways and failed; so I can't give instructions for reproducing. But I am happy to perform any diagnostics you can think of.
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | -------------- |
| Version | 7.10.2 |
| Type | Bug |
| TypeOfFailure | OtherFailure |
| Priority | normal |
| Resolution | Unresolved |
| Component | Package system |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"ghci and ghc-pkg disagree about what's exposed","status":"New","operating_system":"","component":"Package system","related":[],"milestone":"","resolution":"Unresolved","owner":{"tag":"Unowned"},"version":"7.10.2","keywords":[],"differentials":[],"test_case":"","architecture":"","cc":[""],"type":"Bug","description":"I have installed vector-0.10.12.3 and vector-0.11.0.0. At some point, I had hidden both from the package database; however, I later used ghc-pkg expose to mark one as visible again. Now I seem to be in a strange state where ghci and ghc-pkg disagree about what is hidden:\r\n\r\n{{{\r\n% ghci-7.10.2 \r\nGHCi, version 7.10.2: http://www.haskell.org/ghc/ :? for help\r\nPrelude> :m +Data.Vector.Unboxed\r\n\r\n<no location info>:\r\n Could not find module ‘Data.Vector.Unboxed’\r\n It is a member of the hidden package ‘vector-0.11.0.0@vecto_3jMaUrldidp1bqsrn0qsS2’.\r\n It is a member of the hidden package ‘vector-0.10.12.3@vecto_1COyUuV1LrA1IjYnWfJnbs’.\r\nPrelude> \r\nLeaving GHCi.\r\n% ghc-pkg-7.10.2 list vector | cat\r\n/usr/local/lib/ghc-7.10.2/package.conf.d:\r\n (no packages)\r\n/home/dmwit/.ghc/x86_64-linux-7.10.2/package.conf.d:\r\n vector-0.10.12.3\r\n (vector-0.11.0.0)\r\n\r\n% ghc-pkg-7.10.2 describe vector-0.10.12.3 | grep 'key:\\|exposed:'\r\nkey: vecto_1COyUuV1LrA1IjYnWfJnbs\r\nexposed: True\r\n}}}\r\n\r\nI have tried reproducing this with other packages in a handful of ways and failed; so I can't give instructions for reproducing. But I am happy to perform any diagnostics you can think of.","type_of_failure":"OtherFailure","blocking":[]} -->https://gitlab.haskell.org/ghc/ghc/-/issues/10912Support for out of the box static linking2019-07-07T18:33:05Zcrb002Support for out of the box static linkingWhen running in environments like AWS Lambda you don't have access to basic shared libraries Haskell needs like libgmp, libffi. Need good support for static linked libraries of anything included in a basic GHC distribution.
https://gith...When running in environments like AWS Lambda you don't have access to basic shared libraries Haskell needs like libgmp, libffi. Need good support for static linked libraries of anything included in a basic GHC distribution.
https://github.com/commercialhaskell/stack/issues/1032
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | -------------- |
| Version | 7.10.2 |
| Type | FeatureRequest |
| TypeOfFailure | OtherFailure |
| Priority | normal |
| Resolution | Unresolved |
| Component | Package system |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"Support for out of the box static linking","status":"New","operating_system":"","component":"Package system","related":[],"milestone":"","resolution":"Unresolved","owner":{"tag":"Unowned"},"version":"7.10.2","keywords":[],"differentials":[],"test_case":"","architecture":"","cc":[""],"type":"FeatureRequest","description":"When running in environments like AWS Lambda you don't have access to basic shared libraries Haskell needs like libgmp, libffi. Need good support for static linked libraries of anything included in a basic GHC distribution.\r\n\r\nhttps://github.com/commercialhaskell/stack/issues/1032","type_of_failure":"OtherFailure","blocking":[]} -->https://gitlab.haskell.org/ghc/ghc/-/issues/10798Signatures with only types should not be included in unit keys2019-07-07T18:33:50ZEdward Z. YangSignatures with only types should not be included in unit keysSuppose we want to modularize the dependence on `p` in this program:
```
unit p where
module A where
data T = T
mkT = T
unit q where
include p
module B where
import A
bar = mkT
```
The obvious signature to write f...Suppose we want to modularize the dependence on `p` in this program:
```
unit p where
module A where
data T = T
mkT = T
unit q where
include p
module B where
import A
bar = mkT
```
The obvious signature to write for `A` is:
```
signature A where
data T
mkT :: T
```
But this presupposes an implementation of `A` which exports `T`. But `B` doesn't use any export of `T`, and an equally valid implementation would be if `T` was defined in some `Types` module and then imported here.
But suppose we change our signature to be:
```
signature A.Types where
data T
signature A where
import A.Types
mkT :: T
```
This is maximally general, but requires that the module which exports `T` be named `A.Types`. If someone puts the type anywhere else, we have to rename the signature to the real place it was defined, or make a dummy implementation module which reexports the type in question.
Now there is a curious thing, which is that the choice of module we use to fill these extra signature modules currently influences the type identity of anything else in the unit. But we never rely on any code from these signatures: if I make two distinct dummy modules to set `T` to the same type, this really shouldn't have any impact (at all!) on the code generated.
So, my suggestion is that if a signature contains only types (perhaps they could be named something else, like `tysignature`), they should not count towards the calculation of a unit key. This means that a user can freely create dummy modules to fill in these types when they are instantiating; all that is being done is helping Backpack figure out what the identities of types are. (If we wanted to be fancy, Backpack could even look inside type signatures to determine some types, but let's not go there for now.)
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | -------------- |
| Version | 7.11 |
| Type | FeatureRequest |
| TypeOfFailure | OtherFailure |
| Priority | normal |
| Resolution | Unresolved |
| Component | Package system |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"Signatures with only types should not be included in unit keys","status":"New","operating_system":"","component":"Package system","related":[],"milestone":"⊥","resolution":"Unresolved","owner":{"tag":"OwnedBy","contents":"ezyang"},"version":"7.11","keywords":["backpack"],"differentials":[],"test_case":"","architecture":"","cc":[""],"type":"FeatureRequest","description":"Suppose we want to modularize the dependence on `p` in this program:\r\n\r\n{{{\r\nunit p where\r\n module A where\r\n data T = T\r\n mkT = T\r\nunit q where\r\n include p\r\n module B where\r\n import A\r\n bar = mkT\r\n}}}\r\n\r\nThe obvious signature to write for `A` is:\r\n\r\n{{{\r\nsignature A where\r\n data T\r\n mkT :: T\r\n}}}\r\n\r\nBut this presupposes an implementation of `A` which exports `T`. But `B` doesn't use any export of `T`, and an equally valid implementation would be if `T` was defined in some `Types` module and then imported here.\r\n\r\nBut suppose we change our signature to be:\r\n\r\n{{{\r\nsignature A.Types where\r\n data T\r\nsignature A where\r\n import A.Types\r\n mkT :: T\r\n}}}\r\n\r\nThis is maximally general, but requires that the module which exports `T` be named `A.Types`. If someone puts the type anywhere else, we have to rename the signature to the real place it was defined, or make a dummy implementation module which reexports the type in question.\r\n\r\nNow there is a curious thing, which is that the choice of module we use to fill these extra signature modules currently influences the type identity of anything else in the unit. But we never rely on any code from these signatures: if I make two distinct dummy modules to set `T` to the same type, this really shouldn't have any impact (at all!) on the code generated.\r\n\r\nSo, my suggestion is that if a signature contains only types (perhaps they could be named something else, like `tysignature`), they should not count towards the calculation of a unit key. This means that a user can freely create dummy modules to fill in these types when they are instantiating; all that is being done is helping Backpack figure out what the identities of types are. (If we wanted to be fancy, Backpack could even look inside type signatures to determine some types, but let's not go there for now.)","type_of_failure":"OtherFailure","blocking":[]} -->⊥Edward Z. YangEdward Z. Yanghttps://gitlab.haskell.org/ghc/ghc/-/issues/10723Make declarations in signatures "weakly bound" until they are used2019-07-07T18:34:19ZEdward Z. YangMake declarations in signatures "weakly bound" until they are usedSuppose you are the author of a library in a Backpack world, and you publish a signature package which defines the entire public facing interface of your library. The library `foo` which uses of your library decides to `include` the sign...Suppose you are the author of a library in a Backpack world, and you publish a signature package which defines the entire public facing interface of your library. The library `foo` which uses of your library decides to `include` the signature package for convenience, but actually only uses a small portion of the API.
Later, you make a BC-breaking change in one part of the library and release a new signature package. The library `bar` which uses your library includes this NEW signature package, using a different portion of the API which was unaffected by the by the BC change.
Now, a hapless user tries to use `foo` and `bar`, but Backpack complains that the requirements are not compatible.
What's the problem here? The practice of writing reusable signature packages for people to use caused the requirements of `foo` and `bar` to become too large, since they included a lot of junk that these libraries didn't actually use. It would be far better if you could `include` a signature package, but only "require" the bits of it that you actually used!
How can we achieve this?
1. We augment the `ModIface` of signature merges (#10690) to record whether or not a declaration was (transitively) used or not by some module. Used declarations must be filled, but unused ones are treated more flexibly: if they are merged with a different, incompatible but used requirement, they disappear, and we don't check if an implementing module actually implemented the declaration. (If two unused incompatible requirements are merged, we just erase the name.)
1. How do we compute the usage info? I think it will have to be done during shaping (which runs the renamer). We only need to annotate each declaration a signature with the transitive set of names from other signatures that it has used--this can be incrementally computed. (It's not necessary to annotate declarations in modules, since they are always assumed to use holes). Then whenever a declaration from a signature is used in a module, we mark its transitive set as used. This information can then be used later when constructing the merged `ModIface` which represents the "public requirement" of the package.
So, for example, a package containing only signatures would contain all unused declarations (however, they may start being used by a package which includes them). Any unused declaration which isn't mixed with another incompatible declaration can be imported (causing it to be used), but we will complain if you try to use a name and we can't tell which declaration to use.
(PS: another moral here, is that `include`s are bad UNLESS you are including a signature package! Because an include for a concrete module is a dependency you can't override...)Edward Z. YangEdward Z. Yanghttps://gitlab.haskell.org/ghc/ghc/-/issues/10714After implementing new installed package ID (hash of sdist), get rid of packa...2019-07-07T18:34:26ZEdward Z. YangAfter implementing new installed package ID (hash of sdist), get rid of package keysGHC tracking bug for https://github.com/haskell/cabal/issues/2745
Externally, GHC's flags do not have to change much; a user simply passes the installed package ID to the flag currently named `-this-package-key` (but perhaps we should r...GHC tracking bug for https://github.com/haskell/cabal/issues/2745
Externally, GHC's flags do not have to change much; a user simply passes the installed package ID to the flag currently named `-this-package-key` (but perhaps we should rename this.)
Internally, if we can assume that `PackageKey == InstalledPackageId`, we can do away with the `InstalledPackageId` map and get rid of the level of indirection between the bin-pkg-db (which records installed package IDs\`) and GHC's guts (which record package keys).
Blocked on Cabal not actually using ABI hashes to identify packages.
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | -------------- |
| Version | 7.10.2 |
| Type | Bug |
| TypeOfFailure | OtherFailure |
| Priority | normal |
| Resolution | Unresolved |
| Component | Package system |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"After implementing new installed package ID (hash of sdist), get rid of package keys","status":"New","operating_system":"","component":"Package system","related":[],"milestone":"","resolution":"Unresolved","owner":{"tag":"OwnedBy","contents":"ezyang"},"version":"7.10.2","keywords":[],"differentials":[],"test_case":"","architecture":"","cc":[""],"type":"Bug","description":"GHC tracking bug for https://github.com/haskell/cabal/issues/2745\r\n\r\nExternally, GHC's flags do not have to change much; a user simply passes the installed package ID to the flag currently named `-this-package-key` (but perhaps we should rename this.)\r\n\r\nInternally, if we can assume that `PackageKey == InstalledPackageId`, we can do away with the `InstalledPackageId` map and get rid of the level of indirection between the bin-pkg-db (which records installed package IDs`) and GHC's guts (which record package keys). \r\n\r\nBlocked on Cabal not actually using ABI hashes to identify packages.","type_of_failure":"OtherFailure","blocking":[]} -->8.0.1Edward Z. YangEdward Z. Yanghttps://gitlab.haskell.org/ghc/ghc/-/issues/10680Make Backpack order-independent (again)2019-07-07T18:34:41ZEdward Z. YangMake Backpack order-independent (again)When we moved to the new `bkp` file format, we also went back to the a format which is order-dependent: that is to say, the order in which you put the declarations matters. So if you write:
```
unit p where
module A where
import B...When we moved to the new `bkp` file format, we also went back to the a format which is order-dependent: that is to say, the order in which you put the declarations matters. So if you write:
```
unit p where
module A where
import B
module B where
...
```
this fails to type-check, GHC complaining that `B` is not in scope. I did this, in part because it's what the Backpack paper described, and because it was "simpler" to implement.
I think we should move back to an order-independent scheme, for the following reasons:
1. Haskell users are used to not needing pay particularly close attention to the ordering of their modules, and forcing people to linearize their module descriptions would be spectacularly disruptive with large amounts of modules. So un-ordered modules are "more natural for a traditional Haskell user.
1. Order-independence imposes some constraints on how expressive programs are (with order-dependent Backpack, you can do some pretty tricky things by ordering things certain ways); this could simplify some aspects of compiler implementation and make Backpack easier to explain.
1. A particular case of (2): it seems a lot simpler UX-wise to let a user assume that if you import a module `M` in a unit, it doesn't matter where you import it: you always get the same set of identifiers brought into scope. Thus, the incremental results of signatures should not be visible, c.f. #10679
The main idea is that only the surface-syntax is un-ordered: the internal representation of units is a DAG which we work out in an elaboration phase, not altogether unsimilar from what `GhcMake` computes. An important auxiliary idea is that `import A` where `A` is backed by some signatures depends on EVERY signature in scope.
Here are the details:
- \*The intermediate representation.\*\* We translate into an intermediate representation which consists of a directed graph of:
• Each source-level module, signature and include, and
• Each unfilled requirement (called a “signature merge” node).
The edges of the directed graph signify a “depends on” relation, and are defined as follows:
• An include p depends on include q if, for some module name m, p requires m and q provides m.
• An include p depends on a module m if p requires a module named m.
• A module/signature m depends on include p if m imports a module provided by p.
• A module/signature m depends on a module n if m imports n.
• A module/signature m depends on a signature merge n if m imports n.
• A module/signature m depends on a signature n if m {-\# SOURCE \#-} imports n.
• A signature merge m depends on a local signature m (if it exists).
• A signature merge m depends on a include p, if the (renamed) include requires m.
- \*Elaboration.\*\* Take a Backpack file, construct this graph, and topsort it into a DAG of SCCs. SCCs with a single node are compileable as before. SCCs with multiple nodes will have to be managed with some mutual recursion mechanism; see refinements for more thoughts on this.
- \*Refinements:\*\*
1. \*\*Can a signature depend on a (home) module?\*\* Imports of this kind require a retypecheck loop. Consider this situation:
```
unit p where
signature H where
data T
module M where
import H
data S = S T
unit q where
include p
module Q where
import M
signature H where
import Q
data T = T S
```
> Here, signature H in q depends on Q. When we typecheck `Q`, we bring `M.S` into the type environment with a `TyThing` that describes the constructor as accepting an abstract type `T`. However, when we subsequently typecheck the local signature `H`, we must refine all `TyThing`s of `T` with the true description (e.g. constructor information). So you'll need to retypecheck `Q` (and `M`) in order to make sure the `TyThing` is correct.
1. \*\*Can an include depend on a (home) module?\*\* If the module has no (transitive) dependency on signatures, this is fine. However, it's easy to have a circular dependency. Consider:
```
unit p where
signature A -- imports nothing
signature B -- imports nothing
module M
unit q where
include p
module B where
import A
...
```
> `B` depends on `p` for `p/A.hsig`; however, `p` depends on `B` because this module is filling a requirement. However, if we were to include the internal graph of `p` into `q`, the resulting graph would not have an cycles; so this is one possibility of how to untangle this situation. However, if there's still a cycle (e.g. `A` imports `B`), then you will need at least a retypecheck loop, and maybe `hs-boot` style compilation. We're not going to implement this for now.
1. \*\*Can we deal with include-include dependency cycles?\*\* Yes! Just use the Backpack paper's strategy for creating a recursive unit key and compile the two packages `hs-boot` style. But I'm not planning on implementing this yet.
1. \*\*Can we deal with signature-signature dependency cycles?\*\* Ordered Backpack would have supported this:
```
unit a-sig where
signature A where
data T
unit ab-sig where
include a-sig
signature B where
import A
data S = S T
signature A where
import B
data T = T S
```
> In our model, `ab-sig` has a cycle. However, I believe any such cycle can be broken by creating sufficiently many units:
```
unit a-sig where
signature B where
data T
signature A where
data S = S T
unit b-sig where
signature A where
data S
signature B where
data T = T S
unit ab-sig where
include a-sig
include b-sig
```
> In principle, GHC could automatically break import cycles by replacing an import with an import of a reduced signature that simply has abstract type definitions. See #10681. (I'm not sure this is possible for all language features.) This technique would also work for normal modules, assuming that every function is explicitly annotated with a type.Edward Z. YangEdward Z. Yanghttps://gitlab.haskell.org/ghc/ghc/-/issues/10679Generalize hi-boot/hi for signatures, to manage intermediate merged interfaces2019-07-07T18:34:41ZEdward Z. YangGeneralize hi-boot/hi for signatures, to manage intermediate merged interfaces## The problem
In some situations, we need to output multiple interface
files for what is morally the same module name.
### Example 1: Merging external and home signatures
```
unit a-sig where
signature A
unit p where
include ...## The problem
In some situations, we need to output multiple interface
files for what is morally the same module name.
### Example 1: Merging external and home signatures
```
unit a-sig where
signature A
unit p where
include a-sig
signature A
```
Compiling `p/A.hsig` produces an interface file which contains just
the definitions declared in `p`. However, someone including `p`
should see the merge of the interface of `p/A.hsig` AND `a-sig/A.hsig`
(which was included.)
### Example 2: Merging two home signatures
```
unit p where
signature A
signature B where
import A
...
signature A where
import B
...
```
What should we do if a signature is specified multiple times in the same
unit? The compilation of each produces a distinct interface, and the
public interface we want to expose is the merge of the two. (And by the
way, what's the source file name of `A`, if we are not using the inline syntax?)
### Example 3: Merging a signature and a module
```
unit p where
signature A
module B where
import A
...
module A where
import B
...
```
`A` and `B` are mutually recursive, and we want to use a signature file to
break the gap. The signature produces an interface file, only to be
overwritten when we actually define the module proper.
But wait! We have a solution for this example already: the first interface
file for `A` is not saved to `A.hi`, but `A.hi-boot`...
## The proposal
I want to take the `A.hi-boot` versus `A.hi` distinction and
generalize it: we should be able to name intermediate interface
files A.1.hi, A.2.hi, ... and finally A.hi (which
is publically visible outside the unit.) This naming convention applies
to Haskell files too.
### User-visible consequences
Every signature file is numbered, and every import of a signature file
refers to a specific number. This number is unique among all other
modules in a unit which share the same name. For backwards
compatibility, some number/file name extensions are treated specially:
1. `.hs` files compile to `.hi` (implicitly numbered 0)
1. `.hs-boot` files compile to `.hi-boot` (implicitly numbered 1)
1. `.hsig` files compile to `.hi-boot` (implicitly numbered 1)
1. `.n.hsig` files compile to `.n.hi-boot` (numbered n, where n is greater than 1)
- \*Flex point:\*\* We could give `.hsig` files their own file extension
for interface files; just would require some more work to distinguish
between `hs-boot` and `hsig` as well as record the numbering.
To import, the `{-# SOURCE n #-}` pragma can be used (with `{-# SOURCE #-}`
being equivalent `{-# SOURCE 1 #-}`.)
Inline Backpack files can omit numbering, since we can figure it out
based on the ordering of declarations (numbering in REVERSE order
of occurrence). Example 2 can be numbered as follows:
```
signature {-# SOURCE 2 #-} A
signature {-# SOURCE 1 #-} B where
import {-# SOURCE 2 #-} A
...
signature {-# SOURCE 1 #-} A where
import {-# SOURCE 1 #-} B
...
```
### Internal consequences
In many places in the code today, we record a boolean indicating if
we depended on the boot interface `hi-boot` or the normal interface `hi`.
We now replace this marker with an integer which records the numbering.
The primary affected components are dependency recording in interfaces,
interface loading code in GHC, and the implementation of `--make`.
### Interaction with signature merging
Unlike `hs-boot` files, `hsig` files can be included from external
units, in which case the semantics are that all signatures in scope
are merged together. The key rule is that we \*\*generate an hi
file for each partial merge\*\*; this means that whenever we want
to typecheck a module, there is exactly one interface file per
module we import. Consider this example:
```
unit a-sig where
signature A
unit a-sig2 where
signature A
unit p where
include a-sig
module B
include a-sig2
module C
signature A
module D
```
When compiling this, we generate four interface files for `A`:
```
unit p where
include a-sig
-- Produces A.3.hi-boot (a-sig)
module B -- uses A.3.hi-boot
include a-sig2
-- Produces A.2.hi-boot (a-sig + a-sig2)
module C -- uses A.2.hi-boot
signature A
-- Produces A.hi-boot (everything)
module D -- uses A.hi-boot
-- At the end, A.hi-boot copied to A.hi to be publically visible
```
## Can we do anything simpler?
There are a few barriers to doing something simpler:
1. We can avoid generating extra interface files if we instead merge them on-the-fly when we use them. However, this forces later instances of GHC to do repeated work remerging interface files, so it seems desirable from a performance perspective to merge before writing. Another scheme is that we could merge on use for signatures in the home package, and then write out a unified file at the very end, trading off performance for less written interface files.
1. The Backpack language is defined in a way that allows modules, signatures and includes to be ordered in a semantically meaningful way. For example:
```
unit q where
signature M
signature A where
f :: Int -> Int
...
unit p where
signature A where
data T
module M where
import A -- should get T but not f
...
include q -- fill in M
module S where
import A -- should get T and f
```
> This means that even within a unit, the interface of a signature file may differ. We could rule this out, but we would have to work out how to explain this limitation to users. (For example, we could solve the example above by saying that units which define modules do not bring their signatures into scope for a package which imports them; but this is a pretty ad hoc rule! And you still have to deal with repeated signatures, or a signature importing a module importing a signature. There are a lot of cases.)
1. This problem cannot be avoided at all if you are truly doing recursive modules, since you need the intermediate interface file to do compilation at all prior to getting the real implementation.
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | -------------- |
| Version | 7.11 |
| Type | Task |
| TypeOfFailure | OtherFailure |
| Priority | normal |
| Resolution | Unresolved |
| Component | Package system |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"Generalize hi-boot/hi for signatures, to manage intermediate merged interfaces","status":"New","operating_system":"","component":"Package system","related":[],"milestone":"","resolution":"Unresolved","owner":{"tag":"OwnedBy","contents":"ezyang"},"version":"7.11","keywords":[],"differentials":[],"test_case":"","architecture":"","cc":[""],"type":"Task","description":"== The problem ==\r\n\r\nIn some situations, we need to output multiple interface\r\nfiles for what is morally the same module name.\r\n\r\n=== Example 1: Merging external and home signatures ===\r\n\r\n{{{\r\nunit a-sig where\r\n signature A\r\nunit p where\r\n include a-sig\r\n signature A\r\n}}}\r\n\r\nCompiling `p/A.hsig` produces an interface file which contains just\r\nthe definitions declared in `p`. However, someone including `p`\r\nshould see the merge of the interface of `p/A.hsig` AND `a-sig/A.hsig`\r\n(which was included.)\r\n\r\n=== Example 2: Merging two home signatures ===\r\n\r\n{{{\r\nunit p where\r\n signature A\r\n signature B where\r\n import A\r\n ...\r\n signature A where\r\n import B\r\n ...\r\n}}}\r\n\r\nWhat should we do if a signature is specified multiple times in the same\r\nunit? The compilation of each produces a distinct interface, and the\r\npublic interface we want to expose is the merge of the two. (And by the\r\nway, what's the source file name of `A`, if we are not using the inline syntax?)\r\n\r\n=== Example 3: Merging a signature and a module ===\r\n\r\n{{{\r\nunit p where\r\n signature A\r\n module B where\r\n import A\r\n ...\r\n module A where\r\n import B\r\n ...\r\n}}}\r\n\r\n`A` and `B` are mutually recursive, and we want to use a signature file to\r\nbreak the gap. The signature produces an interface file, only to be\r\noverwritten when we actually define the module proper.\r\n\r\nBut wait! We have a solution for this example already: the first interface\r\nfile for `A` is not saved to `A.hi`, but `A.hi-boot`...\r\n\r\n== The proposal ==\r\n\r\nI want to take the `A.hi-boot` versus `A.hi` distinction and\r\ngeneralize it: we should be able to name intermediate interface\r\nfiles A.1.hi, A.2.hi, ... and finally A.hi (which\r\nis publically visible outside the unit.) This naming convention applies\r\nto Haskell files too.\r\n\r\n=== User-visible consequences ===\r\n\r\nEvery signature file is numbered, and every import of a signature file\r\nrefers to a specific number. This number is unique among all other\r\nmodules in a unit which share the same name. For backwards\r\ncompatibility, some number/file name extensions are treated specially:\r\n\r\n1. `.hs` files compile to `.hi` (implicitly numbered 0)\r\n\r\n2. `.hs-boot` files compile to `.hi-boot` (implicitly numbered 1)\r\n\r\n3. `.hsig` files compile to `.hi-boot` (implicitly numbered 1)\r\n\r\n4. `.n.hsig` files compile to `.n.hi-boot` (numbered n, where n is greater than 1)\r\n\r\n**Flex point:** We could give `.hsig` files their own file extension\r\nfor interface files; just would require some more work to distinguish\r\nbetween `hs-boot` and `hsig` as well as record the numbering.\r\n\r\nTo import, the `{-# SOURCE n #-}` pragma can be used (with `{-# SOURCE #-}`\r\nbeing equivalent `{-# SOURCE 1 #-}`.)\r\n\r\nInline Backpack files can omit numbering, since we can figure it out\r\nbased on the ordering of declarations (numbering in REVERSE order\r\nof occurrence). Example 2 can be numbered as follows:\r\n\r\n{{{\r\n signature {-# SOURCE 2 #-} A\r\n signature {-# SOURCE 1 #-} B where\r\n import {-# SOURCE 2 #-} A\r\n ...\r\n signature {-# SOURCE 1 #-} A where\r\n import {-# SOURCE 1 #-} B\r\n ...\r\n}}}\r\n\r\n=== Internal consequences ===\r\n\r\nIn many places in the code today, we record a boolean indicating if\r\nwe depended on the boot interface `hi-boot` or the normal interface `hi`.\r\nWe now replace this marker with an integer which records the numbering.\r\nThe primary affected components are dependency recording in interfaces,\r\ninterface loading code in GHC, and the implementation of `--make`.\r\n\r\n=== Interaction with signature merging ===\r\n\r\nUnlike `hs-boot` files, `hsig` files can be included from external\r\nunits, in which case the semantics are that all signatures in scope\r\nare merged together. The key rule is that we **generate an hi\r\nfile for each partial merge**; this means that whenever we want\r\nto typecheck a module, there is exactly one interface file per\r\nmodule we import. Consider this example:\r\n\r\n{{{\r\nunit a-sig where\r\n signature A\r\nunit a-sig2 where\r\n signature A\r\nunit p where\r\n include a-sig\r\n module B\r\n include a-sig2\r\n module C\r\n signature A\r\n module D\r\n}}}\r\n\r\nWhen compiling this, we generate four interface files for `A`:\r\n\r\n{{{\r\nunit p where\r\n include a-sig\r\n -- Produces A.3.hi-boot (a-sig)\r\n module B -- uses A.3.hi-boot\r\n include a-sig2\r\n -- Produces A.2.hi-boot (a-sig + a-sig2)\r\n module C -- uses A.2.hi-boot\r\n signature A\r\n -- Produces A.hi-boot (everything)\r\n module D -- uses A.hi-boot\r\n -- At the end, A.hi-boot copied to A.hi to be publically visible\r\n}}}\r\n\r\n== Can we do anything simpler? ==\r\n\r\nThere are a few barriers to doing something simpler:\r\n\r\n1. We can avoid generating extra interface files if we instead merge them on-the-fly when we use them. However, this forces later instances of GHC to do repeated work remerging interface files, so it seems desirable from a performance perspective to merge before writing. Another scheme is that we could merge on use for signatures in the home package, and then write out a unified file at the very end, trading off performance for less written interface files.\r\n\r\n2. The Backpack language is defined in a way that allows modules, signatures and includes to be ordered in a semantically meaningful way. For example:\r\n{{{\r\nunit q where\r\n signature M\r\n signature A where\r\n f :: Int -> Int\r\n ...\r\nunit p where\r\n signature A where\r\n data T\r\n module M where\r\n import A -- should get T but not f\r\n ...\r\n include q -- fill in M\r\n module S where\r\n import A -- should get T and f\r\n}}}\r\n This means that even within a unit, the interface of a signature file may differ. We could rule this out, but we would have to work out how to explain this limitation to users. (For example, we could solve the example above by saying that units which define modules do not bring their signatures into scope for a package which imports them; but this is a pretty ad hoc rule! And you still have to deal with repeated signatures, or a signature importing a module importing a signature. There are a lot of cases.)\r\n\r\n3. This problem cannot be avoided at all if you are truly doing recursive modules, since you need the intermediate interface file to do compilation at all prior to getting the real implementation.\r\n","type_of_failure":"OtherFailure","blocking":[]} -->Edward Z. YangEdward Z. Yanghttps://gitlab.haskell.org/ghc/ghc/-/issues/10666Distinguish between semantic module / identity module in TcGblEnv, ModIface a...2019-07-07T18:34:44ZEdward Z. YangDistinguish between semantic module / identity module in TcGblEnv, ModIface and ModGutsWhen we write a signature like
```
package p where
signature H where
data T
```
and compile it to an interface file, there are two ways we might say what the `Module` of this interface is:
1. The \*\*identity module\*\* uniquely...When we write a signature like
```
package p where
signature H where
data T
```
and compile it to an interface file, there are two ways we might say what the `Module` of this interface is:
1. The \*\*identity module\*\* uniquely identifies an interface file, and is used for dependency analysis and tracking. In the example above, the identity module is `p(H -> HOLE:H):H`.
1. The \*\*semantic module\*\* tells us what the `Name`s of the entities defined in the module are supposed to be; e.g., it's used for generating new names when type-checking hs files or interfaces. In the example above, the semantic module is `hole:H`, since this signature exports one entity named `hole:H.T`. The semantic module can always be derived from the identity module.
For normal Haskell modules, the semantic and identity module coincide. However, for signatures they differ: we may have many signatures for the same module; they all share their semantic module but have differing identity modules.
By in large, when GHC manipulates `Module` directly it is interested in the identity module. However, when a `Module` is used with reference to a `Name` (primarily `nameIsLocalOrFrom`), we want to use the SEMANTIC module. (Another example: when we filter out the type environment before making a `ModIface`, need to filter against the semantic module.)
I tried a few ways of factoring GHC's code so we'd be less likely to confuse these two `Module`s when typechecking signatures: the big problem is if you're adding a `getModule` call to `TcRn`, you're probably not going to think too hard whether or not you actually wanted the semantic module or the identity module. But if you pick the wrong thing that will break all sorts of things for signatures.
Here are some things we could do:
1. My initial attempt was to change `tcg_mod`, `mg_module` and `mi_module` to record a new data type `TopModule` which recorded both the semantic and identity module, with `getModule` in `TcRn` continuing to return a semantic module, but `mi_module` returning an identity module. However, the resulting patch was pretty ugly and it's not altogether clear that `getModule` returning the semantic module is always correct.
1. My other idea is to say that these entries always are IDENTITY modules (this will result on fail fast behavior for signatures if you get it wrong), and then rewrite `nameIsLocalOrFrom`, `externaliseAndTidyId`, `initIfaceTcRn`, `newGlobalBinder` so that they always do the right thing (i.e. use the semantic module); thus, the only time you can get it wrong is if you're creating some new functionality that's not these functions that needs to use semantic modules.
Pretty delicate.
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | -------------- |
| Version | 7.10.1 |
| Type | Task |
| TypeOfFailure | OtherFailure |
| Priority | normal |
| Resolution | Unresolved |
| Component | Package system |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"Distinguish between semantic module / identity module in TcGblEnv, ModIface and ModGuts","status":"New","operating_system":"","component":"Package system","related":[],"milestone":"","resolution":"Unresolved","owner":{"tag":"OwnedBy","contents":"ezyang"},"version":"7.10.1","keywords":[],"differentials":[],"test_case":"","architecture":"","cc":[""],"type":"Task","description":"When we write a signature like\r\n\r\n{{{\r\npackage p where\r\n signature H where\r\n data T\r\n}}}\r\n\r\nand compile it to an interface file, there are two ways we might say what the `Module` of this interface is:\r\n\r\n1. The **identity module** uniquely identifies an interface file, and is used for dependency analysis and tracking. In the example above, the identity module is `p(H -> HOLE:H):H`.\r\n\r\n2. The **semantic module** tells us what the `Name`s of the entities defined in the module are supposed to be; e.g., it's used for generating new names when type-checking hs files or interfaces. In the example above, the semantic module is `hole:H`, since this signature exports one entity named `hole:H.T`. The semantic module can always be derived from the identity module.\r\n\r\nFor normal Haskell modules, the semantic and identity module coincide. However, for signatures they differ: we may have many signatures for the same module; they all share their semantic module but have differing identity modules.\r\n\r\nBy in large, when GHC manipulates `Module` directly it is interested in the identity module. However, when a `Module` is used with reference to a `Name` (primarily `nameIsLocalOrFrom`), we want to use the SEMANTIC module. (Another example: when we filter out the type environment before making a `ModIface`, need to filter against the semantic module.)\r\n\r\nI tried a few ways of factoring GHC's code so we'd be less likely to confuse these two `Module`s when typechecking signatures: the big problem is if you're adding a `getModule` call to `TcRn`, you're probably not going to think too hard whether or not you actually wanted the semantic module or the identity module. But if you pick the wrong thing that will break all sorts of things for signatures.\r\n\r\nHere are some things we could do:\r\n\r\n1. My initial attempt was to change `tcg_mod`, `mg_module` and `mi_module` to record a new data type `TopModule` which recorded both the semantic and identity module, with `getModule` in `TcRn` continuing to return a semantic module, but `mi_module` returning an identity module. However, the resulting patch was pretty ugly and it's not altogether clear that `getModule` returning the semantic module is always correct.\r\n\r\n2. My other idea is to say that these entries always are IDENTITY modules (this will result on fail fast behavior for signatures if you get it wrong), and then rewrite `nameIsLocalOrFrom`, `externaliseAndTidyId`, `initIfaceTcRn`, `newGlobalBinder` so that they always do the right thing (i.e. use the semantic module); thus, the only time you can get it wrong is if you're creating some new functionality that's not these functions that needs to use semantic modules.\r\n\r\nPretty delicate.","type_of_failure":"OtherFailure","blocking":[]} -->Edward Z. YangEdward Z. Yanghttps://gitlab.haskell.org/ghc/ghc/-/issues/10566Move "package key" generation to GHC2019-07-07T18:35:13ZEdward Z. YangMove "package key" generation to GHCCurrently in GHC 7.10, we have the following situation:
1. Cabal computes a package key, which in practice (since no one is using Backpack in the wild) is a Merkle tree of the versions of each of the dependencies of the package.
1. This...Currently in GHC 7.10, we have the following situation:
1. Cabal computes a package key, which in practice (since no one is using Backpack in the wild) is a Merkle tree of the versions of each of the dependencies of the package.
1. This package key is passed to GHC via `-this-package-key`
1. GHC handles the package key opaquely
Now, in recent Backpack implementation, we need GHC to be able to compute package keys. (The concrete case: you're type-checking an interface file of an indefinite package, where you want to instantiate it with some assignment of its holes: instantiating those holes you need to instantiate any package keys mentioned in the interface, in which case you really want to be able to compute the hash.) So I want to move package key generation to GHC.
The primary implication is this: does Cabal continue to generate package keys? If it doesn't, we should revert from `-this-package-key` back to `-package-name` from the previous version (but maybe renamed because this name was bad). GHC then computes a package key based on `-package-name` and the explicitly mentioned `-package` dependencies, and Cabal reads it out with something akin to `--abi-hash`. If it does, we need to ensure GHC and Cabal's package key algorithms stay synchronized. I personally lean towards the first option.
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | -------------- |
| Version | 7.10.1 |
| Type | Bug |
| TypeOfFailure | OtherFailure |
| Priority | normal |
| Resolution | Unresolved |
| Component | Package system |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"Move \"package key\" generation to GHC","status":"New","operating_system":"","component":"Package system","related":[],"milestone":"7.12.1","resolution":"Unresolved","owner":{"tag":"OwnedBy","contents":"ezyang"},"version":"7.10.1","keywords":[],"differentials":[],"test_case":"","architecture":"","cc":[""],"type":"Bug","description":"Currently in GHC 7.10, we have the following situation:\r\n\r\n1. Cabal computes a package key, which in practice (since no one is using Backpack in the wild) is a Merkle tree of the versions of each of the dependencies of the package.\r\n2. This package key is passed to GHC via `-this-package-key`\r\n3. GHC handles the package key opaquely\r\n\r\nNow, in recent Backpack implementation, we need GHC to be able to compute package keys. (The concrete case: you're type-checking an interface file of an indefinite package, where you want to instantiate it with some assignment of its holes: instantiating those holes you need to instantiate any package keys mentioned in the interface, in which case you really want to be able to compute the hash.) So I want to move package key generation to GHC.\r\n\r\nThe primary implication is this: does Cabal continue to generate package keys? If it doesn't, we should revert from `-this-package-key` back to `-package-name` from the previous version (but maybe renamed because this name was bad). GHC then computes a package key based on `-package-name` and the explicitly mentioned `-package` dependencies, and Cabal reads it out with something akin to `--abi-hash`. If it does, we need to ensure GHC and Cabal's package key algorithms stay synchronized. I personally lean towards the first option.","type_of_failure":"OtherFailure","blocking":[]} -->8.0.1Edward Z. YangEdward Z. Yanghttps://gitlab.haskell.org/ghc/ghc/-/issues/10550Drop truncated package name prefix from package keys2019-07-07T18:35:22ZEdward Z. YangDrop truncated package name prefix from package keysEarly on when we moved to package keys, we decided it would be useful for documentation purposes to have a package key be something like `ghcpr_8TmvWUcS1U1IKHT0levwg3` rather than just `8TmvWUcS1U1IKHT0levwg3`, which would leave a user w...Early on when we moved to package keys, we decided it would be useful for documentation purposes to have a package key be something like `ghcpr_8TmvWUcS1U1IKHT0levwg3` rather than just `8TmvWUcS1U1IKHT0levwg3`, which would leave a user with no idea what the key is.
I now think that this is more trouble than its worth:
1. We have fixed library \*paths\* so that the full package name, version, key is in the path, e.g. libraries are installed to `/home/ezyang/Dev/ghc-7.10.2/usr/lib/ghc-7.10.2/ghc-prim-0.4.0.0-8TmvWUcS1U1IKHT0levwg3` instead of `/home/ezyang/Dev/ghc-7.10.1/usr/lib/ghc-7.10.1/ghcpr_8TmvWUcS1U1IKHT0levwg3`. This means that the biggest case where users might care about package keys is no longer relevant; a user will always be able to get the real package key.
1. The other case where package keys show up is in symbol names. However, symbol names already contain \*modules names\*, so today we have `procezu0hwN3CTKynhHQqQkChnSdH_SystemziProcess_rawSystem1_info` but `0hwN3CTKynhHQqQkChnSdH_SystemziProcess_rawSystem1_info` seems just about as clear (in particular, five letters just doesn't seem like enough to adequately describe long package names.)
1. Now, the reason why we don't want to truncate the prefix name is because it makes it more difficult to reliably, deterministically generate a package key based on a few parameters about a package: namely, any algorithm needs to know that `take 5 (filter (/= '-') packageName)` and prefix that onto the key. Really, we only care about the hash, but for most equality tests the prefix matters too! So it would be much nicer if we didn't have to care about the hash as well.
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | -------------- |
| Version | 7.11 |
| Type | Bug |
| TypeOfFailure | OtherFailure |
| Priority | normal |
| Resolution | Unresolved |
| Component | Package system |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"Drop truncated package name prefix from package keys","status":"New","operating_system":"","component":"Package system","related":[],"milestone":"7.12.1","resolution":"Unresolved","owner":{"tag":"OwnedBy","contents":"ezyang"},"version":"7.11","keywords":[],"differentials":[],"test_case":"","architecture":"","cc":[""],"type":"Bug","description":"Early on when we moved to package keys, we decided it would be useful for documentation purposes to have a package key be something like `ghcpr_8TmvWUcS1U1IKHT0levwg3` rather than just `8TmvWUcS1U1IKHT0levwg3`, which would leave a user with no idea what the key is.\r\n\r\nI now think that this is more trouble than its worth:\r\n\r\n1. We have fixed library *paths* so that the full package name, version, key is in the path, e.g. libraries are installed to `/home/ezyang/Dev/ghc-7.10.2/usr/lib/ghc-7.10.2/ghc-prim-0.4.0.0-8TmvWUcS1U1IKHT0levwg3` instead of `/home/ezyang/Dev/ghc-7.10.1/usr/lib/ghc-7.10.1/ghcpr_8TmvWUcS1U1IKHT0levwg3`. This means that the biggest case where users might care about package keys is no longer relevant; a user will always be able to get the real package key.\r\n\r\n2. The other case where package keys show up is in symbol names. However, symbol names already contain *modules names*, so today we have `procezu0hwN3CTKynhHQqQkChnSdH_SystemziProcess_rawSystem1_info` but `0hwN3CTKynhHQqQkChnSdH_SystemziProcess_rawSystem1_info` seems just about as clear (in particular, five letters just doesn't seem like enough to adequately describe long package names.)\r\n\r\n3. Now, the reason why we don't want to truncate the prefix name is because it makes it more difficult to reliably, deterministically generate a package key based on a few parameters about a package: namely, any algorithm needs to know that `take 5 (filter (/= '-') packageName)` and prefix that onto the key. Really, we only care about the hash, but for most equality tests the prefix matters too! So it would be much nicer if we didn't have to care about the hash as well.","type_of_failure":"OtherFailure","blocking":[]} -->8.0.1Edward Z. YangEdward Z. Yanghttps://gitlab.haskell.org/ghc/ghc/-/issues/10479Make GHC install libraries to, e.g. xhtml-3000.2.1-0ACfOp3hebWD9jGWE4v4Gh2019-07-07T18:35:45ZEdward Z. YangMake GHC install libraries to, e.g. xhtml-3000.2.1-0ACfOp3hebWD9jGWE4v4GhThe GHC version of https://github.com/haskell/cabal/issues/2437 ; we need to adapt ghc-cabal to use the correct install paths. I have an in-flight patch, see the Phabricator. However, applying the change to HEAD is blocked by https://git...The GHC version of https://github.com/haskell/cabal/issues/2437 ; we need to adapt ghc-cabal to use the correct install paths. I have an in-flight patch, see the Phabricator. However, applying the change to HEAD is blocked by https://github.com/haskell/cabal/issues/2638 since there are some Cabal changes, which necessitate updating to Cabal head.
Cabal maintainers have expressed interest in this being merged into 7.10. To do this, we first have to merge the Cabal change into the 7.10 branch of Cabal, and apply the differential to GHC 7.10. (We shouldn't be blocked by the Cabal HEAD changes as those are not on the 7.10 branch.) If 7.10 release managers agree to let this patch in I volunteer to coordinate the changes.
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | -------------- |
| Version | 7.10.1 |
| Type | Bug |
| TypeOfFailure | OtherFailure |
| Priority | normal |
| Resolution | Unresolved |
| Component | Package system |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"Make GHC install libraries to, e.g. xhtml-3000.2.1-0ACfOp3hebWD9jGWE4v4Gh","status":"New","operating_system":"","component":"Package system","related":[],"milestone":"7.10.2","resolution":"Unresolved","owner":{"tag":"OwnedBy","contents":"ezyang"},"version":"7.10.1","keywords":[],"differentials":[],"test_case":"","architecture":"","cc":[""],"type":"Bug","description":"The GHC version of https://github.com/haskell/cabal/issues/2437 ; we need to adapt ghc-cabal to use the correct install paths. I have an in-flight patch, see the Phabricator. However, applying the change to HEAD is blocked by https://github.com/haskell/cabal/issues/2638 since there are some Cabal changes, which necessitate updating to Cabal head.\r\n\r\nCabal maintainers have expressed interest in this being merged into 7.10. To do this, we first have to merge the Cabal change into the 7.10 branch of Cabal, and apply the differential to GHC 7.10. (We shouldn't be blocked by the Cabal HEAD changes as those are not on the 7.10 branch.) If 7.10 release managers agree to let this patch in I volunteer to coordinate the changes.","type_of_failure":"OtherFailure","blocking":[]} -->7.10.2Edward Z. YangEdward Z. Yanghttps://gitlab.haskell.org/ghc/ghc/-/issues/10352Properly link Haskell shared libs on all systems2023-07-13T13:05:15ZduncanProperly link Haskell shared libs on all systemsSince the early days of building Haskell shared libs on Linux we have been using a scheme that is really a bit of a hack. We should do it properly.
In my blog post on this from 2009 (http://www.well-typed.com/blog/30/) I said:
> If we ...Since the early days of building Haskell shared libs on Linux we have been using a scheme that is really a bit of a hack. We should do it properly.
In my blog post on this from 2009 (http://www.well-typed.com/blog/30/) I said:
> If we use ldd again to look at the libfoo.so that we've made we will notice that it is missing a dependency on the rts library. This is problem that we've yet to sort out, so for the moment we can just add the dependency ourselves:
>
> $ ghc --make -dynamic -shared -fPIC Foo.hs -o libfoo.so \\
> -lHSrts-ghc6.11 -optl-Wl,-rpath,/opt/ghc/lib/ghc-6.11/
>
> The reason it's not linked in yet is because we need to be able to switch which version of the rts we're using without having to relink every library. For example we want to be able to switch between the debug, threaded and normal rts versions. It's quite possible to do this and it just needs a bit more rearranging in the build system to sort it out. Once it's done you'll even be able to switch rts at runtime, eg:
>
> $ LD_PRELOAD=/opt/ghc/lib/ghc-6.11/libHSrts_debug-ghc6.11.so
> $ ./Hello
So in general, if a shared lib requires symbols from another shared lib then it should depend on it. In ELF terminology that means a NEEDED entry to say this lib needs that other lib. This is important to be able to link and load these shared libraries, otherwise they can have dangling dependencies.
But we don't do this. For the specific case of the RTS we do not link Haskell shared libs against the RTS. So they have lots of dangling symbols. These libraries cannot be loaded on their own, e.g. with `dlopen()`. This is bad, and has other knock-on consequences.
Why don't we link to the RTS? It's because historically (with static linking) GHC had had the ability to select the flavour of the RTS when final executables are linked, not when intermediate libraries are created. This works because the RTS flavours share a common ABI. This is a useful feature as it lets us select the SMP or debug or other RTS at final link time. So when we made up the first shared lib scheme on ELF we had to support this.
Our initial scheme was like this: don't link Haskell library DSOs against the RTS, only like the final exe against the RTS. Each RTS flavour has a separate SONAME, e.g. `libHSrts_thr-ghc7.8.4.so` or `libHSrts_debug-ghc7.8.4.so`. This works because the runtime linker looks at the final exe first and loads the RTS, and then when other libs are loaded the symbols all resolve.
Why can't we link all the libraries against the RTS? Currently each RTS flavour has a different SONAME, which is the key that the dynamic linker uses to identify each library. So if we did link all the Haskell libs against "the" RTS we would have to pick which one at the point at which we create the library, and that'd stop us from being able to choose later.
So, can we use a better scheme? We want one that doesn't leave dangling undefined references in intermediate Haskell libs, and is also compatible with the ability to select the flavour of the RTS at final exe link time (or even override it at load time).
Yes we can!
The first thing to note is that to be interchangeable, all the RTS flavours (that share a compatible ABI) need to have the same SONAME. So for example, all the (non-profiling) RTS DSO files have to have the internal SONAME of `libHSrts-ghc7.8.4.so`. Once they all have the same SONAME, then it's ok for all the Haskell libs to specify a NEEDED dependency on that rts SONAME.
But if they have the same SONAME, what do the files get called, where do they live and how are they found? The trick is to make use of the search path. Put each RTS flavour in a different directory, but otherwise with the same filename, e.g. `lib/rts-1.0/thr/libHSrts-ghc7.8.4.so`, `lib/rts-1.0/debug/libHSrts-ghc7.8.4.so` etc.
Each library DSO and exe has its list of NEEDED entries, and it has an RPATH entry used to find those libraries if they're not loaded yet. The key is the "if they're not loaded yet" bit. Remember that the linker uses the SONAME as the key to decide if the lib is loaded yet or not. So the libraries could all have an RPATH entry to say to look for the RTS in the directory containing the default RTS flavour. But then the top level exe (or foreign/export shared lib) can also link to the RTS directly (ie an NEEDED entry) and can specify an RPATH which can be for any of the rts flavours. When the linker loads the top level exe, it will loads the selected RTS using the exe's RPATH, and then when the linker sees other Haskell libs that have a NEEDED entry on the RTS it will ignore them because the RTS's SONAME is already loaded.
So concretely, instead of:
lib/ghc-${ver}/rts-1.0/libHSrts-ghc${ver}.so
lib/ghc-${ver}/rts-1.0/libHSrts_thr-ghc${ver}.so
lib/ghc-${ver}/rts-1.0/libHSrts_debug-ghc${ver}.so
lib/ghc-${ver}/rts-1.0/libHSrts_l-ghc${ver}.so
lib/ghc-${ver}/rts-1.0/libHSrts_thr_l-ghc${ver}.so
> lib/ghc-${ver}/rts-1.0/libHSrts_thr_debug-ghc${ver}.so
each with a different SONAME
we'd have
lib/ghc-${ver}/rts-1.0/libHSrts-ghc${ver}.so
lib/ghc-${ver}/rts-1.0/thr/libHSrts-ghc${ver}.so
lib/ghc-${ver}/rts-1.0/debug/libHSrts-ghc${ver}.so
lib/ghc-${ver}/rts-1.0/l/libHSrts-ghc${ver}.so
lib/ghc-${ver}/rts-1.0/thr_l/libHSrts-ghc${ver}.so
> lib/ghc-${ver}/rts-1.0/thr_debug/libHSrts-ghc${ver}.so
each with the same SONAME
When linking libs we would always use `-lHSrts -rpath lib/ghc-${ver}/rts-1.0`
When linking exes (or shared libs for external consumption) we would use both `-lHSrts` and `-rpath lib/ghc-${ver}/${rtsflavour}/rts-1.0`.
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | -------------- |
| Version | 7.11 |
| Type | Task |
| TypeOfFailure | OtherFailure |
| Priority | normal |
| Resolution | Unresolved |
| Component | Package system |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | |
| Operating system | Linux |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"Properly link Haskell shared libs on ELF systems","status":"New","operating_system":"Linux","component":"Package system","related":[],"milestone":"","resolution":"Unresolved","owner":{"tag":"Unowned"},"version":"7.11","keywords":[],"differentials":[],"test_case":"","architecture":"","cc":[""],"type":"Task","description":"Since the early days of building Haskell shared libs on Linux we have been using a scheme that is really a bit of a hack. We should do it properly.\r\n\r\nIn my blog post on this from 2009 (http://www.well-typed.com/blog/30/) I said:\r\n\r\n> If we use ldd again to look at the libfoo.so that we've made we will notice that it is missing a dependency on the rts library. This is problem that we've yet to sort out, so for the moment we can just add the dependency ourselves:\r\n> \r\n> $ ghc --make -dynamic -shared -fPIC Foo.hs -o libfoo.so \\\r\n> -lHSrts-ghc6.11 -optl-Wl,-rpath,/opt/ghc/lib/ghc-6.11/\r\n> \r\n> The reason it's not linked in yet is because we need to be able to switch which version of the rts we're using without having to relink every library. For example we want to be able to switch between the debug, threaded and normal rts versions. It's quite possible to do this and it just needs a bit more rearranging in the build system to sort it out. Once it's done you'll even be able to switch rts at runtime, eg:\r\n> \r\n> $ LD_PRELOAD=/opt/ghc/lib/ghc-6.11/libHSrts_debug-ghc6.11.so\r\n> $ ./Hello\r\n\r\nSo in general, if a shared lib requires symbols from another shared lib then it should depend on it. In ELF terminology that means a NEEDED entry to say this lib needs that other lib. This is important to be able to link and load these shared libraries, otherwise they can have dangling dependencies.\r\n\r\nBut we don't do this. For the specific case of the RTS we do not link Haskell shared libs against the RTS. So they have lots of dangling symbols. These libraries cannot be loaded on their own, e.g. with `dlopen()`. This is bad, and has other knock-on consequences.\r\n\r\nWhy don't we link to the RTS? It's because historically (with static linking) GHC had had the ability to select the flavour of the RTS when final executables are linked, not when intermediate libraries are created. This works because the RTS flavours share a common ABI. This is a useful feature as it lets us select the SMP or debug or other RTS at final link time. So when we made up the first shared lib scheme on ELF we had to support this.\r\n\r\nOur initial scheme was like this: don't link Haskell library DSOs against the RTS, only like the final exe against the RTS. Each RTS flavour has a separate SONAME, e.g. `libHSrts_thr-ghc7.8.4.so` or `libHSrts_debug-ghc7.8.4.so`. This works because the runtime linker looks at the final exe first and loads the RTS, and then when other libs are loaded the symbols all resolve.\r\n\r\nWhy can't we link all the libraries against the RTS? Currently each RTS flavour has a different SONAME, which is the key that the dynamic linker uses to identify each library. So if we did link all the Haskell libs against \"the\" RTS we would have to pick which one at the point at which we create the library, and that'd stop us from being able to choose later.\r\n\r\nSo, can we use a better scheme? We want one that doesn't leave dangling undefined references in intermediate Haskell libs, and is also compatible with the ability to select the flavour of the RTS at final exe link time (or even override it at load time).\r\n\r\nYes we can!\r\n\r\nThe first thing to note is that to be interchangeable, all the RTS flavours (that share a compatible ABI) need to have the same SONAME. So for example, all the (non-profiling) RTS DSO files have to have the internal SONAME of `libHSrts-ghc7.8.4.so`. Once they all have the same SONAME, then it's ok for all the Haskell libs to specify a NEEDED dependency on that rts SONAME.\r\n\r\nBut if they have the same SONAME, what do the files get called, where do they live and how are they found? The trick is to make use of the search path. Put each RTS flavour in a different directory, but otherwise with the same filename, e.g. `lib/rts-1.0/thr/libHSrts-ghc7.8.4.so`, `lib/rts-1.0/debug/libHSrts-ghc7.8.4.so` etc.\r\n\r\nEach library DSO and exe has its list of NEEDED entries, and it has an RPATH entry used to find those libraries if they're not loaded yet. The key is the \"if they're not loaded yet\" bit. Remember that the linker uses the SONAME as the key to decide if the lib is loaded yet or not. So the libraries could all have an RPATH entry to say to look for the RTS in the directory containing the default RTS flavour. But then the top level exe (or foreign/export shared lib) can also link to the RTS directly (ie an NEEDED entry) and can specify an RPATH which can be for any of the rts flavours. When the linker loads the top level exe, it will loads the selected RTS using the exe's RPATH, and then when the linker sees other Haskell libs that have a NEEDED entry on the RTS it will ignore them because the RTS's SONAME is already loaded.\r\n\r\nSo concretely, instead of:\r\n\r\n lib/ghc-${ver}/rts-1.0/libHSrts-ghc${ver}.so\r\n lib/ghc-${ver}/rts-1.0/libHSrts_thr-ghc${ver}.so\r\n lib/ghc-${ver}/rts-1.0/libHSrts_debug-ghc${ver}.so\r\n lib/ghc-${ver}/rts-1.0/libHSrts_l-ghc${ver}.so\r\n lib/ghc-${ver}/rts-1.0/libHSrts_thr_l-ghc${ver}.so\r\n lib/ghc-${ver}/rts-1.0/libHSrts_thr_debug-ghc${ver}.so\r\n\r\neach with a different SONAME\r\n\r\nwe'd have\r\n\r\n lib/ghc-${ver}/rts-1.0/libHSrts-ghc${ver}.so\r\n lib/ghc-${ver}/rts-1.0/thr/libHSrts-ghc${ver}.so\r\n lib/ghc-${ver}/rts-1.0/debug/libHSrts-ghc${ver}.so\r\n lib/ghc-${ver}/rts-1.0/l/libHSrts-ghc${ver}.so\r\n lib/ghc-${ver}/rts-1.0/thr_l/libHSrts-ghc${ver}.so\r\n lib/ghc-${ver}/rts-1.0/thr_debug/libHSrts-ghc${ver}.so\r\n\r\neach with the same SONAME\r\n\r\nWhen linking libs we would always use `-lHSrts -rpath lib/ghc-${ver}/rts-1.0`\r\n\r\nWhen linking exes (or shared libs for external consumption) we would use both `-lHSrts` and `-rpath lib/ghc-${ver}/${rtsflavour}/rts-1.0`.","type_of_failure":"OtherFailure","blocking":[]} -->Tamar ChristinaTamar Christinahttps://gitlab.haskell.org/ghc/ghc/-/issues/10099cabal install broken with ghc 7.10.1-rc22019-07-07T18:37:30ZPeter Trommlerptrommler@acm.orgcabal install broken with ghc 7.10.1-rc2Trying to install a package with cabal-install and Cabal from ghc 7.10-rc2
consistently gives this error message (`primitive` is just an example):
```
peter@montebre:~> cabal install primitive
Resolving dependencies...
Configuring primi...Trying to install a package with cabal-install and Cabal from ghc 7.10-rc2
consistently gives this error message (`primitive` is just an example):
```
peter@montebre:~> cabal install primitive
Resolving dependencies...
Configuring primitive-0.5.4.0...
cabal: Distribution/Client/Config.hs:(246,37)-(299,9): Missing field in record construction configProf
```
This is my package database:
```
peter@montebre:~> ghc-pkg list
/usr/lib64/ghc-7.10.0.20150123/package.conf.d
Cabal-1.22.1.0
HTTP-4000.2.19
array-0.5.0.1
base-4.8.0.0
bin-package-db-0.0.0.0
binary-0.7.3.0
bytestring-0.10.6.0
containers-0.5.6.2
deepseq-1.4.0.0
directory-1.2.2.0
filepath-1.3.1.0
ghc-7.10.0.20150123
ghc-prim-0.3.1.0
haskeline-0.7.2.0
hoopl-3.10.0.2
hpc-0.6.0.2
integer-gmp-1.0.0.0
mtl-2.2.1
network-2.4.2.3
old-locale-1.0.0.7
old-time-1.1.0.3
parsec-3.1.8
pretty-1.1.2.0
process-1.2.2.0
random-1.0.1.1
rts-1.0
stm-2.4.2
syb-0.4.4
template-haskell-2.10.0.0
terminfo-0.4.0.1
text-1.2.0.4
th-desugar-1.5
th-lift-0.7
time-1.5.0.1
transformers-0.4.2.0
unix-2.7.1.0
xhtml-3000.2.1
zlib-0.5.4.2
```
I updated only those packages (from their versions in Haskell Platform) that would not compile with 7.101-rc2.
I am setting this to highest as this issue would be very annoying for everyone.
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | -------------- |
| Version | 7.10.1-rc2 |
| Type | Bug |
| TypeOfFailure | OtherFailure |
| Priority | highest |
| Resolution | Unresolved |
| Component | Package system |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"cabal install broken with ghc 7.10.1-rc2","status":"New","operating_system":"","component":"Package system","related":[],"milestone":"7.10.1","resolution":"Unresolved","owner":{"tag":"Unowned"},"version":"7.10.1-rc2","keywords":[],"differentials":[],"test_case":"","architecture":"","cc":[""],"type":"Bug","description":"Trying to install a package with cabal-install and Cabal from ghc 7.10-rc2\r\nconsistently gives this error message (`primitive` is just an example):\r\n{{{\r\npeter@montebre:~> cabal install primitive\r\nResolving dependencies...\r\nConfiguring primitive-0.5.4.0...\r\ncabal: Distribution/Client/Config.hs:(246,37)-(299,9): Missing field in record construction configProf\r\n}}}\r\n\r\nThis is my package database:\r\n{{{\r\npeter@montebre:~> ghc-pkg list\r\n/usr/lib64/ghc-7.10.0.20150123/package.conf.d\r\n Cabal-1.22.1.0\r\n HTTP-4000.2.19\r\n array-0.5.0.1\r\n base-4.8.0.0\r\n bin-package-db-0.0.0.0\r\n binary-0.7.3.0\r\n bytestring-0.10.6.0\r\n containers-0.5.6.2\r\n deepseq-1.4.0.0\r\n directory-1.2.2.0\r\n filepath-1.3.1.0\r\n ghc-7.10.0.20150123\r\n ghc-prim-0.3.1.0\r\n haskeline-0.7.2.0\r\n hoopl-3.10.0.2\r\n hpc-0.6.0.2\r\n integer-gmp-1.0.0.0\r\n mtl-2.2.1\r\n network-2.4.2.3\r\n old-locale-1.0.0.7\r\n old-time-1.1.0.3\r\n parsec-3.1.8\r\n pretty-1.1.2.0\r\n process-1.2.2.0\r\n random-1.0.1.1\r\n rts-1.0\r\n stm-2.4.2\r\n syb-0.4.4\r\n template-haskell-2.10.0.0\r\n terminfo-0.4.0.1\r\n text-1.2.0.4\r\n th-desugar-1.5\r\n th-lift-0.7\r\n time-1.5.0.1\r\n transformers-0.4.2.0\r\n unix-2.7.1.0\r\n xhtml-3000.2.1\r\n zlib-0.5.4.2\r\n}}}\r\nI updated only those packages (from their versions in Haskell Platform) that would not compile with 7.101-rc2.\r\n\r\nI am setting this to highest as this issue would be very annoying for everyone.","type_of_failure":"OtherFailure","blocking":[]} -->7.10.1