GHC issueshttps://gitlab.haskell.org/ghc/ghc/-/issues2019-07-07T18:29:49Zhttps://gitlab.haskell.org/ghc/ghc/-/issues/11558Using Cabal 1.22 against GHC 8.0 results in unhelpful errors2019-07-07T18:29:49ZBen GamariUsing Cabal 1.22 against GHC 8.0 results in unhelpful errorsGHC 8.0 will require Cabal \>1.22 due to a variety of interface changes (see #10714, [D1780](https://phabricator.haskell.org/D1780), https://mail.haskell.org/pipermail/ghc-devs/2016-January/010978.html). Unfortunately things currently bl...GHC 8.0 will require Cabal \>1.22 due to a variety of interface changes (see #10714, [D1780](https://phabricator.haskell.org/D1780), https://mail.haskell.org/pipermail/ghc-devs/2016-January/010978.html). Unfortunately things currently blow up late in compilation with a seemingly unrelated error when this requirement isn't met. Namely, we complain about bad interface files for dependencies. For instance,
```
Text/Parsec/Prim.hs:85:1: error:
Bad interface file: /root/.cabal/lib/x86_64-linux-ghc-8.0.0.20160204/mtl-2.2.1-9wMhzcfsAXi7cNtl9n7svO/Control/Monad/Cont/Class.hi
Something is amiss; requested module mtl-2.2.1@mtl-2.2.1-1af5dabd2d7abb688f2145aec87badc4:Control.Monad.Cont.Class differs from name found in the interface file mtl_9wMhzcfsAXi7cNtl9n7svO:Control.Monad.Cont.Class
```
# For users seeing this error
The solution here is to install `Cabal` 1.23 or later and a `cabal-install` linking against it. Currently this requires installing `Cabal` and `cabal-install` from git. This is most easily done using GHC 7.10. Finally, you need to clear your package database.
This can be done with,
```
$ git clone git://github.com/haskell/cabal
$ cd cabal
$ cabal update
$ cabal install Cabal/ cabal-install/
$ rm -R $HOME/.ghc/*-8.0.0*
```8.0.1https://gitlab.haskell.org/ghc/ghc/-/issues/11025Per-database shadowing2019-07-07T18:32:34ZEdward Z. YangPer-database shadowingIn `5b0191f74ab05b187f81ea037623338a615b1619`, I eliminated shadowing, on the justification that we should never pick the same component ID (previously installed package ID) for two distinct packages unless the ABIs of the packages were ...In `5b0191f74ab05b187f81ea037623338a615b1619`, I eliminated shadowing, on the justification that we should never pick the same component ID (previously installed package ID) for two distinct packages unless the ABIs of the packages were the same, so we could just globally assume that component IDs are unique.
However, I've recently come across some cases where we can't easily actually make this be the case. Here are a few of them:
1. GHC's build system assumes that the ghc package will always be a library name of the form HSghc-7.11.a (i.e. that its component ID is ghc-7.11). It's a bit nontrivial to change this assumption, because there are stage1 shenanigans which are rewriting the version number to one without the date. But obviously if you are bootstrapping GHC with itself, the component ID of the GHC you are building will conflict with the component ID of the host GHC.
1. A similar situation exist for bootstrapping packages. When compiling GHC, we want to pick fairly deterministic component IDs for it in order to avoid needless rebuilding. But then if you turn around and use one of these builds to bootstrap GHC, if you use the same deterministic scheme you will end up with conflicting component IDs.
1. More generally, if you are building a package which is already in the global database (and thus not easily removable), it is sometimes VERY CONVENIENT to be able to pick whatever component ID you want, even if it's in the global database. (1) and (2) are cases of this.
So I think what we want to do is bring back shadowing, but have it operate on a PER-DATABASE basis. For example, if we have the following databases:
```
pkg.conf.1
foo-0.1-XXX
bar-0.2-YYY (depends on foo-0.1-XXX)
pkg.conf.2
foo-0.1-XXX
baz-0.3-ZZZ (depends on foo-0.1-XXX)
```
If we stack pkg.conf.2 on top of pkg.conf.1, we invalidate foo-0.1-XXX FROM pkg.conf.1 (which simply means any packages from pkg.conf.1 which refer to it.)
It also occurs to me that we should also be recording the ABIs of packages we depend on, to ensure consistency. Consider:
```
pkg.conf.1
foo-0.1-XXX (abi is AAA)
pkg.conf.2
foo-0.1-XXX (abi is BBB)
pkg.conf.3
bar-0.1-YYY (depends on foo-0.1-XXX)
```
Which is the correct database to use `pkg.conf.3` with? We have no way of telling today! Sure, `pkg.conf.3` is incomplete, but this is a supported mode of operation.
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | -------------- |
| Version | 7.11 |
| Type | Bug |
| TypeOfFailure | OtherFailure |
| Priority | high |
| Resolution | Unresolved |
| Component | Package system |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"Per-database shadowing","status":"New","operating_system":"","component":"Package system","related":[],"milestone":"8.0.1","resolution":"Unresolved","owner":{"tag":"OwnedBy","contents":"ezyang"},"version":"7.11","keywords":[],"differentials":[],"test_case":"","architecture":"","cc":[""],"type":"Bug","description":"In `5b0191f74ab05b187f81ea037623338a615b1619`, I eliminated shadowing, on the justification that we should never pick the same component ID (previously installed package ID) for two distinct packages unless the ABIs of the packages were the same, so we could just globally assume that component IDs are unique.\r\n\r\nHowever, I've recently come across some cases where we can't easily actually make this be the case. Here are a few of them:\r\n\r\n1. GHC's build system assumes that the ghc package will always be a library name of the form HSghc-7.11.a (i.e. that its component ID is ghc-7.11). It's a bit nontrivial to change this assumption, because there are stage1 shenanigans which are rewriting the version number to one without the date. But obviously if you are bootstrapping GHC with itself, the component ID of the GHC you are building will conflict with the component ID of the host GHC.\r\n\r\n2. A similar situation exist for bootstrapping packages. When compiling GHC, we want to pick fairly deterministic component IDs for it in order to avoid needless rebuilding. But then if you turn around and use one of these builds to bootstrap GHC, if you use the same deterministic scheme you will end up with conflicting component IDs.\r\n\r\n3. More generally, if you are building a package which is already in the global database (and thus not easily removable), it is sometimes VERY CONVENIENT to be able to pick whatever component ID you want, even if it's in the global database. (1) and (2) are cases of this.\r\n\r\nSo I think what we want to do is bring back shadowing, but have it operate on a PER-DATABASE basis. For example, if we have the following databases:\r\n\r\n{{{\r\npkg.conf.1\r\n foo-0.1-XXX\r\n bar-0.2-YYY (depends on foo-0.1-XXX)\r\npkg.conf.2\r\n foo-0.1-XXX\r\n baz-0.3-ZZZ (depends on foo-0.1-XXX)\r\n}}}\r\n\r\nIf we stack pkg.conf.2 on top of pkg.conf.1, we invalidate foo-0.1-XXX FROM pkg.conf.1 (which simply means any packages from pkg.conf.1 which refer to it.)\r\n\r\nIt also occurs to me that we should also be recording the ABIs of packages we depend on, to ensure consistency. Consider:\r\n\r\n{{{\r\npkg.conf.1\r\n foo-0.1-XXX (abi is AAA)\r\npkg.conf.2\r\n foo-0.1-XXX (abi is BBB)\r\npkg.conf.3\r\n bar-0.1-YYY (depends on foo-0.1-XXX)\r\n}}}\r\n\r\nWhich is the correct database to use `pkg.conf.3` with? We have no way of telling today! Sure, `pkg.conf.3` is incomplete, but this is a supported mode of operation.","type_of_failure":"OtherFailure","blocking":[]} -->8.0.1Edward Z. YangEdward Z. Yanghttps://gitlab.haskell.org/ghc/ghc/-/issues/10714After implementing new installed package ID (hash of sdist), get rid of packa...2019-07-07T18:34:26ZEdward Z. YangAfter implementing new installed package ID (hash of sdist), get rid of package keysGHC tracking bug for https://github.com/haskell/cabal/issues/2745
Externally, GHC's flags do not have to change much; a user simply passes the installed package ID to the flag currently named `-this-package-key` (but perhaps we should r...GHC tracking bug for https://github.com/haskell/cabal/issues/2745
Externally, GHC's flags do not have to change much; a user simply passes the installed package ID to the flag currently named `-this-package-key` (but perhaps we should rename this.)
Internally, if we can assume that `PackageKey == InstalledPackageId`, we can do away with the `InstalledPackageId` map and get rid of the level of indirection between the bin-pkg-db (which records installed package IDs\`) and GHC's guts (which record package keys).
Blocked on Cabal not actually using ABI hashes to identify packages.
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | -------------- |
| Version | 7.10.2 |
| Type | Bug |
| TypeOfFailure | OtherFailure |
| Priority | normal |
| Resolution | Unresolved |
| Component | Package system |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"After implementing new installed package ID (hash of sdist), get rid of package keys","status":"New","operating_system":"","component":"Package system","related":[],"milestone":"","resolution":"Unresolved","owner":{"tag":"OwnedBy","contents":"ezyang"},"version":"7.10.2","keywords":[],"differentials":[],"test_case":"","architecture":"","cc":[""],"type":"Bug","description":"GHC tracking bug for https://github.com/haskell/cabal/issues/2745\r\n\r\nExternally, GHC's flags do not have to change much; a user simply passes the installed package ID to the flag currently named `-this-package-key` (but perhaps we should rename this.)\r\n\r\nInternally, if we can assume that `PackageKey == InstalledPackageId`, we can do away with the `InstalledPackageId` map and get rid of the level of indirection between the bin-pkg-db (which records installed package IDs`) and GHC's guts (which record package keys). \r\n\r\nBlocked on Cabal not actually using ABI hashes to identify packages.","type_of_failure":"OtherFailure","blocking":[]} -->8.0.1Edward Z. YangEdward Z. Yanghttps://gitlab.haskell.org/ghc/ghc/-/issues/10566Move "package key" generation to GHC2019-07-07T18:35:13ZEdward Z. YangMove "package key" generation to GHCCurrently in GHC 7.10, we have the following situation:
1. Cabal computes a package key, which in practice (since no one is using Backpack in the wild) is a Merkle tree of the versions of each of the dependencies of the package.
1. This...Currently in GHC 7.10, we have the following situation:
1. Cabal computes a package key, which in practice (since no one is using Backpack in the wild) is a Merkle tree of the versions of each of the dependencies of the package.
1. This package key is passed to GHC via `-this-package-key`
1. GHC handles the package key opaquely
Now, in recent Backpack implementation, we need GHC to be able to compute package keys. (The concrete case: you're type-checking an interface file of an indefinite package, where you want to instantiate it with some assignment of its holes: instantiating those holes you need to instantiate any package keys mentioned in the interface, in which case you really want to be able to compute the hash.) So I want to move package key generation to GHC.
The primary implication is this: does Cabal continue to generate package keys? If it doesn't, we should revert from `-this-package-key` back to `-package-name` from the previous version (but maybe renamed because this name was bad). GHC then computes a package key based on `-package-name` and the explicitly mentioned `-package` dependencies, and Cabal reads it out with something akin to `--abi-hash`. If it does, we need to ensure GHC and Cabal's package key algorithms stay synchronized. I personally lean towards the first option.
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | -------------- |
| Version | 7.10.1 |
| Type | Bug |
| TypeOfFailure | OtherFailure |
| Priority | normal |
| Resolution | Unresolved |
| Component | Package system |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"Move \"package key\" generation to GHC","status":"New","operating_system":"","component":"Package system","related":[],"milestone":"7.12.1","resolution":"Unresolved","owner":{"tag":"OwnedBy","contents":"ezyang"},"version":"7.10.1","keywords":[],"differentials":[],"test_case":"","architecture":"","cc":[""],"type":"Bug","description":"Currently in GHC 7.10, we have the following situation:\r\n\r\n1. Cabal computes a package key, which in practice (since no one is using Backpack in the wild) is a Merkle tree of the versions of each of the dependencies of the package.\r\n2. This package key is passed to GHC via `-this-package-key`\r\n3. GHC handles the package key opaquely\r\n\r\nNow, in recent Backpack implementation, we need GHC to be able to compute package keys. (The concrete case: you're type-checking an interface file of an indefinite package, where you want to instantiate it with some assignment of its holes: instantiating those holes you need to instantiate any package keys mentioned in the interface, in which case you really want to be able to compute the hash.) So I want to move package key generation to GHC.\r\n\r\nThe primary implication is this: does Cabal continue to generate package keys? If it doesn't, we should revert from `-this-package-key` back to `-package-name` from the previous version (but maybe renamed because this name was bad). GHC then computes a package key based on `-package-name` and the explicitly mentioned `-package` dependencies, and Cabal reads it out with something akin to `--abi-hash`. If it does, we need to ensure GHC and Cabal's package key algorithms stay synchronized. I personally lean towards the first option.","type_of_failure":"OtherFailure","blocking":[]} -->8.0.1Edward Z. YangEdward Z. Yanghttps://gitlab.haskell.org/ghc/ghc/-/issues/10550Drop truncated package name prefix from package keys2019-07-07T18:35:22ZEdward Z. YangDrop truncated package name prefix from package keysEarly on when we moved to package keys, we decided it would be useful for documentation purposes to have a package key be something like `ghcpr_8TmvWUcS1U1IKHT0levwg3` rather than just `8TmvWUcS1U1IKHT0levwg3`, which would leave a user w...Early on when we moved to package keys, we decided it would be useful for documentation purposes to have a package key be something like `ghcpr_8TmvWUcS1U1IKHT0levwg3` rather than just `8TmvWUcS1U1IKHT0levwg3`, which would leave a user with no idea what the key is.
I now think that this is more trouble than its worth:
1. We have fixed library \*paths\* so that the full package name, version, key is in the path, e.g. libraries are installed to `/home/ezyang/Dev/ghc-7.10.2/usr/lib/ghc-7.10.2/ghc-prim-0.4.0.0-8TmvWUcS1U1IKHT0levwg3` instead of `/home/ezyang/Dev/ghc-7.10.1/usr/lib/ghc-7.10.1/ghcpr_8TmvWUcS1U1IKHT0levwg3`. This means that the biggest case where users might care about package keys is no longer relevant; a user will always be able to get the real package key.
1. The other case where package keys show up is in symbol names. However, symbol names already contain \*modules names\*, so today we have `procezu0hwN3CTKynhHQqQkChnSdH_SystemziProcess_rawSystem1_info` but `0hwN3CTKynhHQqQkChnSdH_SystemziProcess_rawSystem1_info` seems just about as clear (in particular, five letters just doesn't seem like enough to adequately describe long package names.)
1. Now, the reason why we don't want to truncate the prefix name is because it makes it more difficult to reliably, deterministically generate a package key based on a few parameters about a package: namely, any algorithm needs to know that `take 5 (filter (/= '-') packageName)` and prefix that onto the key. Really, we only care about the hash, but for most equality tests the prefix matters too! So it would be much nicer if we didn't have to care about the hash as well.
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | -------------- |
| Version | 7.11 |
| Type | Bug |
| TypeOfFailure | OtherFailure |
| Priority | normal |
| Resolution | Unresolved |
| Component | Package system |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"Drop truncated package name prefix from package keys","status":"New","operating_system":"","component":"Package system","related":[],"milestone":"7.12.1","resolution":"Unresolved","owner":{"tag":"OwnedBy","contents":"ezyang"},"version":"7.11","keywords":[],"differentials":[],"test_case":"","architecture":"","cc":[""],"type":"Bug","description":"Early on when we moved to package keys, we decided it would be useful for documentation purposes to have a package key be something like `ghcpr_8TmvWUcS1U1IKHT0levwg3` rather than just `8TmvWUcS1U1IKHT0levwg3`, which would leave a user with no idea what the key is.\r\n\r\nI now think that this is more trouble than its worth:\r\n\r\n1. We have fixed library *paths* so that the full package name, version, key is in the path, e.g. libraries are installed to `/home/ezyang/Dev/ghc-7.10.2/usr/lib/ghc-7.10.2/ghc-prim-0.4.0.0-8TmvWUcS1U1IKHT0levwg3` instead of `/home/ezyang/Dev/ghc-7.10.1/usr/lib/ghc-7.10.1/ghcpr_8TmvWUcS1U1IKHT0levwg3`. This means that the biggest case where users might care about package keys is no longer relevant; a user will always be able to get the real package key.\r\n\r\n2. The other case where package keys show up is in symbol names. However, symbol names already contain *modules names*, so today we have `procezu0hwN3CTKynhHQqQkChnSdH_SystemziProcess_rawSystem1_info` but `0hwN3CTKynhHQqQkChnSdH_SystemziProcess_rawSystem1_info` seems just about as clear (in particular, five letters just doesn't seem like enough to adequately describe long package names.)\r\n\r\n3. Now, the reason why we don't want to truncate the prefix name is because it makes it more difficult to reliably, deterministically generate a package key based on a few parameters about a package: namely, any algorithm needs to know that `take 5 (filter (/= '-') packageName)` and prefix that onto the key. Really, we only care about the hash, but for most equality tests the prefix matters too! So it would be much nicer if we didn't have to care about the hash as well.","type_of_failure":"OtherFailure","blocking":[]} -->8.0.1Edward Z. YangEdward Z. Yanghttps://gitlab.haskell.org/ghc/ghc/-/issues/4048ghc-pkg should check for existence of extra-libraries2019-07-07T19:00:58ZSimon Marlowghc-pkg should check for existence of extra-librariesIn #3930 we decided that all `extra-libraries` should be found either on the system search path or the paths in the package's `library-dirs` field. We should make `ghc-pkg` check this when registering a package - it can use `gcc -print-s...In #3930 we decided that all `extra-libraries` should be found either on the system search path or the paths in the package's `library-dirs` field. We should make `ghc-pkg` check this when registering a package - it can use `gcc -print-search-dirs` to get the system search path.
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | -------------- |
| Version | 6.12.2 |
| Type | Bug |
| TypeOfFailure | OtherFailure |
| Priority | normal |
| Resolution | Unresolved |
| Component | Package system |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"ghc-pkg should check for existence of extra-libraries","status":"New","operating_system":"","component":"Package system","related":[],"milestone":"7.0.1","resolution":"Unresolved","owner":{"tag":"Unowned"},"version":"6.12.2","keywords":[],"differentials":[],"test_case":"","architecture":"","cc":[""],"type":"Bug","description":"In #3930 we decided that all `extra-libraries` should be found either on the system search path or the paths in the package's `library-dirs` field. We should make `ghc-pkg` check this when registering a package - it can use `gcc -print-search-dirs` to get the system search path.\r\n","type_of_failure":"OtherFailure","blocking":[]} -->8.0.1https://gitlab.haskell.org/ghc/ghc/-/issues/2550Add an option to read file names from a file instead of the command line2019-07-07T19:08:05ZYitzGaleAdd an option to read file names from a file instead of the command lineAs discussed in the following thread:
http://www.haskell.org/pipermail/haskell-cafe/2008-August/046494.html
Add an option to look for modules using manifest files,
rather than assuming that the filesystem contains a hierarchy
of direct...As discussed in the following thread:
http://www.haskell.org/pipermail/haskell-cafe/2008-August/046494.html
Add an option to look for modules using manifest files,
rather than assuming that the filesystem contains a hierarchy
of directories that exactly matches the module hierarchy.
This is desirable for the following reasons:
- It avoids an ugly and ad hoc dependency between two unrelated domains.
- In large and complex projects, it allows the project manager more flexibility in using directory structure to organize teams and manage complex version control setups.
- It avoids problems caused by mismatch between the filesystem and the module hierarchy, such as a non-hierarchical file system or incompatibilities in the way files are named.
- It allows for more than one module per file. This is helpful because the module system is the only way to achieve encapsulation in Haskell, so the system needs to be as lightweight and flexible as possible. It also makes it easier to share and distribute Haskell programs when it is most convenient to do so as a single file, such as literate Haskell via email.
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | -------------- |
| Version | 6.9 |
| Type | FeatureRequest |
| TypeOfFailure | OtherFailure |
| Priority | normal |
| Resolution | Unresolved |
| Component | Compiler |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | |
| Operating system | Unknown |
| Architecture | Unknown |
</details>
<!-- {"blocked_by":[],"summary":"Add an option to use manifest files instead of filesystem hierarchy to find modules","status":"New","operating_system":"Unknown","component":"Compiler","related":[],"milestone":"","resolution":"Unresolved","owner":{"tag":"Unowned"},"version":"6.9","keywords":[],"differentials":[],"test_case":"","architecture":"Unknown","cc":[""],"type":"FeatureRequest","description":"As discussed in the following thread:\r\n\r\nhttp://www.haskell.org/pipermail/haskell-cafe/2008-August/046494.html\r\n\r\nAdd an option to look for modules using manifest files,\r\nrather than assuming that the filesystem contains a hierarchy\r\nof directories that exactly matches the module hierarchy.\r\n\r\nThis is desirable for the following reasons:\r\n\r\n * It avoids an ugly and ad hoc dependency between two unrelated domains.\r\n * In large and complex projects, it allows the project manager more flexibility in using directory structure to organize teams and manage complex version control setups.\r\n * It avoids problems caused by mismatch between the filesystem and the module hierarchy, such as a non-hierarchical file system or incompatibilities in the way files are named.\r\n * It allows for more than one module per file. This is helpful because the module system is the only way to achieve encapsulation in Haskell, so the system needs to be as lightweight and flexible as possible. It also makes it easier to share and distribute Haskell programs when it is most convenient to do so as a single file, such as literate Haskell via email.","type_of_failure":"OtherFailure","blocking":[]} -->8.0.1