GHC issueshttps://gitlab.haskell.org/ghc/ghc/-/issues2023-05-05T13:00:27Zhttps://gitlab.haskell.org/ghc/ghc/-/issues/20489Backpack error message prints out data from interface file instead of source ...2023-05-05T13:00:27Zsheafsam.derbyshire@gmail.comBackpack error message prints out data from interface file instead of source file```haskell
unit sig where
signature Sig where
data A
instance Show A
unit impl where
module Impl where
type A = Int -> Int
unit main where
dependency sig[Sig=impl:Impl]
module Main where
import Impl
main = p...```haskell
unit sig where
signature Sig where
data A
instance Show A
unit impl where
module Impl where
type A = Int -> Int
unit main where
dependency sig[Sig=impl:Impl]
module Main where
import Impl
main = print "no"
```
```
ghc --backpack Error3.bkp
[1 of 3] Processing sig
[1 of 1] Compiling Sig[sig] ( sig\Sig.hsig, nothing ) [Flags changed]
[2 of 3] Processing impl
Instantiating impl
[1 of 1] Compiling Impl ( impl\Impl.hs, impl\Impl.o ) [Flags changed]
[3 of 3] Processing main
Instantiating main
[1 of 1] Including sig[Sig=impl:Impl]
Instantiating sig[Sig=impl:Impl]
[1 of 1] Compiling Sig[sig] ( sig\Sig.hsig, sig\sig-Absk5cIXTXe6UYhGMYGber\Sig.o )
sig\sig-Absk5cIXTXe6UYhGMYGber\..\Sig.hi:1:1: error:
* No instance for (GHC.Show.Show A)
arising when attempting to show that
instance [safe] GHC.Show.Show A -- Defined at Error3.bkp:4:14
is provided by `impl:Impl'
(maybe you haven't applied a function to enough arguments?)
* while checking that impl:Impl implements signature Sig in sig[Sig=impl:Impl]
|
1 | ☺??d♂90320210928?????☻???☺???☺?☺?☺☺☻☺☺☺☻☺☻??????☺????X?☺?Ã?????☺?????<???????☺???◄►???????????????☺?v????☺????????☺???
?????☺???????B???????☺☻???☻?☻♥?☻♦
| ^
```
It seems the problem is that we decide to report information contained in the interface file `Sig.hi`, which doesn't make any sense.https://gitlab.haskell.org/ghc/ghc/-/issues/20488Incorrect module name in .bkp instantiation causes a "Failed to load interfac...2023-05-05T13:00:21Zsheafsam.derbyshire@gmail.comIncorrect module name in .bkp instantiation causes a "Failed to load interface" errorWhen attempting to compile the following .bkp file:
```haskell
unit basic where
signature Basic where
unit mod where
module Mod where
unit main where
dependency basic
[ Basic = mod:Nonsense ]
```
GHC produces the error:
``...When attempting to compile the following .bkp file:
```haskell
unit basic where
signature Basic where
unit mod where
module Mod where
unit main where
dependency basic
[ Basic = mod:Nonsense ]
```
GHC produces the error:
```
ghc --backpack Error2.bkp
[1 of 3] Processing basic
[2 of 3] Processing mod
Instantiating mod
[1 of 1] Compiling Mod ( mod\Mod.hs, mod\Mod.o )
[3 of 3] Processing main
Instantiating main
[1 of 1] Including basic[Basic=mod:Nonsense]
Instantiating basic[Basic=mod:Nonsense]
[1 of 1] Compiling Basic[sig] ( basic\Basic.hsig, basic\basic-ER8okxhuIcsFJ32OYv1Qcp\Basic.o )
<no location info>: error:
Failed to load interface for `Nonsense'
There are files missing in the `mod' package,
try running 'ghc-pkg check'.
Use -v (or `:set -v` in ghci) to see a list of the files searched for.
```
It would be better to catch this problem earlier on, and report a proper error message.https://gitlab.haskell.org/ghc/ghc/-/issues/20440Allow user-written Coercible instances in signature files2022-02-23T14:38:52Zsheafsam.derbyshire@gmail.comAllow user-written Coercible instances in signature filesIt would be nice to allow users to write `Coercible` instances in signature files, to indicate "this can be instantiated to any type that is `Coercible` to some other type". Random example:
```haskell
{-# LANGUAGE MagicHash #-}
{-# LANG...It would be nice to allow users to write `Coercible` instances in signature files, to indicate "this can be instantiated to any type that is `Coercible` to some other type". Random example:
```haskell
{-# LANGUAGE MagicHash #-}
{-# LANGUAGE PolyKinds #-}
{-# LANGUAGE DataKinds #-}
{-# LANGUAGE UnboxedTuples #-}
unit number-unknown where
signature NumberUnknown where
import Data.Coerce
data Number
instance Coercible Number Int
op :: Number -> Number -> Number
module NumberStuff where
import Data.Coerce
import NumberUnknown
funcA :: Number -> Number -> Int
funcA x y = coerce (op x y)
unit number-int where
module NumberUnknown where
type Number = Int
op :: Int -> Int -> Int
op = (+)
unit number-times where
module NumberUnknown where
newtype Number = MkNumber Int
op :: Number -> Number -> Number
op (MkNum x) (MkNum y) = MkNum (x * y)
unit main where
dependency number-unknown[NumberUnknown=number-times:NumberUnknown]
module Main where
import NumberUnknown
import NumberStuff
import GHC.Types
main = print (funcA (MkNumber 3) (MkNumber 5))
```
This is currently rejected with:
```
error:
* Class `Coercible' does not support user-specified instances.
* In the instance declaration for `Coercible Number Int'
|
10 | instance Coercible Number Int
| ^^^^^^^^^^^^^^^^^^^^
```
However, it's not as simple as disabling the check, as it seems that GHC's solver for `~#R` is not smart enough to pick up the `Coercible` instance, as we then get:
```
error:
* Couldn't match representation of type `Number' with that of `Int'
arising from a use of `coerce'
NB: The type constructor `Number' is abstract
* In the expression: coerce (op x y)
In an equation for `funcA': funcA x y = coerce (op x y)
|
16 | funcA x y = coerce (op x y)
| ^^^^^^
```
I assume that this could be remedied by allowing top-level interactions for `(~#)`, similar to what I have done for `Concrete` in !6164.https://gitlab.haskell.org/ghc/ghc/-/issues/20358Allow "instances" for non-typeclass constraints in signature files2023-11-21T15:47:16Zsheafsam.derbyshire@gmail.comAllow "instances" for non-typeclass constraints in signature filesThe [`contiguous`](https://hackage.haskell.org/package/contiguous/docs/Data-Primitive-Contiguous-Class.html) library defines a typeclass for primitive arrays, with an associated constraint:
```haskell
type Contiguous :: ( Type -> Type )...The [`contiguous`](https://hackage.haskell.org/package/contiguous/docs/Data-Primitive-Contiguous-Class.html) library defines a typeclass for primitive arrays, with an associated constraint:
```haskell
type Contiguous :: ( Type -> Type ) -> Constraint
class Contiguous arr where
type Element arr :: Type -> Constraint
-- ... array methods
```
Now, with backpack one might want to declare:
- an array type that is an instance of `Contiguous`,
- a key type which is an allowed element of the array.
Specifically:
```haskell
data Key
data Array k
instance Contiguous Array
instance Element Array Key
```
This currently isn't allowed:
```haskell
* Illegal instance for a associated type family
A class instance must be for a class
* In the instance declaration for `Element Array Key'
|
| instance Element Array Key
| ^^^^^^^^^^^^^^^^^
```
However I think it would make sense to allow this; I don't think there's any other available mechanism to specify that any instantiation of `Key` and `Array` should satisfy the additional constraint `Element Array Key`. Of course it's not strictly an instance (as `Element` is not a typeclass, but a constraint synonym), but the alternative would be to invent backpack-specific syntax.https://gitlab.haskell.org/ghc/ghc/-/issues/20133Backpack can end up confused when a non-nullary type synonym implements abstr...2021-10-19T10:34:45Zsheafsam.derbyshire@gmail.comBackpack can end up confused when a non-nullary type synonym implements abstract dataGHC seems to get confused with the following `.bkp` backpack file modified from test case `T13955` in which an argument is added to the `Rep` declaration:
```haskell
{-# LANGUAGE MagicHash #-}
{-# LANGUAGE PolyKinds #-}
{-# LANGUAGE Dat...GHC seems to get confused with the following `.bkp` backpack file modified from test case `T13955` in which an argument is added to the `Rep` declaration:
```haskell
{-# LANGUAGE MagicHash #-}
{-# LANGUAGE PolyKinds #-}
{-# LANGUAGE DataKinds #-}
unit number-unknown where
signature NumberUnknown where
import GHC.Types
data Rep a :: RuntimeRep
data Number a :: TYPE (Rep a)
plus :: Number a -> Number a -> Number a
multiply :: Number a -> Number a -> Number a
module NumberStuff where
import NumberUnknown
funcA :: Number a -> Number a -> Number a
funcA x y = plus x (multiply x y)
unit number-unboxed-int where
module NumberUnknown where
import GHC.Types
import GHC.Prim
type Rep a = IntRep
type Number a = Int#
plus :: Int# -> Int# -> Int#
plus = (+#)
multiply :: Int# -> Int# -> Int#
multiply = (*#)
unit main where
dependency number-unknown[NumberUnknown=number-unboxed-int:NumberUnknown]
module Main where
import NumberStuff
import GHC.Types
main = print (I# (funcA 2# 3#))
```
Compiled with `ghc T20133.bkp --backpack`, we get the error:
```
T13955b.bkp:15:14: error:
* Expected kind `k0 -> *',
but `Number' has kind `TYPE 'GHC.Types.IntRep'
* In the type signature: funcA :: Number a -> Number a -> Number a
|
15 | funcA :: Number a -> Number a -> Number a
| ^^^^^^^^
T13955b.bkp:15:26: error:
* Expected kind `k0 -> *',
but `Number' has kind `TYPE 'GHC.Types.IntRep'
* In the type signature: funcA :: Number a -> Number a -> Number a
|
15 | funcA :: Number a -> Number a -> Number a
| ^^^^^^^^
T13955b.bkp:15:38: error:
* Expected kind `k0 -> *',
but `Number' has kind `TYPE 'GHC.Types.IntRep'
* In the type signature: funcA :: Number a -> Number a -> Number a
|
15 | funcA :: Number a -> Number a -> Number a
| ^^^^^^^^
```
It looks like GHC gets confused when typechecking `main` (indeed, commenting it out prevents the error); so the line numbers should be understood as referring to the instantiation of the `NumberUnknown` signature.
This is what `-ddump-tc-trace` looks like starting from there:
```
[3 of 3] Processing main
Instantiating main
[1 of 1] Including number-unknown[NumberUnknown=number-unboxed-int:NumberUnknown]
Instantiating number-unknown[NumberUnknown=number-unboxed-int:NumberUnknown]
[1 of 2] Compiling NumberUnknown[sig] ( number-unknown\NumberUnknown.hsig, nothing ) [Source file changed]
checkHsigIface
[ryc :-> Identifier `multiply', ryd :-> Identifier `plus',
rye :-> Type constructor `Number', ryf :-> Type constructor `Rep',
ryg :-> Identifier `number-unboxed-int:NumberUnknown.$trModule',
ryU :-> Identifier `number-unboxed-int:NumberUnknown.$tcNumber',
ryV :-> Identifier `number-unboxed-int:NumberUnknown.$tcRep']
[]
[multiply, plus, Number{Number}, Rep{Rep}]
[2 of 2] Compiling NumberStuff ( number-unknown\NumberStuff.hs, nothing ) [Source file changed]
Adding diagnostic:
testsuite\tests\backpack\should_run\T13955b.bkp:13:3:
Module `Prelude' implicitly imported
checkFamInstConsistency [Prelude, NumberUnknown]
Tc2 (src)
Tc3
tcExtendKindEnvList []
tcExtendKindEnvList []
tcDeriving []
rnd
Adding instances:
Tc3b
Tc3c
tcSemigroupWarnings
Tc4
Tc4a
Tc5
tcExtendKindEnvList []
tcHsSigType { Number a_ayY -> Number a_ayY -> Number a_ayY
pushLevelAndSolveEqualitiesX { Called from tc_lhs_sig_type
newAnonMetaTyVar t_az1[tau:1]
pushLevelAndCaptureConstraints { 2
newMetaKindVar k_az2[tau:2]
bindImplicitTKBndrsX
[a_ayY]
[a_ayY[sk:2]]
tc_extend_local_env
[(a_ayY, Type variable `a_ayY' = a_ayY[sk:2] :: k_az2[tau:2])]
tcExtendBinderStack [a_ayY a_ayY[sk:2]]
newAnonMetaTyVar t_az3[tau:2]
newAnonMetaTyVar t_az4[tau:2]
lk1 Number
tcInferTyApps {
Number
[HsValArg a_ayY]
newMetaKindVar k_az5[tau:2]
newMetaKindVar k_az6[tau:2]
u_tys
tclvl 2
TYPE 'GHC.Types.IntRep ~ k_az5[tau:2] -> k_az6[tau:2]
a type equality[True] TYPE 'GHC.Types.IntRep
~
k_az5[tau:2] -> k_az6[tau:2]
```
We see here that GHC gets confused about the kind of `Number` (it thinks `Number :: TYPE IntRep`, somehow ignoring that `Number` takes an argument), and it all goes downhill from there.
Sorry for the ticket title, I didn't really know how to summarise the issue. I'm not very familiar with backpack, so I apologise if I made an elementary error somewhere.https://gitlab.haskell.org/ghc/ghc/-/issues/20030--make driver should "make" use of graph more2022-02-23T14:48:29ZDivam Narula--make driver should "make" use of graph more## Problem
I have found a few issues in the --make driver code, which all relate to the graph in some ways
1. Compilation of backpack indefinite package is broken with -j
The reason behind this is that the `InstantiationNode`'s compil...## Problem
I have found a few issues in the --make driver code, which all relate to the graph in some ways
1. Compilation of backpack indefinite package is broken with -j
The reason behind this is that the `InstantiationNode`'s compilation does not wait on any other nodes.
Nor does the compilation of `ModuleNodes` wait on `InstantiationNode`.
2. `parUpsweep` contains a logic for handling the `ext_loop_deps`,
which is not explicitly handled in the original graph, nor in the `upsweep`.
3. `parUpsweep` recomputes the deps
The graph created by `moduleGraphNodes` is correct, but doing topological sort removes all the dependency information
For single threaded `upsweep` it is not an issue (as long as `-fkeep-going` is not used*)
, but the `parUpsweep` needs to figure out all the dependencies again.
## Solution
I think the right solution here is that `upsweep` and `parUpweep` should work on an input graph, and there should not be a need to do any extra patching over the graph.
Fixing this properly requires these two
* The `parUpweep` specifically should not compute the deps info, it should be an input computed from the graph.
* The graph should take into consideration the `ext_loop_deps`
Here is the summary of a solution I have in mind
1. Make the initial graph, just like currently being done by `moduleGraphNodes`
2. Do topological sort and filter out all cyclic modules
3. For each ACyclic SCC
a. Create another graph
b. Determine all the module loops
c. For each module loop, fix the edges of external modules which directly depend on any module in the loop.
(ie change the head of the edge to the loop closer.)
4. Use the updated graph to construct a suitable datastructure
which contains the set of MVars the `parUpweep` needs to wait on.
(eg. `[BuildModule, [MVar ...]]`)
**Open question: In case of multiple connected module loops, what should be the behavior of typecheckLoop**
### Example
Below is concrete example of how these modifications would happen
(omitting the `InstantiationNode`, since it behaves similar to a normal `HsSrc` node with respect to this algo)
1. Initial graph created by direct source imports
```mermaid
graph LR;
A.hs-boot --> B.hs;
A.hs-boot --> D.hs;
B.hs --> E.hs-boot;
B.hs --> A.hs;
C.hs-boot --> A.hs;
A.hs --> C.hs;
A.hs --> F.hs;
E.hs-boot --> G.hs;
G.hs --> H.hs;
H.hs --> E.hs;
G.hs --> I.hs;
J.hs-boot --> J.hs;
J.hs-boot --> K.hs;
```
3a. Determine module loops
Number of module loops == Number of HsBoot modules
Loops could be represented by a NonEmpty style list, where head is the loop closer
```
A :| [B]
C :| [A]
E :| [G, H]
J :| []
```
**As far as I could determine the ordering of the processing of loops does not matter here**
I. Process A.hs-boot
```mermaid
graph LR;
A.hs-boot --> B.hs; %% [isLoopMember]
A.hs --> D.hs; %% (modified by I)
A.hs --> E.hs-boot; %% (modified by I)
B.hs --> A.hs;
C.hs-boot --> A.hs;
A.hs --> C.hs;
A.hs --> F.hs;
E.hs-boot --> G.hs;
G.hs --> H.hs;
H.hs --> E.hs;
G.hs --> I.hs;
J.hs-boot --> J.hs;
J.hs-boot --> K.hs;
```
II. Process C.hs-boot
```mermaid
graph LR;
A.hs-boot --> B.hs; %% [isLoopMember]
C.hs --> D.hs; %% (modified by I, II)
C.hs --> E.hs-boot; %% (modified by I, II)
B.hs --> A.hs;
C.hs-boot --> A.hs; %% [isLoopMember]
A.hs --> C.hs;
C.hs --> F.hs; %% (modified by II)
E.hs-boot --> G.hs;
G.hs --> H.hs;
H.hs --> E.hs;
G.hs --> I.hs;
J.hs-boot --> J.hs;
J.hs-boot --> K.hs;
```
III. Process E.hs-boot
```mermaid
graph LR;
A.hs-boot --> B.hs; %% [isLoopMember]
C.hs --> D.hs; %% (modified by I, II)
C.hs --> E.hs-boot; %% (modified by I, II)
B.hs --> A.hs;
C.hs-boot --> A.hs;
A.hs --> C.hs;
C.hs --> F.hs; %% (modified by II)
E.hs-boot --> G.hs;
G.hs --> H.hs;
H.hs --> E.hs;
E.hs --> I.hs;
J.hs-boot --> J.hs;
J.hs-boot --> K.hs;
```
IV. Process J.hs-boot
```mermaid
graph LR;
A.hs-boot --> B.hs; %% [isLoopMember]
C.hs --> D.hs; %% (modified by I, II)
C.hs --> E.hs-boot; %% (modified by I, II)
B.hs --> A.hs;
C.hs-boot --> A.hs;
A.hs --> C.hs;
C.hs --> F.hs; %% (modified by II)
E.hs-boot --> G.hs;
G.hs --> H.hs;
H.hs --> E.hs;
E.hs --> I.hs;
J.hs-boot --> J.hs;
J.hs --> K.hs; %% (modified by IV)
```
### Problematic case
```mermaid
graph TB;
subgraph one
L1.hs-boot --> L1_1.hs;
L1_1.hs --> L1.hs;
end
subgraph three
M.hs
end
subgraph two
L2.hs-boot --> L2_1.hs;
L2_1.hs --> L2.hs;
end
L1_1.hs --> M.hs;
L2_1.hs ==> L1.hs;
M.hs --> L2.hs;
```
In this case the L1 depends on L2_1 ( highlighted edge), but it should not be changed to L2.
(It is ok for M to depend on L1, as shown by dotted)
```mermaid
graph TB;
subgraph one
L1.hs-boot --> L1_1.hs;
L1_1.hs --> L1.hs;
end
subgraph three
M.hs
end
subgraph two
L2.hs-boot --> L2_1.hs;
L2_1.hs --> L2.hs;
end
L1_1.hs --> M.hs;
L1.hs -..-> M.hs;
L2_1.hs ==> L1.hs;
M.hs --> L2.hs;
```
Also this somehow magically works on master branch, as the L1 is considered part of the L2 loop!
```
comp_graph_loops
[[L2, L2 {-# SOURCE #-}, L2_1, L1, L2],
[L1, L1 {-# SOURCE #-}, L1_1, L1]]
```Divam NarulaDivam Narulahttps://gitlab.haskell.org/ghc/ghc/-/issues/19496Duplicate signatures clobbered in .bkp files2021-06-08T00:43:12ZhdgarroodDuplicate signatures clobbered in .bkp files## Summary
In `.bkp` files, it is possible to declare a signature multiple times within a unit, and it appears that only the last signature of a given name is considered; all others are ignored. I'm not certain if this is actually a bug...## Summary
In `.bkp` files, it is possible to declare a signature multiple times within a unit, and it appears that only the last signature of a given name is considered; all others are ignored. I'm not certain if this is actually a bug, or how this is supposed to behave. cc @ezyang
## Steps to reproduce
Write the following into a file `test.bkp`:
```
unit test where
-- These two declarations of @signature A@ should be mergeable
signature A where
f :: Int
signature A where
g :: String
-- These two declarations of @signature B@ are certainly not mergeable
signature B where
f :: Int
signature B where
f :: String
```
Then run `ghc --backpack test.bkp`, and inspect the generated `.hi` files with `ghc --show-iface test/A.hi` and `ghc --show-iface test/B.hi`.
Observed behaviour: GHC succeeds and writes out `hi` files for both A and B. `A.hi` lists only `g :: String` among its exports, and `B.hi` lists only `f :: String` among its exports.
## Expected behavior
I am not entirely sure how signature merging is supposed to work in this scenario, but I expected one of two things to happen here:
1. GHC would point out that `A` and `B` have both been declared multiple times, and no `.hi` files would be produced at all
2. GHC would produce an `A.hi` file which listed both `f :: Int` and `g :: String` as exports, but would fail with an error at B, saying that the signatures were not mergeable
## Environment
* GHC version used: reproduced on both 8.10.4 and 9.0.1
Optional:
* Operating System: Ubuntu linux
* System Architecture: x86_64https://gitlab.haskell.org/ghc/ghc/-/issues/18563-Wunused-packages doesn't understand backpack semantics2023-11-08T23:23:31ZGesh-Wunused-packages doesn't understand backpack semantics## Summary
`-Wunused-packages` warns that signatures don't need their dependencies and that indefinite packages don't need their signatures.
## Steps to reproduce
Consider this simple backpack project:
```
==> bug.cabal <==
cabal-vers...## Summary
`-Wunused-packages` warns that signatures don't need their dependencies and that indefinite packages don't need their signatures.
## Steps to reproduce
Consider this simple backpack project:
```
==> bug.cabal <==
cabal-version: 3.0
name: bug
version: 0.1.0.0
common default
ghc-options: -Werror -Wunused-packages
default-language: Haskell2010
default-extensions: NoImplicitPrelude
build-depends: base
library sig
import: default
signatures: Sig
build-depends: semigroups
hs-source-dirs: sig
library lib
import: default
exposed-modules: Lib
hs-source-dirs: lib
build-depends: sig
library imp
import: default
exposed-modules: Imp
build-depends: semigroups
hs-source-dirs: imp
executable app
import: default
main-is: App.hs
hs-source-dirs: app
build-depends:
, imp
, lib
mixins: imp (Imp as Sig)
==> sig/Sig.hsig <==
signature Sig (T, t, f) where
import Data.List.NonEmpty (NonEmpty)
import Prelude (Show)
data T
instance Show T
t :: T
f :: NonEmpty T -> T
==> lib/Lib.hs <==
module Lib (u) where
import Sig (T, f, t)
u :: T
u = f t
==> imp/Imp.hs <==
module Imp (T, t, f) where
import Data.List.NonEmpty (NonEmpty, head)
import Prelude (Show)
data T = L | R deriving (Show)
t :: T
t = L
f :: NonEmpty T -> T
f = head
==> app/App.hs <==
import Lib (u)
import Prelude (print, IO)
main :: IO ()
main = print u
```
Then `cabal build sig` and `imp` complain that
```
<no location info>: error: [-Wunused-packages, -Werror=unused-packages]
The following packages were specified via -package or -package-id flags,
but were not needed for compilation:
- semigroups-0.19.1
```
If we suppress this by passing `-Wno-unused-packages` to the `sig` build,
then `cabal build lib` complains that
```
<no location info>: error: [-Wunused-packages, -Werror=unused-packages]
The following packages were specified via -package or -package-id flags,
but were not needed for compilation:
- bug-0.1.0.0:sig[Sig=<Sig>]
```
Full error logs have been posted at ix.io, for [no suppression](http://ix.io/2tSF), [sig suppression](http://ix.io/2tSJ) and [sig+lib suppression](http://ix.io/2tSS)
## Expected behavior
This shouldn't complain at all. However, judging by #13493, this might be a bit difficult to implement, so at least having this check have a toggle to switch between off/no-backpack/on would help.
## Environment
* GHC version used: 8.10.1Matthew PickeringMatthew Pickeringhttps://gitlab.haskell.org/ghc/ghc/-/issues/17942Backpack fails with: Can't find interface-file declaration2020-03-22T17:51:26ZAndrew MartinBackpack fails with: Can't find interface-file declaration## Summary
When using an indefinite module to fill in a signature, backpack succeeds. However, it fails later during instantiation with:
```
typecheckIfaceForInstantiate
Declaration for write#:
Can't find interface-file declaration f...## Summary
When using an indefinite module to fill in a signature, backpack succeeds. However, it fails later during instantiation with:
```
typecheckIfaceForInstantiate
Declaration for write#:
Can't find interface-file declaration for type constructor or class M
Probable cause: bug in .hi-boot file, or inconsistent .hi file
Use -ddump-if-trace to get an idea of which file caused the error
```
At https://github.com/andrewthad/vex-minimal-reproducer, there is a somewhat minimal reproducer. The repo also includes a readme with more detail about the motivation and some investigation of the error itself.
## Steps to reproduce
```
git clone https://github.com/andrewthad/vex-minimal-reproducer
cd vex-minimal-reproducer
cabal v2-build
```
## Expected behavior
The build should succeed.
## Environment
* GHC version used: 8.8.3, 8.10.1rc, HEAD
cc @ezyangEdward Z. YangEdward Z. Yanghttps://gitlab.haskell.org/ghc/ghc/-/issues/17525Backpack documentation is lacking2021-12-29T11:30:29ZBen GamariBackpack documentation is lackingSince its merge Backpack has been long lacking in the documentation department. In particular, there are several questions of the implementation that are currently unaddressed in the codebase:
* What is the motivation for and semantics...Since its merge Backpack has been long lacking in the documentation department. In particular, there are several questions of the implementation that are currently unaddressed in the codebase:
* What is the motivation for and semantics of the `.bkp` file format?
* How do Cabal (and other build systems) interact with
* Where are the Notes that do discuss Backpack found?
* How does Backpack interact with the EPS? How are hole instantiations incorporated into `UnitId`s?
* What shortcomings does the current implementation suffer from? For instance, until very recently (#17188) `-j` was not supported under Backpack. There are also known limitations regarding promotion (#13149). Do we have a comprehensive list of such limitations somewhere (perhaps in the user documentation)?
Furthermore, the users guide has not a single mention of the feature, its motivation, or user-facing interface.Matthew Pickeringsheafsam.derbyshire@gmail.comMatthew Pickeringhttps://gitlab.haskell.org/ghc/ghc/-/issues/17522Document .bkp files2023-05-05T12:59:39ZRichard Eisenbergrae@richarde.devDocument .bkp filesBackpack testing uses `.bkp` files, which are slurped in when GHC is passed the `-backpack` flag. From some conversations I've had, this flag is meant only for internal use, in our testsuite.
Yet this aspect of GHC seems entirely undocu...Backpack testing uses `.bkp` files, which are slurped in when GHC is passed the `-backpack` flag. From some conversations I've had, this flag is meant only for internal use, in our testsuite.
Yet this aspect of GHC seems entirely undocumented. `.bkp` files have a concrete syntax implemented in the parser, a potentially user-accessible flag, and quite a bit of code to support it all... but with no description that I have seen. (Maybe I've just looked in the wrong place.)
This ticket is therefore to track the documentation of this feature. Generally, I would expect at least a passing mention in the manual ("GHC has this `-backpack` flag, meant for internal use only; not a user-facing feature") and a long Note describing how the files are used in practice in the testsuite and how they are implemented.
(This was born from a discussion in !1767, but no need to look there to understand this ticket.)https://gitlab.haskell.org/ghc/ghc/-/issues/17224Backpack and instance defaults2020-10-18T02:01:41ZJohn EricsonBackpack and instance defaults[N.B. #17190 may be useful for some backstory.]
In Haskell today, if an associated type or data family has a default instance, hsig and hs-boot interfaces unconditionally opt-into that default. Unlike regular methods, the definition i...[N.B. #17190 may be useful for some backstory.]
In Haskell today, if an associated type or data family has a default instance, hsig and hs-boot interfaces unconditionally opt-into that default. Unlike regular methods, the definition is meaningful for an abstract instance because we use it at the type level. And yes, that means with dependent Haskell we will have this problem regular methods too, when everything can be reasoned about equationally (#13149).
Also, in the short term interfaces don't support type families so this is extra bad because it means one cannot use defaults with interfaces at all, not just when they intend to declare an instance that doesn't necessary opt into the default. (symptom with defaults #17190, underlying issue with type families #8441).
Given all that, I think we should choose a syntax for opting in or out of defaults in signatures. I furthermore think we should choose it real soon because #8441 is said by @ezyang to be had to implement, and I'd like to resolve #17190 in both a forwards-compatible fashion without crossing that minefield.
I see two choices:
1. `default foo, Foo` syntax to indicate a method (Type or term level) opts into a default.
2. Deprecate default methods. It keeps on requiring new syntax (default sigs, this) and isn't as flexible as deriving-via. Fix any outstanding issues with deriving-via and default methods.https://gitlab.haskell.org/ghc/ghc/-/issues/15391Maybe ghc-pkg register should unregister packages with "incompatible" signatures2021-09-07T14:56:49ZEdward Z. YangMaybe ghc-pkg register should unregister packages with "incompatible" signaturesConsider the following situation:
1. I register package 'p-0.1-inplace' instantiated with 'P = base:Data.Map'
1. I register package 'p-0.1-inplace' instantiated with 'P = base:Control.Monad'
Even though the IPID of these two packages i...Consider the following situation:
1. I register package 'p-0.1-inplace' instantiated with 'P = base:Data.Map'
1. I register package 'p-0.1-inplace' instantiated with 'P = base:Control.Monad'
Even though the IPID of these two packages is the same, we clearly want to keep both of them in the database. This thus motivates the following lines in ghc-pkg
```
removes = [ RemovePackage p
| not multi_instance,
p <- packages db_to_operate_on,
mungedId p == mungedId pkg,
-- Only remove things that were instantiated the same way!
instantiatedWith p == instantiatedWith pkg ]
```
OK... now consider the following situation
1. I register package 'p-0.1-inplace' instantiated with 'P = base:Data.Map'
1. I register package 'p-0.1-inplace', instantiated with nothing (I have edited 'p' such that it has no more signatures)
So... ghc-pkg is going to keep both of these packages (per the logic above). But is that really what we want? Now the package database is in a half-consistent state, where the instances of 'p-0.1-inplace' are not consistent with the number of holes they are supposed to have. So let us suggest an invariant:
INVARIANT: All packages which have the same IPID in a package database are self-consistent with each other.
OK, so given this invariant, what do we have to do in step 2? It would seem that when we get a prospective package to register, we must \*find all packages which are incompatible with it\*, and remove them from the package database. So, given a package with a set of holes H, we must remove all packages with holes H' such that H = H'
I'm not completely convinced this is a good idea. Usually ghc-pkg only removes a single package in response to a new registration, and this behavior could result in lots of packages getting unregistered, more than you might expect. On the other hand, if you think of the package database in a more hierarchical manner, it makes sense that changing the root registration would invalidate all the children.
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | ------------ |
| Version | 8.4.3 |
| Type | Bug |
| TypeOfFailure | OtherFailure |
| Priority | normal |
| Resolution | Unresolved |
| Component | ghc-pkg |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | int-index |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"Maybe ghc-pkg register should unregister packages with \"incompatible\" signatures","status":"New","operating_system":"","component":"ghc-pkg","related":[],"milestone":"8.6.1","resolution":"Unresolved","owner":{"tag":"Unowned"},"version":"8.4.3","keywords":["backpack"],"differentials":[],"test_case":"","architecture":"","cc":["int-index"],"type":"Bug","description":"Consider the following situation:\r\n\r\n1. I register package 'p-0.1-inplace' instantiated with 'P = base:Data.Map'\r\n2. I register package 'p-0.1-inplace' instantiated with 'P = base:Control.Monad'\r\n\r\nEven though the IPID of these two packages is the same, we clearly want to keep both of them in the database. This thus motivates the following lines in ghc-pkg\r\n\r\n{{{\r\n removes = [ RemovePackage p\r\n | not multi_instance,\r\n p <- packages db_to_operate_on,\r\n mungedId p == mungedId pkg,\r\n -- Only remove things that were instantiated the same way!\r\n instantiatedWith p == instantiatedWith pkg ]\r\n}}}\r\n\r\nOK... now consider the following situation\r\n\r\n1. I register package 'p-0.1-inplace' instantiated with 'P = base:Data.Map'\r\n2. I register package 'p-0.1-inplace', instantiated with nothing (I have edited 'p' such that it has no more signatures)\r\n\r\nSo... ghc-pkg is going to keep both of these packages (per the logic above). But is that really what we want? Now the package database is in a half-consistent state, where the instances of 'p-0.1-inplace' are not consistent with the number of holes they are supposed to have. So let us suggest an invariant:\r\n\r\nINVARIANT: All packages which have the same IPID in a package database are self-consistent with each other.\r\n\r\nOK, so given this invariant, what do we have to do in step 2? It would seem that when we get a prospective package to register, we must *find all packages which are incompatible with it*, and remove them from the package database. So, given a package with a set of holes H, we must remove all packages with holes H' such that H = H'\r\n\r\nI'm not completely convinced this is a good idea. Usually ghc-pkg only removes a single package in response to a new registration, and this behavior could result in lots of packages getting unregistered, more than you might expect. On the other hand, if you think of the package database in a more hierarchical manner, it makes sense that changing the root registration would invalidate all the children.","type_of_failure":"OtherFailure","blocking":[]} -->https://gitlab.haskell.org/ghc/ghc/-/issues/14478Abstract pattern synonyms (for hsig and hs-boot)2019-07-07T18:16:52ZEdward Z. YangAbstract pattern synonyms (for hsig and hs-boot)Most declaration forms (data types, values, type families, etc) support forward declaration in hs-boot/hsig files. However, pattern synonyms do not. This seems like a major omission!
Some problems to solve:
- The obvious syntax `patter...Most declaration forms (data types, values, type families, etc) support forward declaration in hs-boot/hsig files. However, pattern synonyms do not. This seems like a major omission!
Some problems to solve:
- The obvious syntax `pattern Foo :: pat_ty` is insufficient to specify whether or not a pattern is bidirectional or unidirectional. How should this be represented syntactically?
- What is the interaction with bundling? Should it be possible to export a bundle of abstract pattern synonyms, with the intent that this means an implementation must also have bundled them together
- See also #12717; abstract pattern synonym should be implementable with a plain old constructor
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | ----------------------- |
| Version | 8.2.1 |
| Type | FeatureRequest |
| TypeOfFailure | OtherFailure |
| Priority | normal |
| Resolution | Unresolved |
| Component | Compiler (Type checker) |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"Abstract pattern synonyms (for hsig and hs-boot)","status":"New","operating_system":"","component":"Compiler (Type checker)","related":[],"milestone":"","resolution":"Unresolved","owner":{"tag":"Unowned"},"version":"8.2.1","keywords":["backpack"],"differentials":[],"test_case":"","architecture":"","cc":[""],"type":"FeatureRequest","description":"Most declaration forms (data types, values, type families, etc) support forward declaration in hs-boot/hsig files. However, pattern synonyms do not. This seems like a major omission!\r\n\r\nSome problems to solve:\r\n\r\n* The obvious syntax `pattern Foo :: pat_ty` is insufficient to specify whether or not a pattern is bidirectional or unidirectional. How should this be represented syntactically?\r\n* What is the interaction with bundling? Should it be possible to export a bundle of abstract pattern synonyms, with the intent that this means an implementation must also have bundled them together\r\n* See also #12717; abstract pattern synonym should be implementable with a plain old constructor","type_of_failure":"OtherFailure","blocking":[]} -->https://gitlab.haskell.org/ghc/ghc/-/issues/14212Give better error message with non-supported Backpack/TH use2022-02-23T14:30:13ZEdward Z. YangGive better error message with non-supported Backpack/TH useThis is something of a follow up to #13268.
Suppose you have an indefinite package. If you use TemplateHaskell with a splice that was defined in the indefinite package itself, there is no reasonable way to expect this to work, since in ...This is something of a follow up to #13268.
Suppose you have an indefinite package. If you use TemplateHaskell with a splice that was defined in the indefinite package itself, there is no reasonable way to expect this to work, since in general you can't have compiled the the imported module before you need to run the splice (and the splice needs to be run before you know how the holes are instantiated.)
But say you try and do it anyway. GHC will give an error like this:
```
ghc: ^^ Could not load 'qzm0zi1zminplacezmENEL5G3hx7IK2t0f6HloyH_A_f_closure', dependency unresolved. See top entry above.
ByteCodeLink.lookupCE
During interactive linking, GHCi couldn't find the following symbol:
qzm0zi1zminplacezmENEL5G3hx7IK2t0f6HloyH_A_f_closure
This may be due to you not asking GHCi to load extra object files,
archives or DLLs needed by your current session. Restart GHCi, specifying
the missing library using the -L/path/to/object/dir and -lmissinglibname
flags, or simply by naming the relevant files on the GHCi command line.
Alternatively, this link failure might indicate a bug in GHCi.
If you suspect the latter, please send a bug report to:
glasgow-haskell-bugs@haskell.org
```
This is not a good error message. A good error message would be to say that you can't use in a splice a function defined in an indefinite package (as per the current library.)
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | ---------------- |
| Version | 8.2.1 |
| Type | Bug |
| TypeOfFailure | OtherFailure |
| Priority | normal |
| Resolution | Unresolved |
| Component | Template Haskell |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"Give better error message with non-supported Backpack/TH use","status":"New","operating_system":"","component":"Template Haskell","related":[],"milestone":"","resolution":"Unresolved","owner":{"tag":"Unowned"},"version":"8.2.1","keywords":["backpack"],"differentials":[],"test_case":"","architecture":"","cc":[""],"type":"Bug","description":"This is something of a follow up to #13268.\r\n\r\nSuppose you have an indefinite package. If you use TemplateHaskell with a splice that was defined in the indefinite package itself, there is no reasonable way to expect this to work, since in general you can't have compiled the the imported module before you need to run the splice (and the splice needs to be run before you know how the holes are instantiated.)\r\n\r\nBut say you try and do it anyway. GHC will give an error like this:\r\n\r\n{{{\r\nghc: ^^ Could not load 'qzm0zi1zminplacezmENEL5G3hx7IK2t0f6HloyH_A_f_closure', dependency unresolved. See top entry above.\r\n\r\n\r\nByteCodeLink.lookupCE\r\nDuring interactive linking, GHCi couldn't find the following symbol:\r\n qzm0zi1zminplacezmENEL5G3hx7IK2t0f6HloyH_A_f_closure\r\nThis may be due to you not asking GHCi to load extra object files,\r\narchives or DLLs needed by your current session. Restart GHCi, specifying\r\nthe missing library using the -L/path/to/object/dir and -lmissinglibname\r\nflags, or simply by naming the relevant files on the GHCi command line.\r\nAlternatively, this link failure might indicate a bug in GHCi.\r\nIf you suspect the latter, please send a bug report to:\r\n glasgow-haskell-bugs@haskell.org\r\n}}}\r\n\r\nThis is not a good error message. A good error message would be to say that you can't use in a splice a function defined in an indefinite package (as per the current library.)","type_of_failure":"OtherFailure","blocking":[]} -->https://gitlab.haskell.org/ghc/ghc/-/issues/14210bkp files cannot find TemplateHaskell symbols (even without Backpack features)2023-05-05T12:59:22ZEdward Z. Yangbkp files cannot find TemplateHaskell symbols (even without Backpack features)```
{-# LANGUAGE TemplateHaskell #-}
unit th where
module TH where
f = [d| x = True |]
unit p where
dependency th
module B where
import TH
$(f)
```
When run, this gives:
```
ghc: ^^ Could not load 't...```
{-# LANGUAGE TemplateHaskell #-}
unit th where
module TH where
f = [d| x = True |]
unit p where
dependency th
module B where
import TH
$(f)
```
When run, this gives:
```
ghc: ^^ Could not load 'th_TH_f_closure', dependency unresolved. See top entry above.
ByteCodeLink.lookupCE
During interactive linking, GHCi couldn't find the following symbol:
th_TH_f_closure
This may be due to you not asking GHCi to load extra object files,
archives or DLLs needed by your current session. Restart GHCi, specifying
the missing library using the -L/path/to/object/dir and -lmissinglibname
flags, or simply by naming the relevant files on the GHCi command line.
Alternatively, this link failure might indicate a bug in GHCi.
If you suspect the latter, please send a bug report to:
glasgow-haskell-bugs@haskell.org
```
I am not sure how easy or hard the fix is. We should bump this priority up if we get more serious about supporting Template Haskell with Backpack.
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | ------------------ |
| Version | 8.2.1 |
| Type | FeatureRequest |
| TypeOfFailure | OtherFailure |
| Priority | low |
| Resolution | Unresolved |
| Component | Compiler (Linking) |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"bkp files cannot find TemplateHaskell symbols (even without Backpack features)","status":"New","operating_system":"","component":"Compiler (Linking)","related":[],"milestone":"","resolution":"Unresolved","owner":{"tag":"Unowned"},"version":"8.2.1","keywords":["backpack"],"differentials":[],"test_case":"","architecture":"","cc":[""],"type":"FeatureRequest","description":"{{{\r\n{-# LANGUAGE TemplateHaskell #-}\r\nunit th where\r\n module TH where\r\n f = [d| x = True |]\r\nunit p where\r\n dependency th\r\n module B where\r\n import TH\r\n $(f)\r\n}}}\r\n\r\nWhen run, this gives:\r\n\r\n{{{\r\nghc: ^^ Could not load 'th_TH_f_closure', dependency unresolved. See top entry above.\r\n\r\n\r\nByteCodeLink.lookupCE\r\nDuring interactive linking, GHCi couldn't find the following symbol:\r\n th_TH_f_closure\r\nThis may be due to you not asking GHCi to load extra object files,\r\narchives or DLLs needed by your current session. Restart GHCi, specifying\r\nthe missing library using the -L/path/to/object/dir and -lmissinglibname\r\nflags, or simply by naming the relevant files on the GHCi command line.\r\nAlternatively, this link failure might indicate a bug in GHCi.\r\nIf you suspect the latter, please send a bug report to:\r\n glasgow-haskell-bugs@haskell.org\r\n}}}\r\n\r\nI am not sure how easy or hard the fix is. We should bump this priority up if we get more serious about supporting Template Haskell with Backpack.","type_of_failure":"OtherFailure","blocking":[]} -->https://gitlab.haskell.org/ghc/ghc/-/issues/14063Compiling with --backpack with undefined dependency results in "the 'impossib...2023-05-05T12:59:10ZrcookCompiling with --backpack with undefined dependency results in "the 'impossible' happened"Using GHC 8.2.1:
```
> ghc --version
The Glorious Glasgow Haskell Compilation System, version 8.2.1
```
And compiling with `--backpack` as follows:
```
> ghc --backpack foo.bkp
[1 of 4] Processing foo-indef
[1 of 2] Compiling Str[si...Using GHC 8.2.1:
```
> ghc --version
The Glorious Glasgow Haskell Compilation System, version 8.2.1
```
And compiling with `--backpack` as follows:
```
> ghc --backpack foo.bkp
[1 of 4] Processing foo-indef
[1 of 2] Compiling Str[sig] ( foo-indef\Str.hsig, nothing )
[2 of 2] Compiling Foo ( foo-indef\Foo.hs, nothing )
[2 of 4] Processing foo-string
Instantiating foo-string
[1 of 1] Compiling Str ( foo-string\Str.hs, foo-string\Str.o )
[3 of 4] Processing foo-int
Instantiating foo-int
[1 of 1] Compiling Str ( foo-int\Str.hs, foo-int\Str.o )
[4 of 4] Processing main
ghc.EXE: panic! (the 'impossible' happened)
(GHC version 8.2.1 for x86_64-unknown-mingw32):
no package name
CallStack (from HasCallStack):
error, called at compiler\backpack\DriverBkp.hs:573:32 in ghc:DriverBkp
Please report this as a GHC bug: http://www.haskell.org/ghc/reportabug
make: *** [foo-indef/Foo.hi] Error 1
```
See attached `foo.bkp` file. While the Backpack file *is* invalid, in that the `main` unit mentions the `foo` dependency, which does not exist, this shouldn't lead to a GHC panic.
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | ------------ |
| Version | 8.2.1 |
| Type | Bug |
| TypeOfFailure | OtherFailure |
| Priority | normal |
| Resolution | Unresolved |
| Component | Compiler |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"Compiling with --backpack with undefined dependency results in \"the 'impossible' happened\"","status":"New","operating_system":"","component":"Compiler","related":[],"milestone":"","resolution":"Unresolved","owner":{"tag":"Unowned"},"version":"8.2.1","keywords":["Backpack"],"differentials":[],"test_case":"","architecture":"","cc":[""],"type":"Bug","description":"Using GHC 8.2.1:\r\n\r\n{{{\r\n> ghc --version\r\nThe Glorious Glasgow Haskell Compilation System, version 8.2.1\r\n}}}\r\n\r\nAnd compiling with `--backpack` as follows:\r\n\r\n{{{\r\n> ghc --backpack foo.bkp\r\n[1 of 4] Processing foo-indef\r\n [1 of 2] Compiling Str[sig] ( foo-indef\\Str.hsig, nothing )\r\n [2 of 2] Compiling Foo ( foo-indef\\Foo.hs, nothing )\r\n[2 of 4] Processing foo-string\r\n Instantiating foo-string\r\n [1 of 1] Compiling Str ( foo-string\\Str.hs, foo-string\\Str.o )\r\n[3 of 4] Processing foo-int\r\n Instantiating foo-int\r\n [1 of 1] Compiling Str ( foo-int\\Str.hs, foo-int\\Str.o )\r\n[4 of 4] Processing main\r\nghc.EXE: panic! (the 'impossible' happened)\r\n (GHC version 8.2.1 for x86_64-unknown-mingw32):\r\n no package name\r\nCallStack (from HasCallStack):\r\n error, called at compiler\\backpack\\DriverBkp.hs:573:32 in ghc:DriverBkp\r\n\r\nPlease report this as a GHC bug: http://www.haskell.org/ghc/reportabug\r\n\r\nmake: *** [foo-indef/Foo.hi] Error 1\r\n}}}\r\n\r\nSee attached `foo.bkp` file. While the Backpack file ''is'' invalid, in that the `main` unit mentions the `foo` dependency, which does not exist, this shouldn't lead to a GHC panic.\r\n","type_of_failure":"OtherFailure","blocking":[]} -->Edward Z. YangEdward Z. Yanghttps://gitlab.haskell.org/ghc/ghc/-/issues/13765GHC cannot parse valid Haskell98 whose first identifier is named signature2019-07-07T18:20:11ZEdward Z. YangGHC cannot parse valid Haskell98 whose first identifier is named signatureThis no longer parses in GHC 8.2:
```
ezyang@sabre:~$ cat A.hs
signature = 2
ezyang@sabre:~$ ghc-8.0 -c A.hs
A.hs:1:1: error:
The IO action ‘main’ is not defined in module ‘Main’
ezyang@sabre:~$ ghc-head -c A.hs
A.hs:1:11: error:
...This no longer parses in GHC 8.2:
```
ezyang@sabre:~$ cat A.hs
signature = 2
ezyang@sabre:~$ ghc-8.0 -c A.hs
A.hs:1:1: error:
The IO action ‘main’ is not defined in module ‘Main’
ezyang@sabre:~$ ghc-head -c A.hs
A.hs:1:11: error:
parse error on input ‘=’
Perhaps you need a 'let' in a 'do' block?
e.g. 'let x = 5' instead of 'x = 5'
|
1 | signature =
```
The problem is that signature is only a keyword in the module header and not in the rest of the code, which means that there is a shift/reduce conflict when you have a `signature` at the top of the module.
This is fairly benign bug as you can only trigger it when you have no imports; nevertheless, we should fix it. One possibility is to have a completely different parser entry when you are parsing an `hsig` file so that `signature` never occurs in a module header when you parse a normal Haskell file. ISTR this was modestly more irritating to implement but perhaps it is still not too bad. (Note that the shift/reduce conflict still exists, but since you'll never declare a signature without a module header it is essentially irrelevant.)
I discovered this when mpickering pointed out that adding 'signature' to the module header increased the number of shift reduce conflicts. (https://github.com/haskell-suite/haskell-src-exts/pull/355\#issuecomment-304536290). This wasn't the case in GHC's parser but only accidentally: there is already a shift reduce conflict when you have a top-level Haddock doc block!
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | ----------------- |
| Version | 8.2.1-rc2 |
| Type | Bug |
| TypeOfFailure | OtherFailure |
| Priority | low |
| Resolution | Unresolved |
| Component | Compiler (Parser) |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"GHC cannot parse valid Haskell98 whose first identifier is named signature","status":"New","operating_system":"","component":"Compiler (Parser)","related":[],"milestone":"","resolution":"Unresolved","owner":{"tag":"Unowned"},"version":"8.2.1-rc2","keywords":[],"differentials":[],"test_case":"","architecture":"","cc":[""],"type":"Bug","description":"This no longer parses in GHC 8.2:\r\n\r\n{{{\r\nezyang@sabre:~$ cat A.hs\r\nsignature = 2\r\nezyang@sabre:~$ ghc-8.0 -c A.hs\r\n\r\nA.hs:1:1: error:\r\n The IO action ‘main’ is not defined in module ‘Main’\r\nezyang@sabre:~$ ghc-head -c A.hs\r\n\r\nA.hs:1:11: error:\r\n parse error on input ‘=’\r\n Perhaps you need a 'let' in a 'do' block?\r\n e.g. 'let x = 5' instead of 'x = 5'\r\n |\r\n1 | signature = \r\n}}}\r\n\r\nThe problem is that signature is only a keyword in the module header and not in the rest of the code, which means that there is a shift/reduce conflict when you have a `signature` at the top of the module.\r\n\r\nThis is fairly benign bug as you can only trigger it when you have no imports; nevertheless, we should fix it. One possibility is to have a completely different parser entry when you are parsing an `hsig` file so that `signature` never occurs in a module header when you parse a normal Haskell file. ISTR this was modestly more irritating to implement but perhaps it is still not too bad. (Note that the shift/reduce conflict still exists, but since you'll never declare a signature without a module header it is essentially irrelevant.)\r\n\r\nI discovered this when mpickering pointed out that adding 'signature' to the module header increased the number of shift reduce conflicts. (https://github.com/haskell-suite/haskell-src-exts/pull/355#issuecomment-304536290). This wasn't the case in GHC's parser but only accidentally: there is already a shift reduce conflict when you have a top-level Haddock doc block!","type_of_failure":"OtherFailure","blocking":[]} -->https://gitlab.haskell.org/ghc/ghc/-/issues/13493Recompilation avoidance and Backpack2020-08-11T13:37:30ZEdward Z. YangRecompilation avoidance and BackpackToday, recompilation avoidance is centered around two major mechanisms:
1. First, we keep track of entities we \*use\* (`tcg_dus`), which is done by reading off all external names from the renamed source code of a Haskell source file.
1...Today, recompilation avoidance is centered around two major mechanisms:
1. First, we keep track of entities we \*use\* (`tcg_dus`), which is done by reading off all external names from the renamed source code of a Haskell source file.
1. Second, we keep track of what we \*import\* (`tcg_imports`), which tracked when we rename imports.
These two pieces of information get assembled into a module-indexed series of usages in `mk_mod_usage_info`. The general idea is that when an entity is used, we must record the hash of the entity; when a module is imported, we must record its export hash.
There is an implicit assumption here, which is that a (direct) import is the only way we depend on the exports of a module, and an occurrence of a name in the renamed syntax is the only way we depend on an actual entity.
Backpack breaks these assumptions:
- When we perform signature merging, we depend on the exports and entities of each of the signatures we merge in. Furthermore, it is important to distinguish each of these by identity module (not semantic module, which collapses the distinction.)
- When we instantiate a module, we depend on the exports and entities of the implementing module.
When I initially implemented Backpack, I slowly added extra information to fix recompilation problems as I noticed them. I thus accreted the following recompilation avoidance mechanisms:
- When signature merging occurs, we specially record the module hash for each used merge requirement as a special new field `UsageMergedRequirement`, and recomp if the module hash changed at all. We also add each merged signature to ImportAvails (but not as an "import") to ensure we pick up family instances.
- When we instantiate a module, we treat it as if we had a direct import of it (not yet merged, in https://phabricator.haskell.org/D3381). Since instantiations are always referencing non-local modules, we'll always record a module hash in such cases.
This is quite a hodgepodge, and I have no confidence that it is correct. For example, if an implementing module reexports an entity from another module, and that original entity changes, I doubt we recompile at this point. We "accidentally" handle the case when it's not a reexport because we record the module hash for the entire instantiating module.
It seems that it would be better if we can recast this in terms of imports and usages. Here is a try at the design:
- In both instantiation and merging, we must record the export hash of the modules we instantiated/merged in. It is a little troublesome to think of these as imports, however, because they're not (and if you try to implement this, you find yourself making a fake ImportedModVal for an import that doesn't exist); I think the correct thing here is to introduce a new notion of dependency for things that don't correspond to source level imports (another possibility is to add another constructor to ImportedModVal but the effect of this on existing code would have to be determined.)
- The usages when we instantiate a signature are the (instantiated) usages of the original signature (in particular, this picks up the usages from instance lookup), plus a usage for each entity that we match against (because we must rematch if the type changes.)
- Usages for signature merging are a little trickier. We want a usage for every entity that we end up merging in (so, we must record usages post thinning), BUT we must make sure the usage points at the identity module of the signature that originally provided it, not the semantic module (which will invariably point to the current module under compilation.)
One more thing: when we instantiate a module on-the-fly, we need to account for how we instantiated it (to put it differently, the recompilation information we compute when we do on-the-fly should be the (morally) the same as what we would get if we actually compiled the modules in question. This is a bit troublesome since we don't have detailed information relating how a signature was instantiated and what we used (the on-the-fly instantiation process shortcuts this). The simplest thing is probably to just record the module hashes of each module that was used to instantiate an imported module (recursively); we might be able to do this even by just twiddling `mi_mod_hash` hash when we instantiate (the alternative is to switch to recording InstalledModule/InstalledUnitId only in hashes, and augmenting usage information to also carry along instantiations.)
Another problem is that we record usages for Module (instantiated things), but hashes are actually on an InstalledModule basis.
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | ------------ |
| Version | 8.1 |
| Type | Bug |
| TypeOfFailure | OtherFailure |
| Priority | normal |
| Resolution | Unresolved |
| Component | Compiler |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"Recompilation avoidance and Backpack","status":"New","operating_system":"","component":"Compiler","related":[],"milestone":"","resolution":"Unresolved","owner":{"tag":"Unowned"},"version":"8.1","keywords":["backpack","recomp"],"differentials":[],"test_case":"","architecture":"","cc":[""],"type":"Bug","description":"Today, recompilation avoidance is centered around two major mechanisms:\r\n\r\n1. First, we keep track of entities we *use* (`tcg_dus`), which is done by reading off all external names from the renamed source code of a Haskell source file.\r\n\r\n2. Second, we keep track of what we *import* (`tcg_imports`), which tracked when we rename imports.\r\n\r\nThese two pieces of information get assembled into a module-indexed series of usages in `mk_mod_usage_info`. The general idea is that when an entity is used, we must record the hash of the entity; when a module is imported, we must record its export hash.\r\n\r\nThere is an implicit assumption here, which is that a (direct) import is the only way we depend on the exports of a module, and an occurrence of a name in the renamed syntax is the only way we depend on an actual entity.\r\n\r\nBackpack breaks these assumptions:\r\n\r\n* When we perform signature merging, we depend on the exports and entities of each of the signatures we merge in. Furthermore, it is important to distinguish each of these by identity module (not semantic module, which collapses the distinction.)\r\n\r\n* When we instantiate a module, we depend on the exports and entities of the implementing module.\r\n\r\nWhen I initially implemented Backpack, I slowly added extra information to fix recompilation problems as I noticed them. I thus accreted the following recompilation avoidance mechanisms:\r\n\r\n* When signature merging occurs, we specially record the module hash for each used merge requirement as a special new field `UsageMergedRequirement`, and recomp if the module hash changed at all. We also add each merged signature to ImportAvails (but not as an \"import\") to ensure we pick up family instances.\r\n\r\n* When we instantiate a module, we treat it as if we had a direct import of it (not yet merged, in https://phabricator.haskell.org/D3381). Since instantiations are always referencing non-local modules, we'll always record a module hash in such cases.\r\n\r\nThis is quite a hodgepodge, and I have no confidence that it is correct. For example, if an implementing module reexports an entity from another module, and that original entity changes, I doubt we recompile at this point. We \"accidentally\" handle the case when it's not a reexport because we record the module hash for the entire instantiating module.\r\n\r\nIt seems that it would be better if we can recast this in terms of imports and usages. Here is a try at the design:\r\n\r\n* In both instantiation and merging, we must record the export hash of the modules we instantiated/merged in. It is a little troublesome to think of these as imports, however, because they're not (and if you try to implement this, you find yourself making a fake ImportedModVal for an import that doesn't exist); I think the correct thing here is to introduce a new notion of dependency for things that don't correspond to source level imports (another possibility is to add another constructor to ImportedModVal but the effect of this on existing code would have to be determined.)\r\n\r\n* The usages when we instantiate a signature are the (instantiated) usages of the original signature (in particular, this picks up the usages from instance lookup), plus a usage for each entity that we match against (because we must rematch if the type changes.)\r\n\r\n* Usages for signature merging are a little trickier. We want a usage for every entity that we end up merging in (so, we must record usages post thinning), BUT we must make sure the usage points at the identity module of the signature that originally provided it, not the semantic module (which will invariably point to the current module under compilation.)\r\n\r\nOne more thing: when we instantiate a module on-the-fly, we need to account for how we instantiated it (to put it differently, the recompilation information we compute when we do on-the-fly should be the (morally) the same as what we would get if we actually compiled the modules in question. This is a bit troublesome since we don't have detailed information relating how a signature was instantiated and what we used (the on-the-fly instantiation process shortcuts this). The simplest thing is probably to just record the module hashes of each module that was used to instantiate an imported module (recursively); we might be able to do this even by just twiddling `mi_mod_hash` hash when we instantiate (the alternative is to switch to recording InstalledModule/InstalledUnitId only in hashes, and augmenting usage information to also carry along instantiations.)\r\n\r\nAnother problem is that we record usages for Module (instantiated things), but hashes are actually on an InstalledModule basis.","type_of_failure":"OtherFailure","blocking":[]} -->https://gitlab.haskell.org/ghc/ghc/-/issues/13469-fdefer-type-errors for Backpack2019-07-07T18:21:44ZEdward Z. Yang-fdefer-type-errors for BackpackIt would be great if `-fdefer-type-errors` worked with Backpack. There are two senses in which it could work:
1. When matching signatures, if the instantiating module didn't provide a required function, we could just create one out of t...It would be great if `-fdefer-type-errors` worked with Backpack. There are two senses in which it could work:
1. When matching signatures, if the instantiating module didn't provide a required function, we could just create one out of thin air so we can continue building.
1. When matching signature, if the implementation doesn't match the signature, we instead create a deferred type error.
(2) is more difficult because it basically requires us to solve #12703.
<details><summary>Trac metadata</summary>
| Trac field | Value |
| ---------------------- | -------------- |
| Version | 8.1 |
| Type | FeatureRequest |
| TypeOfFailure | OtherFailure |
| Priority | normal |
| Resolution | Unresolved |
| Component | Compiler |
| Test case | |
| Differential revisions | |
| BlockedBy | |
| Related | |
| Blocking | |
| CC | |
| Operating system | |
| Architecture | |
</details>
<!-- {"blocked_by":[],"summary":"-fdefer-type-errors for Backpack","status":"New","operating_system":"","component":"Compiler","related":[],"milestone":"","resolution":"Unresolved","owner":{"tag":"Unowned"},"version":"8.1","keywords":["backpack"],"differentials":[],"test_case":"","architecture":"","cc":[""],"type":"FeatureRequest","description":"It would be great if `-fdefer-type-errors` worked with Backpack. There are two senses in which it could work:\r\n\r\n1. When matching signatures, if the instantiating module didn't provide a required function, we could just create one out of thin air so we can continue building.\r\n\r\n2. When matching signature, if the implementation doesn't match the signature, we instead create a deferred type error.\r\n\r\n(2) is more difficult because it basically requires us to solve #12703.","type_of_failure":"OtherFailure","blocking":[]} -->