ghc-iserv should support being linked against a different template-haskell version to GHC
In !12306 (closed), I'm working on making it possible to build GHC such that the stage1 compiler gets built against the boot compiler's version of template-haskell
.
We get the following scenario:
Stage | GHC version | GHC linked against template-haskell | GHC produces splices for template-haskell | GHC (internally) runs splices for template-haskell |
---|---|---|---|---|
stage0 | X | Y | Y | Y |
stage1 | X+1 | Y | Y+1 | impossible (we would need to run splices built with stage0 against template-haskell-Y) |
stage2 | X+1 | Y+1 | Y+1 | Y+1 |
We would like to be able to run template haskell splices as part of the bootstrapping process (#22069), but as the above table explains that's not possible.
If we wanted to run a splice while building stage2, it would need to be ABI compatible with stage1, which means that it would need to be built by stage0 using the same dependencies as stage1. This might be theoretically achievable, but would be quite complex.
An alternative approach is to use ghc-iserv
. Rather than trying to load the bytecode for the splice into the stage1 GHC process, we would load it into iserv
. This iserv
process would need to be built by stage1 and linked against template-haskell-(Y+1) (as that is the version produced by stage1). In the parlance of hadrian
, this is stage1:exe:ghc-iserv
. (Note: the iserv for stage1 is the stage2 iserv).
If you try this out on my branch, you will find that this links and runs the bytecode alright, but you run into deserialisation errors. Why is that? Recall that stage1 was built against template-haskell-Y, but produces splices for template-haskell-(Y+1). When we run a splice on the iserv
process, it correctly produces a TH AST for version (Y+1), it gets serialised, and sent over the pipe. But then, stage1 ghc cannot parse it as its looking for a template-haskell-Y AST. Just like it would if we were running the internal interpreter.
The issue here is that the interface between ghc and iserv amounts to a function that looks like Bytecode -> TH.Expr
. This fixes the version of template-haskell to be the same one as we compiled ghc against, which we use for the internal interpreter.
The solution is to adjust the interface. Rather than serialising TH ASTs, we should serialise ghc ASTs. So the function should look like Bytecode -> HsExpr ...
. In practice, rather than converting from TH ASTs to ghc ASTs on the ghc side, we should do it on the iserv side.
Then iserv could be compiled against template-haskell-(Y+1), and everything would work fine.
Currently these conversion functions are defined in GHC.ThToHs
. I'm proposing moving the actual conversion functions into a new package say ghc-th-ast-conversion
that sits between lib:ghc
and ghc-bin
. The conversion functions would be abstract in lib:ghc
and then the dependency is injected in ghc-bin
.
As a diagram:
flowchart
subgraph stage2
C2[ghc-th-ast-conversion]
E[iserv] --> C2
end
subgraph stage1
A[ghc-bin] --> C1
A --> B
B[ghc]
D2[template-haskell-Y+1]
C1 --> B
C2 --> D2
end
subgraph boot compiler's packages
D1[template-haskell-Y]
C1[ghc-th-ast-conversion] --> D1
C2 --> B
end
Edit: This doesn't quite work because ghc needs to depend on template-haskell anyway. We would need to follow (3) from #23536 (closed) to allow iserv to practically depend on two versions of tempalte-haskell.