This ticket describes the plan and follows the implementation of separating the Haskell AST and parsing stage to packages `haskell-syntax` and `haskell-parser` to achieve modularity and code reuse across clients We propose the following goals to reach this package separation: 1. A module hierarchy `Language.Haskell.Syntax` which contains the base Haskell AST. Just the data types, very few functions, since there is virtually nothing you can do without knowing _any_ of the type family instances. 2. A module hierarchy `Language.Haskell.Parser` that defines `HsExpr Parsed`, and has the properties enumerated below. Presumably this package buys into the full `Anno` infrastructure to decorate the parse tree. 3. Rejig GHC to use `HsExpr Parsed` rather than `HsExpr (GhcPass Parsed)`. 4. Move the first two from `ghc` to an `ghc-haskell-syntax` package. 5. Figure out what needs to be done to create a `ghc-haskell-parser` package, later. --- Properties: 1. A (pre-processed) string can be parsed into `HsExpr Parsed`. Let's call this operation `parse`. (By "pre-processed", we mean that there is no CPP left in the input string.) 2. `HsExpr Parsed` can be pretty-printed to a string. Let's call this operation `print`. 3. `parse` and `print` are exact inverses, in both directions. 4. `HsExpr Parsed` is a suitable data structure to use as the input to a compilation pipeline. It is structured and recursive. 5. `HsExpr Parsed` depends on a minimum of infrastructure. In particular, no aspects of e.g. type-checking or code generation should be depended on by any module defining the `HsExpr` type, nor the `parse` or `print` functions. 6. `HsExpr` is extensible, fitting with the TTG framework for extension in other phases; the unextended `HsExpr` contains the essence of the Haskell AST, with the parts that are useful to many syntax processors. --- For (1) we need to remove all dependencies of GHC from all L.H.S modules. This is well underway (move HsModule) https://gitlab.haskell.org/ghc/ghc/-/merge_requests/8228 (move MatchGroup Origin field) https://gitlab.haskell.org/ghc/ghc/-/merge_requests/8301 (TTG for splices) https://gitlab.haskell.org/ghc/ghc/-/merge_requests/7821 (removing other dependencies) https://gitlab.haskell.org/ghc/ghc/-/merge_requests/8308 (dealing with FastString) https://gitlab.haskell.org/ghc/ghc/-/issues/21628. We need to keep reviewing these and enforcing a criteria: the base AST should represent the complexity of the source code, but doesn't need to keep e.g. annotations, that's a Parser extension to the AST. For (2) we should start moving the parser modules to `Language.Haskell.Parser`. I don't think we should do this yet, (1) must be complete first For (3) we need (2) For (4), we can have it before (3), but need (1,2). Let's keep tuning this description with the plan. --- For history, here is the original ticket description: This issue follows a previous discussion in a TTG related MR. I want to further this discussion, in particular because of some design decisions concerning my MR https://gitlab.haskell.org/ghc/ghc/-/merge_requests/8228 (see below) Quoting @rae in https://gitlab.haskell.org/ghc/ghc/-/merge_requests/4782#note_415958: > I want to add a small new thought: What if the result of \> parsing weren't GhcPs, but Parsed. That is, we have a client-independent phase indicator Parsed :: Type that lives in \> \> the client-independent space. GhcPass would just encompass GhcRn and GhcTc. For me, the fact that you can parse into \> \> and pretty-print from an AST is its fundamental property, and a great part to start any design discussion. > One challenge is that, to really have the parse/pretty-print property, you need information about whitespace, comments, \> keyword spellings, unicode syntax, and such. This information is mostly useless after parsing. Since our TTG structure offers no way to remove fields, putting this information in the base tree might make it less usable downstream. That \> \> might be why we use extension points for this data today. > So maybe I advocate for a compromise position: put Parser as a general, client-independent pass. All the source annotation stuff would also live in the client-independent space. But design the AST so that there are extensions for Parser, around annotations it is unlikely for clients to need. It would be a judgment call around what information is likely useless to clients (for example, one could argue that the distinction between (x `f`) and f x is "useless"), but I think we are likely to make mostly good judgments around this. I'm currently working in moving HsModule to L.H.S in https://gitlab.haskell.org/ghc/ghc/-/merge_requests/8228, and when dealing with GHC.Hs.ImpExp, I arrive at this: ```hs -- | Imported or exported entity. data IE pass = IEVar (XIEVar pass) (LIEWrappedName pass (IdP pass)) ... | IEGroup (XIEGroup pass) Int (LHsDoc pass) -- ^ Doc section heading | IEDoc (XIEDoc pass) (LHsDoc pass) -- ^ Some documentation | IEDocNamed (XIEDocNamed pass) String -- ^ Reference to named doc ``` The first reason why I brought this discussion up again is because of `HsDoc`. I was trying to understand whether `IEGroup` and `IEDoc` should be moved to the GHC specific part, or whether `HsDoc` should be moved to the client independent stage. In light of the comment advocating the independent `Parsed` stage which is also independent from the independent base AST, I would say that `HsDoc` would be a good example of something to be put in the independent `Parsed` stage, but not in the base AST. For now I'll move the constructors depending on `HsDoc` to the GHC specific part, and if we later on pick up the `Parsed` idea, we can move them to there. I mixed a bit of https://gitlab.haskell.org/ghc/ghc/-/merge_requests/8228 talk here, but to motivate further discussion. For comments regarding this specific TTG Module x HsDoc discussion use the linked MR Thank you, \~romes