Plugin-Swappable Parser
I originally started writing this as a GHC proposal, but apparently Plugin-API changes do not need one. This is a work-in-progress. Feedback welcome
Update 1: Added 2 parameters to the function in ReplacementParser
.
Plugin-Swappable Parser
For many people in the community, GHC plugins are a quick and easy way to extend the compiler with additional features or to prototype implementations. While plugins allow their author to transform or analyze a module as well as changing the (type) constraint solver, it is currently not possible to change the static syntax accepted by GHC.
I propose to make the parser implementation replaceable using a GHC plugin. This way, a plugin author can not only change the semantics, but also the accepted syntax for a module.
Motivation
At the moment, the only way to use different syntax in a Haskell module is to use quasiquotation. However, this leads to some ugly code without any support from IDEs. In the case that someone just wants to alter GHC's accepted syntax in a tiny way, this is unsatisfactory. By replacing the parser using a plugin, most syntax highlighter will keep working even when a definition contains a new keyword. Other features of an IDE might also continue working, such as hovering over a definition to see its type. This is not possible if the full definition was created by a quasi quoter.
I can see different use cases for such a parser plugin.
- Using GHC to implement Haskell-adjacent languages. In my case, I am trying to implement Curry using GHC, a language that uses Haskell's syntax with a few modifications (new keyword for example). Maybe the people working on Mu (Haskell dialect from Standard Chartered) would be interested in this as well.
- Prototyping proposed, syntactic changes to GHC. Many GHC proposals introduce new syntax. I am not very familiar with the process of implementing a proposal, but I could imagine that it would be useful to prototype an implementation to test syntactic changes without re-building the compiler every time.
- Restricting the syntax accepted by GHC for teaching. Racket offers different languages specifically for teaching. These restrict the language to certain features, depending on the level, to make it easier to use for beginners. At the moment, Haskell can be overwhelming for beginner students. Having a way to restrict the language can help with that. Although the current plugin infrastructure might be enough to implement a purely restrictive change.
Proposed Change Specification
I propose to augment the Plugin
data type as follows:
data Plugin = Plugin {
-- keep all current constructors
, parserPlugin :: ParserPlugin
}
type ParserPlugin = [CommandLineOption] -> ParserSpecification
data ParserSpecification = DefaultParser -- ^ Do not replace the parser
| ReplacementParser (DynFlags -> RealSrcLoc -> StringBuffer -> ParseResult ParsedModule) -- ^ Replace the parser with the given function.
Here, DynFlags
, RealSrcLoc
, StringBuffer
and ParseResult
are currently used by GHC's parser and my proposal does not affect them.
We need all of these parameters to at least be able to use GHC's own parser in a plugin.
As long as none of the plugins active for a module uses the ReplacementParser
constructor, nothing special happens.
When more than one plugin specifies a ReplacementParser
, compilation is aborted with an error.
We can only replace the parser with a single definiton , so having two plugins specifying different parsers is a problem and we should abort.
In the case that exactly one plugin specifies a new parser to be used, the module is parsed according to the function given in the ReplacementParser
constructor.
Note that the new parser still has to use GHC's current AST representation for its output.
Thus, any plugin has to transform its own syntax to fit into the current AST.
When a GHC (error) message refers back to these transformed sections, e.g. in a type error, it should continue using the existing SrcLoc
(and friends) annotations.
Any plugin author can decide to give meaningful source locations and fix any error messages via different mechanisms, if desired.
Effect and Interactions
The effect of a replaceable parser is that it allows a plugin author to modify the syntax accepted by GHC. This increases the flexibility and usability of GHC as a compiler tool.
I am not involved in GHC development, so it is hard for me do imagine any interactions with this feature. However, I will extend this section with any interactions from the comments.
Costs and Drawbacks
I estimate that the cost of this change is relatively low, since it is only concerned with a small part of the compiler. The change is also backwards compatible and should have a low maintenance cost.
Drawbacks:
- Completely replacing the parser does not compose when using multiple parser plugins as mentioned before. Adopting this solution makes it harder to reach a composable solution without being backwards-incompatible. However, I do not believe that there will be a better solution in the foreseeable future.
- To be extended
Alternatives
I can imagine that a parser based on parser combinators (i.e. not using Happy) is easier to extend with a plugin. Thus, one alternative could be to replace the current parser implementation with an implementation based on parser combinators. At each syntax rule, the parser could then look at all active parser plugins and try any parsing combinators as an alternative. Such a solution would compose better with multiple plugins, but it would incur a significant implementation and maintenance burden for a small benefit. Additionally, the performance of the parser will probably suffer as well.
Unresolved Questions
- Should a plugin be allowed to change the syntax of the module header and pragmas at the top of a module? Often, a plugin is activated per-module using
OPTIONS_GHC
. Specifying a plugin that alters the parsing of the header in the header itself is pretty weird. - Should the GHC AST be adapted to accommodate plugin-parsed syntax by adding a constructor that behaves similarly as the
HsExpansion
data type? E.g.data HsPluginExpansion a = HsPluginExpansion String a ^ -- original source plus generated AST`
Implementation Plan
I could implement this, but would probably need some guidance.