Skip to content
Snippets Groups Projects
Commit 36104d7a authored by Jedai's avatar Jedai
Browse files

RichTokenStream support

This patch adds support for raw token streams, that contain more
information than normal token streams (they contains comments at
least). The "lexTokenStream" function brings this support to the
Lexer module. In addition to that, functions have been added to
the GHC module to make easier to recover of the token stream of 
a module ("getTokenStream").

Building on that, I added what could be called "rich token
stream": token stream to which have been added the source string
corresponding to each token, the function addSourceToToken takes
a StringBuffer and a starting SrcLoc and a token stream and build
this rich token stream. getRichTokenStream is a convenience
function to get a module rich token stream. "showRichTokenStream"
use the SrcLoc information in such a token stream to get a string
similar to the original source (except unsignificant
whitespaces). Thus "putStrLn . showRichTokenStream =<<
getRichTokenStream s mod" should print a valid module source, the
interesting part being to modify the token stream between the get
and the show of course.
parent c9bf1a2c
No related branches found
No related tags found
No related merge requests found
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment