2

I'm trying to implement Python-like white-space indentation(read as: emit indent/dedent tokens where needed) with FsLexYacc.

It seems like FsLexYacc is not able to use unput which is what the C/C++ examples for lexing white-space based indentation use. I tried to use an additional argument as "indent stack" during lexing but not being able to return more than one token per lex rule made it impossible to return all pending dedents at the end of the file or multiple dedents when needed inbetween.

Is there a way to implement white-space based indentation in FsLexYacc without the need to tokenize the complete string first and applying a separate pass over all tokens to replace the whitespaces with indents/dedents where appropriate? (even this possible solution seems hard to get to work with a (LexBuffer<char> -> token) signature, to be able to pass it into the generated Parser)

prydain
  • 365
  • 3
  • 11
  • 1
    Off-the-cuff idea, but instead of defining `Indent` and `Dedent` tokens, could you perhaps define `Indent of levels:int` and `Dedent of levels:int` tokens instead? Then you could return a single token that represents indenting or dedenting by any number of indentation levels. – rmunn Nov 07 '16 at 11:07
  • @munn How would I use the different Indent levels in the parser? Is it possible to read a Indent level 1 token and put back a Indent n-1? That would be needed for recursive rules. – prydain Nov 10 '16 at 20:55
  • @prydain: Did you ever find a solution using ocamllex -- I am having the same problem now -- considering my own unput by modifying the lexbuf but that seems too hacky. – Adam Van Prooyen Jan 24 '19 at 23:06

0 Answers0