Parse EBNF with Megaparsec nested sepBy

Question

As an exercise I try to parse a EBNF/ABNF grammar with Megaparsec. I got trivial stuff like terminals and optionals working, but I'm struggling with alternatives. With this grammar:

S ::= 'hello' ['world'] IDENTIFIER LITERAL | 'test';

And this code:

production :: Parser Production
production = sepBy1 alternativeTerm (char '|') >>= return . Production

alternativeTerm :: Parser AlternativeTerm
alternativeTerm = sepBy1 term space >>= return . AlternativeTerm

term :: Parser Term
term = terminal
    <|> optional
    <|> identifier
    <|> literal

I get this error:

unexpected '|'
expecting "IDENTIFIER", "LITERAL", ''', '[', or white space

I guess the alternativeTerm parser is not returning to the production parser when it encounters a sequence that it cannot parse and throws an error instead.

What can I do about this? Change my ADT of an EBNF or should I somehow flatten the parsing. But then again, how can I do so?

Does it work how you want it if you use `(try alternativeTerm)` in `production`? Just a guess, since it seems like you need backtracking. — Lazersmoke, May 23 '17 at 12:14
It seems like the parsing does not escape the inner `sepBy1`. It tries to parse the | as `term` and fails. What about using `sepEndBy1` instead? — Phil Kiener, May 23 '17 at 13:33
The one in `alternativeTerm`. This only works when there is a guaranteed space between the last term and the |, though. — Phil Kiener, May 23 '17 at 14:02

Phil Kiener · Accepted Answer · 2017-05-23T20:16:38.530

1

It's probably best to expand my previous comment into a full answer.

Your grammar is basically a list of list of terms seperated (and ended) by whitespace, which in turn is seperated by |. Your solution with sepBy1 does not work because there is a trailing whitespace after LITERAL - sepBy1 assumes there is another term following that whitespace and tries to apply term to the |, which fails.

If your alternativeTerm is guaranteed to end with a whitespace character (or multiple), rewrite your alternativeTerm as follows:

alternativeTerm = (term `sepEndBy1` space) >>= return . AlternativeTerm

edited May 23 '17 at 20:16

answered May 23 '17 at 14:41

Phil Kiener

778
1
7
15

Thanks, that works now. I will most likely add a tokenizer, so I dont have to deal with spaces any more – Jan van Brügge May 24 '17 at 07:07

Parse EBNF with Megaparsec nested sepBy

1 Answers1