Cannot figure out why I'm getting error when parsing a string in Dart/Flutter with petitparser: uppercase letter expected at 1:1

Question

UPDATE 1: It looks like changing digit().plus() to word().plus() works. Does that seem right?

Petitparser has proven to be very powerful! I have the following flutter/dart code:

  testString = '((S|71|L || S|70|L || S|69|L || S|72|L) && (F|54|L || F|52|L || H|5|L))';
  final builder = ExpressionBuilder();
  final primitive = (uppercase() & char('|') & digit().plus() & char('|') & uppercase()).flatten().trim();
  builder.group().primitive(primitive);
  builder.group()
      .wrapper(char('(').trim(), char(')').trim(), (l, v, r) => v);
  builder.group()
    ..left(string('&&').trim(), (a, op, b) => ['&&', a, b])
    ..left(string('||').trim(), (a, op, b) => ['||', a, b]);
  final parser = builder.build().end();
  final result = parser.parse(testString);

This works perfectly. However, when I change the testString slightly to the following (note the word test instead of the number 5), I get the error in the title uppercase letter expected at 1:1:

testString = '((S|71|L || S|70|L || S|69|L || S|72|L) && (F|54|L || F|52|L || H|test|L))';

I can't seem to figure out what it thinks should be an uppercase letter, or how to change the parser. I was thinking maybe it is something with the digit().plus(), since that's the part of my primitive that I'm changing, but I tried changing that to any().plus() and it shows the same error.

Can anyone help? Thanks!

Answer 1

digit() parses any digit between 0 and 9 (like \d or [0-9] in regexp; word() parses any lowercase, uppercase, digit or the underscore character (like \w or [0-9a-zA-Z_] in regexp); and any() parses all characters (like . in regexp). Which one you pick, depends on the input you want to accept.

The reason H|test|L with the digit() in the primitive reports an error uppercase letter expected at 1:1 is the following: Since the parser cannot accept H|test|L it backtracks to surrounding ||-expression and tries to complete (F|54|L... with the parser of the next highest precedence, the primitive parser. This does not work either, so it backtracks to the surrounding &&-expression and tries to complete ((S|71|L... with the primitive parser. Since all possibilities have been exhausted it reports the last error, i.e. that it cannot parse the primitive at the beginning of the input.

Some notes:

1. PetitParser comes with debugging tools in import 'package:petitparser/debug.dart'; that can show you exactly what is happening given a specific input. For example, with trace(parser).parse('(F|52|L || H|test|L)'); you see how the parser is progressing through the input and what intermediate errors it encounters to finally produce the result.

2. The error reporting of your grammar is not ideal. By providing an error message as an argument to the flatten parser you can improve the readability of your error messages: .flatten('Invalid token'). Not that this solves the root cause of the problem, but it makes the output at least a bit more readable.

3. The behavior of the ExpressionBuilder to report the last seen error is not ideal. I've reported a bug for this: https://github.com/petitparser/dart-petitparser/issues/143. In the meantime I unfortunately don't see a way to customize the error reporting when using the ExpressionBuilder.