Python ANTLR4 example - Parser doesn't seem to parse correctly

Question

To demonstrate the problem, I'm going to create a simple grammar to merely detect Python-like variables.

I create a virtual environment and install antlr4-python3-runtime in it, as mentioned in "Where can I get the runtime?":

Then, I create a PyVar.g4 file with the following content:

grammar         PyVar;

program:        IDENTIFIER+;

IDENTIFIER:     [a-zA-Z_][a-zA-Z0-9_]*;

NEWLINE:        '\n' | '\r\n';

WHITESPACE:     [ ]+ -> skip;

Now if I test the grammar with grun, I can see that the grammar detects the variables just fine:

Now I'm trying to write a parser in Python to do just that. I generate the Lexer and Parser, using this command:

antlr4 -Dlanguage=Python3 PyVar.g4

And they're generated with no errors:

But when I use the example provided in "How do I run the generated lexer and/or parser?", I get no output:

What am I not doing right?

score 1 · Accepted Answer · answered Dec 25 '19 at 01:11

There are two problems here.

1. The grammar:

In the line where I had,

program:        IDENTIFIER+;

the parser will only detect one or more variables, and it will not detect any newline. The output you see when running grun is the output created by the lexer, that's why newlines are present in the tokens. So I had to replace it with something like this, for the parser to detect newlines.

program:        (IDENTIFIER | NEWLINE)+;

2. Printing the output of parser

In PyVar.py file, I created a tree with this line:

tree = parser.program()

But it didn't print its output, nor did I know how to, but the OP's comment on this accepted answer suggests using tree.toStringTree().

Now if we fix those, we can see that it works:

Python ANTLR4 example - Parser doesn't seem to parse correctly

1 Answers1