Antlr: Unintended behavior

Question

Why this simple grammar

grammar Test;

expr
    :    Int | expr '+' expr;

Int
:    [0-9]+;

doesn't match the input 1+1 ? It says "No method for rule expr or it has arguments" but in my opition it should be matched.

score 3 · Accepted Answer · edited May 23 '17 at 12:17

3

It looks like I haven't used ANTLR for a while... ANTLRv3 did not support left-recursive rules, but ANTLRv4 does support immediate left recursion. It also supports the regex-like character class syntax you used in your post. I tested this version and it works in ANTLRWorks2 (running on ANTLR4):

grammar Test;

start : expr
      ;
expr  : expr '+' expr
      | INT
      ;
INT   : [0-9]+
      ;

If you add the start rule then ANTLR is able to infer that EOF goes at the end of that rule. It doesn't seem to be able to infer EOF for more complex rules like expr and expr2 since they're recursive...

There are a lot of comments below, so here is (co-author of ANTLR4) Sam Harwell's response (emphasis added):

You still want to include an explicit EOF in the start rule. The problem the OP faced with using expr directly is ANTLR 4 internally rewrote it to be expr[int _p] (it does so for all left recursive rules), and the included TestRig is not able to directly execute rules with parameters. Adding a start rule resolves the problem because TestRig is able to execute that rule. :)

I've posted a follow-up question with regard to EOF: When is EOF needed in ANTLR 4?

edited May 23 '17 at 12:17

Community

1
1

answered Jul 24 '13 at 17:16

DaoWen

32,589
6
74
101

But why didn't compiler marked as an error and why [here](https://github.com/antlr/grammars-v4/blob/master/java/Java.g4) on line 509 `expression` is defined as left-recursive and it doesn't seem to be a problem – sve Jul 24 '13 at 17:24
@IvanPetkov - There were more problems than I thought. I've edited my answer and tested the grammar in ANTLRWorks. It works now. – DaoWen Jul 24 '13 at 17:27
Thanks, now's a little bit clearer but about [this](https://github.com/antlr/grammars-v4/blob/master/java/Java.g4). `expression` is defined as left-recursive. – sve Jul 24 '13 at 17:37
@IvanPetkov - I did a bit of research, and it looks like they added support for _direct_ left recursion in ANTLRv4. I added a link in my answer to a page on the ANTLR website that discusses this change. – DaoWen Jul 24 '13 at 17:39
And still I can't answer my first question. Why `1+1` is not being matched? Does not make sense. – sve Jul 24 '13 at 17:42
1

@IvanPetkov - OK, tested again and now it actually parses `1+1`. Lack of an `EOF` was making it not accept the whole input before. – DaoWen Jul 24 '13 at 17:58
You mean that after changing the rule to `expr: INT | expr '+' expr EOF;` it should be fine? – sve Jul 24 '13 at 18:11
@IvanPetkov - Yes, I think that would work if you want to match `1+1`, but not for `1+2+3`. I've updated my answer again with a left-recursive version I tested in ANTLRWorks2 (I was using 1.5 before). – DaoWen Jul 24 '13 at 18:12
Ok, and why if we have only the rules `expr: INT | expr '+' expr EOF; INT: '0'..'9'+;` `1+1` is not being matched? – sve Jul 24 '13 at 18:23
@IvanPetkov - I have no idea... `expr : INT '+' expr | INT ; INT : [0-9]+;` Also seems to work—but that still doesn't answer your question. – DaoWen Jul 24 '13 at 18:37
2

You still want to include an explicit `EOF` in the `start` rule. The problem the OP faced with using `expr` directly is ANTLR 4 internally rewrote it to be `expr[int _p]` (it does so for all left recursive rules), and the included TestRig is not able to directly execute rules with parameters. Adding a `start` rule resolves the problem because TestRig is able to execute that rule. :) – Sam Harwell Jul 24 '13 at 19:57
I deleted the first half of this answer because it was almost entirely incorrect, and the parts which were correct weren't relevant to the OP's question. – Sam Harwell Jul 24 '13 at 20:00
1

Also, `Int` and `INT` are equally valid as lexer rules in ANTLR (versions 3 and 4). The only reason to favor the form `INT` is it produces constants in code which follow Java conventions for constants (all caps). – Sam Harwell Jul 24 '13 at 20:02
@280Z28 - Thanks for the edits. I'm glad one of the authors came along to clarify this, because as you pointed out my initial answer was all ANTLR3-specific and not really applicable. I'm still confused about `EOF` though, so I posted [a separate question on the subject](http://stackoverflow.com/questions/17844248/when-is-eof-needed-in-antlr-4). – DaoWen Jul 24 '13 at 20:32

score 1 · Answer 2 · answered Jan 04 '15 at 06:50

If your command looks like this:

grun MYGRAMMAR xxx -tokens

And this exception is thrown:

No method for rule xxx or it has arguments

Then this exception will get thrown with the rule you specified in the command above. It means the rule probably doesn't exist.

System.err.println("No method for rule "+startRuleName+" or it has arguments");

So startRuleName here, should print xxx if it's not the first (start) rule in the grammar. Put xxx as the first rule in your grammar to prevent this.

Antlr: Unintended behavior

2 Answers2