Parser reducing too early

Question

I've got a grammar that basically looks like :

start ::= groups.
groups ::= groups group.
groups ::= group.
group(A) ::= IDENTIFIER identparams CURLY_OPEN assignments CURLY_CLOSE SEMICOLON.
group(A) ::= IDENTIFIER CURLY_OPEN assignments CURLY_CLOSE SEMICOLON.
assignments ::= assignments assignment.
assignment ::= IDENTIFIER ASSIGNMENT bool_expr SEMICOLON.

It parses something like :

name {
     name = "value";
     name2 = "value";
};

That's a named like config indeed. What happens is that :

name = "value" results in assignments ::= assignments assignment. beeing reduced. I would expect that assignments was a constant value, but that isn't the case :

P assignment(0x807e778) ::= IDENTIFIER(0x807e728) ASSIGNMENT mvalue SEMICOLON.

P assignments((nil)) ::= assignments((nil)) assignment(0x807e778).
P append 0x807e778 to 0x807e838
P mvalue ::= string.
P assignment(0x807e750) ::= IDENTIFIER(0x807e7c8) ASSIGNMENT mvalue SEMICOLON.
P assignments((nil)) ::= assignments(0x807e838) assignment(0x807e750).
P append 0x807e750 to 0x807e910
P mvalue ::= string.
P assignment(0x807e7f0) ::= IDENTIFIER(0x807e7a0) ASSIGNMENT mvalue SEMICOLON.
P assignments((nil)) ::= assignments(0x807e910) assignment(0x807e7f0).
P append 0x807e7f0 to 0x807e9e8
P group(0x807e7a0) assignments(0x807e9e8) : bind
P groups ::= group(0x807e7a0).

The lemon debug output is http://pastebin.com/yHNkNRpf

which results in name2 being only added to the list. I'm puzzled by this. I understand the reduce, but not why assignents keep being set to null. There are ways around this, but I prefer a proper fix.

The lemon debug output is http://pastebin.com/yHNkNRpf

Any clues ?

score 0 · Answer 1 · answered Jun 07 '13 at 14:32

Lemon parser debug output on pastebin shows that it parses your input tokens perfectly well. It allows to conclude that complete grammar is correct. What is unclear from the question is how you treat parsed values in reduce actions.

Consider the following implementation. Two-level list is introduced to store parsed data -- it is list of groups that are in their turn lists of assignments. On reduce of every assignment rule parser call an action that adds the entry to a temporary list of assignments. On reduce of group rules this temporary list is included into groups list as a new group entry. Entire list is dropped in case of syntax error. It contains all parsed values when parsing is successfully complete.

According to your program output, it seems that you use new list or every new assignment value:

P append 0x807e778 to 0x807e838
P append 0x807e750 to 0x807e910
P append 0x807e7f0 to 0x807e9e8

It explains why you have only the last value at the end of parsing. New list for assignments should be initialized at the beginning of parsing for the first group and in the action handler of groups rule for the next group.

score 0 · Accepted Answer · answered Jul 21 '16 at 10:53

0

Better late then never : The re2c lexer used didn't had all token paths covered. In other words : Some tokens didn't match a rule, and a default wasn't set. In that case, the behaviour is undefined. In this case, it returned the wrong token.

answered Jul 21 '16 at 10:53

Igmar Palsenberg

637
4
10

Parser reducing too early

2 Answers2