How to read alternates in EBNF grammars

Question

I have an EBNF grammar that has a few rules with this pattern:

sequence ::=
    item
    | item extra* sequence

Is the above equivalent to the following?

sequence ::=
    item (extra* sequence)*

Edit

Due to some of you observing bugs or ambiguities in both sequences, I'll give a specific example. The SVG specification provides a grammar for path data. This grammar has several producers with this pattern:

lineto-argument-sequence:
    coordinate-pair
    | coordinate-pair comma-wsp? lineto-argument-sequence

Could the above be rewritten as the following?

lineto-argument-sequence:
    coordinate-pair (comma-wsp? lineto-argument-sequence)*

Do you want to know whether you can do that, or whether you should do that? The whole point in EBNF over BNF is to (among others) remove the need to use recursion for loop constructs. Having looked at the spec you linked to I can tell you it is highly ambiguous and will probably require a bunch of lookahead rules. I will amend my answer to take your edit into account. — Andre Artus, Jun 24 '10 at 01:34
Well, one thing I was just wondering was if there'd be a particular reason the SVG spec would use the first pattern style when the second style seems to be equivalent while also being shorter, or why they didn't use a recursionless version like what you proposed. — AJM, Jun 25 '10 at 00:41
You may find that these file formats have historically had hand written parsers. When a human being reads the grammar they see the recursion and say to themselves, "look a loop", but even the best parser generator is not as smart as a human. Like any programming language (and that is what the grammar is to the generator) the program does what it is told to do, not necessarily what you want it to do. — Andre Artus, Jun 25 '10 at 01:56
Many well known compilers were, at least at some point, rolled by hand. No generator can beat the speed and flexibility of a hand rolled parser. What a parser generator gives you is the ability to get up and running in a short time. There are some powerful generators out there that can handle all kinds of fluffy grammars, but they have performance penalties. If you want to design a language that parses like a bat out of hell then you try to limit backtracking and lookahead as much as possible. — Andre Artus, Jun 25 '10 at 02:12
The problem with the way you modified the original grammar (which was valid in BNF) is that you kept the recursive element which in BNF would mean "rince and repeat", and enclosed it in an [EBNF] optional loop, which means the same thing. If you are translating a grammar from BNF to EBNF then you try as much as you can to remove the recursive elements and empty productions that is inherent in BNF grammars. — Andre Artus, Jun 25 '10 at 02:38

Andre Artus · Accepted Answer · 2010-06-24T22:09:29.020

Not really, they seem to have different bugs. The first sequence is ambiguous around "item" seeing that "extra" is optional. You could rewrite it as the following to remove ambiguity:

sequence3 ::= 
    item extra* sequence3

The second one is ambigous around "extra", seeing as it is basically two nested loops both starting with "extra". You could rewrite it as the following to remove ambiguity:

sequence4 ::=
    item ((extra|item))*

Your first version will likely choke on an input sequence consisting of a single "item" (it depends on the parser implementation) because it won't disambiguate.

My rewrites assume you want to match a sequence starting with "item" and optionally followed by a series of (0 or more) "item" or "extra" in any order.

e.g.

item
item extra 
item extra item
item extra extra item
item item item item 
item item item item extra

etc.

Without additional information I would be personally inclined towards the option I labled "sequence4" as all the other options are merely using recursion as an expensive loop construct. If you are willing to give me more information I may be able to give a better answer.

EDIT: based on Jorn's excellent observation (with a small mod).

If you rewrite "sequence3" to remove recursion you get the following:

sequence5 ::= 
    (item extra*)+

It think this will be my prefered version, not "sequence4".

I have to point out that all three versions above are functionally equivalent (as recognizers or generators). The parse trees for 3 would be different to 4 and 5, but I cannot think that that would affect anything other than perhaps performance.

EDIT: Concerning the following:

lineto-argument-sequence:
    coordinate-pair
    | coordinate-pair comma-wsp? lineto-argument-sequence

What this production says is that a lineto-argument-sequence is composed of at least one coordinate-pair followed by zero or more coordinate-pairs seperated by optional white/comma. Any of the following would constitute a lineto-argument-sequence (read -> as 'becomes'):

1,2        -> (1, 2)
1.5.6      -> (1.5, 0.6)
1.5.06     -> (1.5, 0.06)
2 3 3 4    -> (2,3) (3,4)
2,3-3-4    -> (2,3) (-3,-4)
2 3 3      -> ERROR

So a coordinate-pair is really any 2 consecutive numbers.

I have mocked up a grammar in ANTLR that seems to work. Note the pattern used for lineto_argument_sequence is similar to the one Jorn and I recommended previously.

grammar SVG;

lineto_argument_sequence
    : coordinate_pair (COMMA_WSP? coordinate_pair)*
    ;

coordinate_pair
    : coordinate COMMA_WSP? coordinate
    ;

coordinate
    : NUMBER
    ;

COMMA_WSP
    : ( WS+|WS*','WS*) //{ $channel=HIDDEN; }
    ;

NUMBER
    : '-'? (INT | FLOAT) ;

fragment
INT
    : '0'..'9'+ ;

fragment
FLOAT
    : ('0'..'9')+ '.' ('0'..'9')* EXPONENT?
    | '.' ('0'..'9')+ EXPONENT?
    | ('0'..'9')+ EXPONENT
    ;

fragment
WS  : ' '  | '\t' | '\r' | '\n'  ;

fragment
EXPONENT
    : ('e'|'E') ('+'|'-')? ('0'..'9')+ ;

Given the following input:

2, 3 -3 -4 5.5.65.5.6

it produces this parse tree.

alt text http://www.freeimagehosting.net/uploads/85fc77bc3c.png

You may want to remove "{$channel=HIDDEN;}" from COMMA_WSP if it messes with other productions, I just put it in there because COMMA_WSP cluttered up the graph. — Andre Artus, Jun 24 '10 at 02:26
Oh, it gives the same result for input like "2,3-3-4 5.5.65.5.6" — Andre Artus, Jun 24 '10 at 02:28

score 2 · Answer 2 · answered Jun 20 '10 at 21:00

2

This rule would also be equivalent to sequence ::= (item extra*)*, thus removing the recursion on sequence.

answered Jun 20 '10 at 21:00

Jörn Horstmann

33,639
11
75
118

Good one, I missed that. The only change I would make is to change the last "*" to "+", otherwise it matches an empty string. – Andre Artus Jun 20 '10 at 22:29

score 0 · Answer 3 · answered Jun 20 '10 at 20:25

0

Yes, those two grammars describe the same language.

But is that really EBNF? Wikipedia article on EBNF does not include the Kleene star operator.

answered Jun 20 '10 at 20:25

aioobe

413,195
112
811
826

1

EBNF is more like a class/group of languages, than a single language. Wirth uses one notation, the ISO standard uses a different one. And every other book I have on the subject uses a slightly different notation. That is not even taking into account the differences in versions of BNF, and then tool specific changes to either. When I did the spec for my first compiler (an embarrasing bastard child of C and Pascal) I wrote the grammar in a horrible style with arrows, greek letters (e.g. epsilon for no match) and tail recursion for loops. I did the coding by hand as the generated code was a mess. – Andre Artus Jun 20 '10 at 22:47

How to read alternates in EBNF grammars

3 Answers3