Parsing Expression Grammars: detect next token?

Question

I'm beginning PEG's with PEG.js.

There's something I can't get my head around...I'm sure it's simple but it's giving me a headache trying to understand the concept...

Consider this two-rule grammar:

name
  = name:.* {return name.join("")}

put
  = "put " a:name " into " b:name "." {put(a,b)}

I want to be able to pass this parser "put foo into bar." and cause put("foo","bar") to evaluate.

But PEG.js gives me the error Expected " into " or any character but end of input found.

I think I could fix this problem if the regex for the name rule where more specific than .* but why does it have to be? Can't the parser be smart enough to look ahead and see that " into " is coming up, as well as the "." at the end?

How can I achieve what I'm looking for? Is this perhaps the difference between a "bottom-up" and "top-down" grammar?

Edit: The regex /put (.*) into (.*)/g works like I want -- if I pass it "put foo into bar", it gives me $1="foo" and $2="bar". I'm just asking if I can get this same functionality (taking the whole string into account before deciding where the token boundaries are) using PEGjs or Jison.

I don't have personal experience with peg.js, but the online documentation suggests that `.*` will greedily consume every character available to it. — 500 - Internal Server Error, Aug 04 '14 at 22:38
I'm really curious if someone can comment about this problem in relation to "bottom-up" vs "top-down" parsing. Would it make a difference? — themirror, Aug 05 '14 at 15:35

Dale Stanbrough · Answer 1 · 2014-08-05T13:29:33.053

1

I'm fairly sure that "themirror" is correct - the first rule will eat all the input. Try it without that rule.

Also you should have another rule to allow arbitrary spaces. I found this online...

_
  = [ \r\n\t]*

The underscore will match any number of whitespace characters. Then you can rewrite your rule as...

put
   = "put" _ a:name _ "into" _  b:name _  "." {put(a,b)}

edited Aug 05 '14 at 13:29

answered Aug 05 '14 at 11:30

Dale Stanbrough

423
3
11

score 0 · Answer 2 · answered Oct 11 '14 at 02:32

Sorry for my bad english.

The first rule .* try to read every character, so end of file is reached.

You can make name rule more specific (any char but not SPACE or DOT):

name = [^ .]+

This one works... But not for name with spaces inside.

put = "put " a:name " into " b:name "." {put(a,b);}

name = c:[^ .]+ {return c.join("");}

For Node.js

var PEG = require('pegjs');
var text = 'put foo into bar.';
var parser = PEG.buildParser('{function put(a,b){ console.log(a,b); }}put = "put " a:name " into " b:name "." {put(a,b);}\nname = c:[^ .]+ {return c.join("");}');
parser.parse(text);

I have no good solution for spaces in name but try something like this:

nameBeforeInto = (!" into ".)+

nameBeforeDot = [^.]+

put = "put " nameBeforeInto " into " nameBeforeDot "."

nameBeforeInto return a multi-dimension array

Fabrice

Parsing Expression Grammars: detect next token?

2 Answers2