accessing regexp subtree in parsetree

Question

I have the following Rascal module:

module foo

import IO;
import ParseTree;
extend lang::std::Layout;

lexical CHAR = [ab];
start syntax CharList = CHAR hd (','  CHAR)+ tl ';';

My question is how to get to the individual elements of the tl part, after having parsed something. E.g.:

rascal>import foo;
ok

rascal>pt = parse(#start[CharList], "a, b;");
start[CharList]: `a, b;`
Tree: appl(...
rascal>pt.top.tl;
(',' CHAR)+: `, b`
Tree: appl(regular(...

Now how do I access the , b element? pt.top.tl[0] seems not to be the right way.

Thanks for any help.

Mark Hills · Accepted Answer · 2015-10-14T12:31:06.103

3

Probably the easiest thing to do, unless you need to define lists in the way you've done above with a separate head and tail, is to use Rascal's built-in separated list construct, like so:

start syntax CharList = {CHAR ","}+ chars ';';

(If you need the separate head and tail, please see Jurgen's answer below. You can use this same notation there as well.)

This defines a list of 1 or more (because of the +) comma-separated CHARs. If you need 0 or more you would instead use *. Rascal allows you to iterate over separated lists, so you can get the characters back like so:

rascal> chars = [c | c <- pt.top.chars ];

For the list in your example, this gives me back the following:

list[CHAR]: [appl(
    prod(
      lex("CHAR"),
      [\char-class([range(97,98)])],
      {}),
    [char(97)])[
    @loc=|unknown:///|(0,1,<1,0>,<1,1>)
  ],appl(
    prod(
      lex("CHAR"),
      [\char-class([range(97,98)])],
      {}),
    [char(98)])[
    @loc=|unknown:///|(3,1,<1,3>,<1,4>)
  ]]

You could also turn these into strings if you wanted to view them more easily or do something with their string values:

rascal>charsAsStrings = ["<c>" | c <- pt.top.chars ];
list[str]: ["a","b"]

edited Oct 14 '15 at 12:31

answered Oct 14 '15 at 12:11

Mark Hills

1,028
5
4

he! I took longer to answer than 12 minutes :-) sorry we crossed. – Jurgen Vinju Oct 14 '15 at 12:24
Thanks, Mark. I hadn't thought of using the comprehension as a way to collect things into a list. Also the notation `""` to turn those mysterious things into strings is very helpful! – Dennis Oct 14 '15 at 12:52
Afterthought: I find it slightly scary to enumerate over something that I don't quite understand the structure of, namely the parsetree. Rascal parsetrees are on the one hand quite complex, and the language seems to make an attempt at hiding this complexity from the programmer; but I'm not (yet?) convinced that it succeeds at doing so. – Dennis Oct 14 '15 at 13:06
Well, in this case you are enumerating over a list, it just happens to be a list of concrete syntax items, which is how it's stored in the parse tree. When you enumerate over the list Rascal skips the whitespace and list separator automatically since it's a concrete syntax list. You could also use this same technique inside a for loop, so you don't need to save it into another Rascal list first. If you want more details of what it is doing internally feel free to ask. – Mark Hills Oct 15 '15 at 02:03

Jurgen Vinju · Answer 2 · 2015-10-14T12:31:05.423

1

First it could be written as such, to make it easier:

start syntax CharList =  CHAR hd "," {CHAR ","}+ tl ';';

Then:

rascal>t = parse(#start[CharList], "a,b;");
start[CharList]: `a,b;`
rascal>u = t.top;
CharList: `a,b;`
rascal>v = u.tl;
{CHAR  "," }+: `b`

v[0] is broken, although that would seem logical here it returns the prod of the meta representation instead of the actual first element of the list, but we can iterate over the elements over the list and select only the first:

if (CHAR e <- v) println(e); 
b
ok

edited Oct 14 '15 at 12:31

answered Oct 14 '15 at 12:23

Jurgen Vinju

6,393
1
15
26

Thanks, Jurgen. As for using enumeration, see my comments on Mark's solution above. I also expected `v[0]` to work, and was suprised it didn't give me what I wanted. But I haven't been around here long enough to therefore conclude it's broken... – Dennis Oct 14 '15 at 13:12
Actually we discovered this behavior yesterday ourselves and are writing new test cases now. Weird coincidence. – Jurgen Vinju Oct 14 '15 at 13:13

accessing regexp subtree in parsetree

2 Answers2