Combine Java parser in custom parser for expressions

Question

I'm trying to create a custom domain specific language for creating HTML templates using java expressions statements.

For instance it should parse tags combined with java statements:

<div>
    if (someValue == true) {

        <span>"someValue was true"</span>

    }
</div>

Now I know how to write such a parser. However it would be very much simpler if I could just use the standard Java Parser for the expressions, so that I do not have to reimplement a part of the Java Parser. What I'm trying to achieve could look like:

IfStatementNode parseIfStatement() {

    scanner.expect("if");

    scanner.expect("(");

    JavaExpression expression = scanner.parseJavaExpression(); // <--- how to implement this?

    scanner.expect(")");

    return new IfStatementNode(expression);
}

How can this be done so that scanner.parseJavaExpression parses any Java Expression?

By writing some code I guess. Not sure what you are asking for. — GhostCat, Mar 05 '17 at 11:03
@GhostCat I'm not sure how this isn't clear. I want to parse a Java expression using, preferably, the standard way of doing that. — Tim, Mar 05 '17 at 11:04
If you really want to support all valid Java code, you will have to write your own compiler (or find such library). If you do want to write your own compiler, start by reading the JLS: https://docs.oracle.com/javase/specs/ A much better solution would be making your own language which supports a subset of java expressions. — Nulano, Mar 05 '17 at 11:05
@Nulano I prefer using a library which supports (almost) all valid Java code. Writing my own is ofcourse not really an option. — Tim, Mar 05 '17 at 11:07
Whether you use a library or write your own compiler, it would still be much more complicated than you can probably imagine. Maybe you could instead make an application that translates your language to valid Java code you can then compile with javac (e.g. the "some text" could be replaced by System.out.println("some text");). For this you could (in an extreme case) even use just a series of regex replace 'macros'. — Nulano, Mar 05 '17 at 11:11
Thing is : you only told us what you want, but only to a small degree what you have done so far. — GhostCat, Mar 05 '17 at 12:17

Stefan Haustein · Answer 1 · 2017-03-05T11:27:16.607

One option would be to add some generate code to turn the expression into valid java code for a Function object. Then you can parse the generated function (containing the expression for the return value) using the regular java compiler, which is available at runtime.

This would still require that you are able to find the end of the expression. StreamTokenizer might be sufficient / convenient for this.

Another problem will be keeping track of variables when/if you add support for them.

An alternative approach might be to expand the parts of the template that are not Java (e.g. turn <span> into output.append("<span>")), turning your template into valid Java code, which can be compiled as a whole.

score 1 · Accepted Answer · answered Mar 05 '17 at 11:23

1

You are right: you can reuse JavaParser and I would suggest you do so because parsing Java expressions is not trivial at all. JavaParser is well mantained, and it is working on building support for Java 9, which you will get for free if you use JavaParser.

Using JavaParser in this case it is so simple that I am almost ashamed of writing this code:

Expression expression = JavaParser.parseExpression(code);

And you are done.

Now, that we throw exceptions if there are errors in the code. If you want to get a nice list of issues to report them to your users you could do that like this:

  ParseResult<Expression> result = new JavaParser().parse(ParseStart.EXPRESSION, new StringProvider(code));
  for (Problem p : result.getProblems()) {
      System.err.println("Problem at " + p.getLocation().get() + 
              ": " + p.getMessage());
  }

If you want then to process the code to calculate types or resolve references to external classes you may want to look at JavaSymbolSolver, which is built on top of JavaParser.

Source: I am a JavaParser contributor

answered Mar 05 '17 at 11:23

Federico Tomassetti

2,100
1
19
26

Thanks! Exactly what I need. – Tim Mar 05 '17 at 11:24
1

Happy to hear that! It would be great if you could stop on the github page of JavaParser and let us know more about your prokect! – Federico Tomassetti Mar 05 '17 at 14:05
I will! Though, I am stuck on the following input string: `
`. JavaParser naturally tries to parse `2+2>` and it gets stuck parsing the `>` as it sees it as an operator (which is not the case). I just need the longest possible parse. Any idea? (Attribute values should be parsed as expressions in my case (I am aware of the `<` and `>` problems)
– Tim Mar 05 '17 at 15:28
You could do different things here. The first one (suggested) is to put the value of the attribute between inverted commas and parse the content of the inverted commas. The other thing is to try to parse strings of different length: you parse 2+2> and it does not work? Then you parse one character less or stop where JavaParser give you an error. In other words JavaParser parse Java, you need to figure out which part of your input is Java code and send just that to JavaParser – Federico Tomassetti Mar 06 '17 at 07:49
Thanks for your input. I've decided to first look a head for nested parenthesis which will invoke the expression parser using your first option. So that every Java expression should be wrapped in `(` and `)`. Now it would be great if this wasn't required for anything but operators such as: `product.getBrand()`, `"just a string"` and `123`. I couldn't find a ParseStart for these expression types. Is this possible? If not I think it would be achievable to write it myself though. – Tim Mar 06 '17 at 09:32

Combine Java parser in custom parser for expressions

2 Answers2