1

I'm writing a framework that uses ANTLR to parse Java-style expressions. I had in mind to create a new type of free-form literal. The literal will look similar to a string, so I thought to extend the Java8 grammar I'm using with a new literal identical to StringLiteral, but bounded by '`' characters instead of '"'.

So I created:

ExternalLiteral
    :   '`' StringCharacters? '`'
    ;

in the Lexer and modified:

fragment
StringCharacter
    :   ~["`\\\r\n]
    |   EscapeSequence
    ;

and

fragment
EscapeSequence
    :   '\\' [btnfr"'`\\]
    |   OctalEscape
    |   UnicodeEscape // This is not in the spec but prevents having to preprocess the input
    ;

so '`' would be treated as a special character, identically to '"'. Then in the grammar I modified

literal
    :   IntegerLiteral
    |   FloatingPointLiteral
    |   BooleanLiteral
    |   CharacterLiteral
    |   StringLiteral
    |   ExternalLiteral
    |   NullLiteral
    ;

That seems like it would work to me, but when I try to parse any such expressions, e.g.`0`, I get:

line 1:3 mismatched input '<EOF>' expecting {'boolean', 'byte', 'char', 'double', 'float', 'int', 'long', 'new', 'short', 'super', 'this', 'void', IntegerLiteral, FloatingPointLiteral, BooleanLiteral, CharacterLiteral, StringLiteral, ExternalLiteral, 'null', '(', '!', '~', '++', '--', '+', '-', Identifier, '@'}
line 1:3 missing {IntegerLiteral, FloatingPointLiteral, BooleanLiteral, CharacterLiteral, StringLiteral, ExternalLiteral, 'null'} at '<EOF>'

I've had fights with ANTLR before, I don't know if it's ANTLR or me that is more the problem. Does anyone with more experience than me see what I might've done wrong?

Andrew B
  • 23
  • 1
  • 4
  • 1
    The grammar changes you made are ok. It parses backquote-strings just fine. Make sure you have a good build process on top of things, e.g., "make". BTW, you should probably use grammars-v4/java/java not grammars-v4/java/java8 since the java8 grammar is slow. – kaby76 May 10 '22 at 16:48
  • Thank you very much for the input. My build is in ant: – Andrew B May 10 '22 at 17:23
  • @kaby76 did you test what I did or just eyeball it? It doesn't work for me. – Andrew B May 10 '22 at 17:25
  • I tested those changes. https://github.com/kaby76/so72189805 For any claims I make in SO, I always test everything. – kaby76 May 10 '22 at 18:00
  • I believe you, and so it sounded from your comment. It just won't work for me. I'll try to switch over to the java grammar, but it's gonna take a while--they're pretty different. Thanks again. Might visit here again if I can't figure it out. – Andrew B May 10 '22 at 18:04
  • 1
    I did a preliminary switch to the java grammar and made similar changes there to support the backquote-strings. I'm able to parse them correctly now, so now I just have to recode all the other expression types for the new grammar and I'll be in better shape than before. Thanks again. – Andrew B May 10 '22 at 18:27

0 Answers0