Processing a String with ANTLR4

Question

I'm trying to convert my grammar from v3 to v4 and having some trouble finding all the right pieces.

In v3 to process a String, I used:

public static DataExtractor create(String dataspec) {
    CharStream stream = new ANTLRStringStream(dataspec);
    DataSpecificationLexer lexer = new DataSpecificationLexer(stream);
    CommonTokenStream tokens = new CommonTokenStream(lexer);
    DataSpecificationParser parser = new DataSpecificationParser(tokens);

    return parser.dataspec();
}

How do I change this to work in v4?

Dimitar II · Answer 1 · 2017-06-04T16:16:01.077

28

For ANTLR 4.7 the API was changed a little (ANTLRInputStream is deprecated):

InputStream stream = new ByteArrayInputStream(input.getBytes(StandardCharsets.UTF_8));
lexer.setInputStream(CharStreams.fromStream(stream, StandardCharsets.UTF_8));
parser.setInputStream(new CommonTokenStream(lexer));

Hint: if you want to re-use the parser+lexer instances, call their 'reset()' methods after setting their input streams.

edited Jun 04 '17 at 16:16

answered Jun 04 '17 at 15:42

Dimitar II

2,299
32
33

3

There is also a `CharStreams#fromString` in the class. This helps even more if one only got a String and wants it to be lexed and parsed. But thank you for the hint :) – oruckdeschel Jul 09 '19 at 10:40

score 18 · Accepted Answer · answered Aug 07 '13 at 17:45

The changes that were made are:

ANTLRStringStream has been replaced with a constructor in ANTLRInputStream that takes a String
parser rules now return a context object which has a public field named according to the returns clause of your rule.

So if the dataspec rule says "returns [DataExtractor extractor]", v4 the method becomes:

public static DataExtractor create(String dataspec) {
    CharStream stream = new ANTLRInputStream(dataspec);
    DataSpecificationLexer lexer = new DataSpecificationLexer(stream);
    CommonTokenStream tokens = new CommonTokenStream(lexer);
    DataSpecificationParser parser = new DataSpecificationParser(tokens);

    return parser.dataspec().extractor;
}

Processing a String with ANTLR4

2 Answers2