ANTLR: Lexer Throwing NoViableAltException

Question

After a break of a few weeks, it's time to fight with ANTLR again...

Anyhow, I have the following Lexer tokens defined:

fragment EQ: '=';
fragment NE: '<>';

BOOLEAN_FIELD
  : ('ISTRAINED'|'ISCITIZEN')
  ;

BOOLEAN_CONSTANT
  : ('TRUE'|'FALSE'|'Y'|'N')
  ;

BOOLEAN_LOGICAL
  : BOOLEAN_FIELD (EQ|NE) (BOOLEAN_FIELD|BOOLEAN_CONSTANT)
  ;

Unfortunately, the BOOLEAN_LOGICAL token is throwning NoViableAltException on simple terms such as "ISTRAINED = ISTRAINED".

I know some of the responses are going to be "This should be in the parser". It WAS previously in the parser, however, I'm trying to offload some simple items into the lexer since I just need a "Yes/No, is this text block valid?"

Any help is appreciated.

Are you really using double quotes? `"ISTRAINED"` should be `'ISTRAINED'` — Bart Kiers, Feb 28 '13 at 13:23
No, that was a "copying example to SO" mistake. The original used single quotes. — Jason, Feb 28 '13 at 14:00

Bart Kiers · Accepted Answer · 2013-02-28T15:11:39.270

2

BOOLEAN_LOGICAL should not be a lexer rule. A lexer rule must (or should) be a single token. As a lexer rule, there cannot be any spaces between BOOLEAN_FIELD and (EQ|NE) (you might have skipped spaces during lexing, but that will only cause spaces to be skipped from inside parser rules!).

Do this instead:

boolean_logical
  : BOOLEAN_FIELD (EQ|NE) (BOOLEAN_FIELD|BOOLEAN_CONSTANT)
  ;

which would also mean that EQ and NE cannot be fragment rules anymore:

EQ : '=';
NE : '<>';

edited Feb 28 '13 at 15:11

answered Feb 28 '13 at 14:08

Bart Kiers

166,582
36
299
288

1

Also, `EQ` and `NE` should not be `fragment` rules. – Sam Harwell Feb 28 '13 at 14:56

score 1 · Answer 2 · answered Feb 28 '13 at 15:27

1

This does look like it should be a parser rule. However, if you want to keep it as a lexer rule, you need to allow whitespace.

BOOLEAN_LOGICAL
  : BOOLEAN_FIELD WS+ (EQ|NE) WS+ (BOOLEAN_FIELD|BOOLEAN_CONSTANT)
    ;

answered Feb 28 '13 at 15:27

Troy Daniels

3,270
2
25
57

If Jason has a `WS` rule defined in which he `skip()`s this token (or puts the token on the `HIDDEN` channel), using this `WS` rule will also cause `BOOLEAN_LOGICAL` to be skipped/hidden. – Bart Kiers Feb 28 '13 at 16:24
I figured that out later. Unfortunately, I have to use so much embedded code (in order to throw custom exceptions instead of the usual NoViableAltException or MismatchedSetException, and what circumstances they are thrown in, etc). – Jason Feb 28 '13 at 16:31

ANTLR: Lexer Throwing NoViableAltException

2 Answers2