Another no viable alternative at ...

Question

I have a math expression grammar file like the one from the online tutorials: http://javadude.com/articles/antlr3xtut/

But now I want to add the option for functions and I have troubles to get the lexer/parser rule working. I can use an ugly lexer rule to get the code working but I want to use a more clean token to geht the parser rule working.

If I try to do so, I will catch a "line 1:9 no viable alternative at input 'Test('" exception for a expression like 'a*b/13.2*Test(3,2)'

Please check the comment in the following grammar file to see what my exact problem is

grammar ExpressionOnly;

options {
  language = Java;
}

@header {
  package kic.engine.grammar;
}

@lexer::header {
  package kic.engine.grammar;
}


// Top Rule
eval
  : expression
  ;

term 
  : func                     
  | '(' op1=expression ')'   
  | array                    
  | element                  
  ;

  // Sub Terms
  func
    // :  f=FUNC                   // Works but this is very ugly because FUNC contains '(';
    :  f=IDENT '('                 // <---------------------------- why does this not work: line 1:9 no viable alternative at input 'Test('
         (arg=expression (',' arg=expression)*)? 
       ')'
    ;

  array
    : '['  ele=element (',' ele=element)* ']'
    ;

  element
    : b=(K_TRUE | K_FALSE)      
    | NUMBER                    
    | IDENT                     
    | DATE                      
    | SQ_STRING                 
    | K_NULL                    
    ;

negation
  @init{ boolean negate = false; } 
  : (K_NOT | '!' { negate = true;} )? 
    term  
  ;

unary 
  @init{ boolean positive = true; }
  : ('+' | '-')* 
    negation 
  ;

power
  : op1=unary 
    ( '^'  op2=unary 
    )*  
  ;

multiply
  : op1=power          
    ( '*'  op2=power   
    | '/'  op2=power   
    | '%'  op2=power   
    )* 
  ;

add 
  : op1=multiply         
    ( '+' op2=multiply   
    | '-' op2=multiply   
    )*
  ;

relation
  : op1=add 
    ( '='   op2=add   
    | '!='  op2=add   
    | '<'   op2=add   
    | '<='  op2=add   
    | '>'   op2=add   
    | '>='  op2=add   
    )* 
  ;

expression
  : op1=relation
    ( (K_AND | '&')  op2=relation    
    | (K_OR | '|')   op2=relation   
    )*
  ;


// Case-insensitive alpha characters
fragment A: ('a'|'A');
fragment B: ('b'|'B');
fragment C: ('c'|'C');
fragment D: ('d'|'D');
fragment E: ('e'|'E');
fragment F: ('f'|'F');
fragment G: ('g'|'G');
fragment H: ('h'|'H');
fragment I: ('i'|'I');
fragment J: ('j'|'J');
fragment K: ('k'|'K');
fragment L: ('l'|'L');
fragment M: ('m'|'M');
fragment N: ('n'|'N');
fragment O: ('o'|'O');
fragment P: ('p'|'P');
fragment Q: ('q'|'Q');
fragment R: ('r'|'R');
fragment S: ('s'|'S');
fragment T: ('t'|'T');
fragment U: ('u'|'U');
fragment V: ('v'|'V');
fragment W: ('w'|'W');
fragment X: ('x'|'X');
fragment Y: ('y'|'Y');
fragment Z: ('z'|'Z');


// Fragments
fragment DIGIT    : '0' .. '9';  
fragment UPPER    : 'A' .. 'Z';
fragment LOWER    : 'a' .. 'z';
fragment LETTER   : LOWER | UPPER;
fragment WORD     : LETTER | '_' | '$' | '#' | '.';
fragment ALPHANUM : WORD | DIGIT;
fragment ESCAPE[StringBuilder buf] : 
  '\\'
  ( 't' { buf.append('\t'); }
  | 'n' { buf.append('\n'); }
  | 'r' { buf.append('\r'); }
  | '"' { buf.append('\"'); }
  | '\\' { buf.append('\\'); }
  )
  ;

// Keyowords
K_FALSE : F A L S E;
K_NULL : N U L L;
K_TRUE : T R U E;
K_AND : A N D;
K_NOT : N O T;
K_OR : O R;

// Tokens;
FUNC : LETTER+ '(';

IDENT : LETTER ALPHANUM*;

ARRAY_INDEX : IDENT '[';

DQ_STRING 
  @init { final StringBuilder buf = new StringBuilder(); } 
  : '"' 
    ( ESCAPE[buf]
    | i = ~('\\' | '"') { buf.appendCodePoint(i); }
    )*
  { setText(buf.toString()); }
  ;  

NUMBER: DIGIT+ ('.' DIGIT+)? (('e'|'E')('+'|'-')? DIGIT+)?;

DATE: '\'' DIGIT DIGIT DIGIT DIGIT '-' DIGIT DIGIT '-' DIGIT DIGIT (' ' DIGIT DIGIT ':' DIGIT DIGIT ':' DIGIT DIGIT ('.' DIGIT+)?)? '\'';

SQ_STRING : '\'' .* '\'';

// hidden tokens
WS : (' ' | '\t' | '\r' | '\n')+ {$channel=HIDDEN;};

COMMENTS : '/*' .* '*/' {$channel=HIDDEN;};

Any ideas how I can get the function rule working with a IDENT token?

score 3 · Accepted Answer · answered Dec 25 '12 at 18:36

The input Test(3,2) turns into the following tokens:

[FUNC : Test(] [NUMBER : 3] [',' : ,] [NUMBER : 2] [')' : )]

No parser rule currently expects a FUNC token, so the parser produces an error that prints the token's contents: line 2:1 no viable alternative at input 'Test('.

Comment out the FUNC lexer rule, regenerate everything, and rerun. Now the same input produces these tokens without error:

[IDENT : Test] ['(' : (] [NUMBER : 3] [',' : ,] [NUMBER : 2] [')' : )]

For testing purposes, I made the grammar output an AST and changed the term f=IDENT in parser rule func to f=IDENT^, to make it easier to see in the AST whether the parser recognized a function.

Now, with input a*b/13.2*Test(3,2) I get the following AST:

AST

Input Test(3,2) is correctly recognized as a function and the AST is produced accordingly.

Ha! Thank you very much I would have never thought about a lexer issue, I was too focused on a parser issue. — KIC, Dec 25 '12 at 21:36
@KIC When you're stumped and in doubt, just dump all your tokens out! (Haha, I'm a poet.) Bart Kiers [has a useful grammar snippet](http://stackoverflow.com/a/13684401/1201210) that makes doing so easy. Merry Christmas/Happy Holidays. :) — user1201210, Dec 25 '12 at 22:08

Another no viable alternative at ...

1 Answers1