I've been recently working on a small demo expression grammar in Xtext:
The Grammar
grammar org.example.expressions.Expressions with org.eclipse.xtext.common.Terminals
generate expressions "http://www.example.org/expressions/Expressions"
ExpressionsModel:
elements+=AbstractElement*;
AbstractElement:
Variable | EvalExpression;
Variable:
'var' name=ID '=' expression=Expression;
EvalExpression:
'eval' expression=Expression;
/*-----------------------------------EXPRESSIONS-----------------------------------*/
//Note that expressions on the same precedence get grouped into a single rule, but not necessarily into the same AST node.
//Examples of this pattern are compound rules such as the AddOrSubExpression, or the ComparisonOrInExpression
Expression:
ConditionalExpression;
ConditionalExpression returns Expression:
OrExpression ({ConditionalExpression.condition=current} '?' trueExp=OrExpression ':' falseExp=OrExpression)*;
OrExpression returns Expression:
AndExpression ({BinaryExpression.left=current} operator='||' right=AndExpression)*;
AndExpression returns Expression:
BinOrExpression ({BinaryExpression.left=current} operator='&&' right=BinOrExpression)*;
BinOrExpression returns Expression:
BinXorExpression ({BinaryExpression.left=current} operator='|' right=BinXorExpression)*;
BinXorExpression returns Expression:
BinAndExpression ({BinaryExpression.left=current} operator='^' right=BinAndExpression)*;
BinAndExpression returns Expression:
EqualityExpression ({BinaryExpression.left=current} operator='&' right=EqualityExpression)*;
EqualityExpression returns Expression:
ComparisonOrInExpression ({BinaryExpression.left=current} operator=('==' | '!=') right=ComparisonOrInExpression)*;
ComparisonOrInExpression returns Expression:
BitShiftExpression (({BinaryExpression.left=current} operator=('<' | '>' | '<=' | '>=') right=BitShiftExpression) // ComparisonExpression
| ({InExpression.left=current} 'in' (right=BitShiftExpression | ('[' openRangeList+=OpenRangeValue (','
openRangeList+=OpenRangeValue)* ']'))) // InExpression
)*;
OpenRangeValue:
lowBound=Expression ('..' highBound=Expression)?;
BitShiftExpression returns Expression:
AddOrSubExpression ({BinaryExpression.left=current} operator=('<<' | '>>') right=AddOrSubExpression)*;
AddOrSubExpression returns Expression:
MulOrDivExpression ({BinaryExpression.left=current} operator=('+' | '-') right=MulOrDivExpression)*;
MulOrDivExpression returns Expression:
PowExpression ({BinaryExpression.left=current} operator=('*' | '/' | '%') right=PowExpression)*;
PowExpression returns Expression:
BitSliceExpression ({BinaryExpression.left=current} operator='**' right=BitSliceExpression)*;
BitSliceExpression returns Expression:
FieldOperationExpression ({BitSliceExpression.operand=current} '[' highBound=Expression ':' lowBound=Expression ']')*;
FieldOperationExpression returns Expression:
UnaryExpression (({CollectionAccess.operand=current} '[' index=Expression ']') // CollectionAccessExpression
| ({FunctionCallExpression.targetObject=current} '.' name=ID '(' (params+=Expression (','
params+=Expression)*)? ')') // FieldAccessFunction
| ({FieldAccessExpression.targetObject=current} '.' fieldIdentifier=ID))*; // FieldAccessExpression
UnaryExpression returns Expression:
PrimaryExpression
| {UnaryExpression} operator=UnaryOperator operand=PrimaryExpression;
PrimaryExpression returns Expression:
'(' Expression ')'
| FunctionCallExpression
| CompileHasExpression
| StaticRefrencePathExpression
| {Expression} NullRefrenceExperession
| StringConstantExpression
| BoolConstantExpression
| IntConstantExpression;
FunctionCallExpression:
name=ID '(' (params+=Expression (',' params+=Expression)*)? ')';
CompileHasExpression:
'compile' 'has' staticRefPath=StaticRefrencePathExpression;
StaticRefrencePathExpression:
(globalScope?='::')? (identifiers+=ID ('::' identifiers+=ID)*);
NullRefrenceExperession:
'null';
IntConstantExpression:
value=INT;
StringConstantExpression:
value=STRING;
BoolConstantExpression:
value=BoolValue;
/*-----------------------------------ENUM_TERMINALS-----------------------------------*/
enum UnaryOperator:
ARITH_NEGATE='-' | BOOL_NEGATE='!' | BIN_NEGATE='~' | BIN_AND='&' | BIN_OR='|' | BIN_XOR='^';
enum BoolValue:
TRUE='true' | FALSE='false';
Everything here seemed to work well, but when I added the BitSliceExpression and tried to process the grammar, I've received the following error:
The Error:
error(211): ../org.example.expressions/src-gen/org/example/expressions/parser/antlr/internal/InternalExpressions.g:1626:3: [fatal] rule ruleFieldOperationExpression has non-LL(*) decision due to recursive rule invocations reachable from alts 1,4. Resolve by left-factoring or using syntactic predicates or using backtrack=true option.
After a bit of testing, I reached the conclusion that this problem is caused by ambiguity between BitSliceExpression And CollectionAccessExpression
//BitSliceExpression example:
[5:3]
//CollectionAccessExpression example:
[5]
I have very little knowledge of Xtext, Antlr3, and parsing algorithms in general, but I can only assume that since Expression is a recursive rule, the parser is unable too look past it and check for the semicolon literal, which is the distinguishing mark between BitSliceExpression And CollectionAccessExpression at runtime, and thus an ambiguity is born.
I have spent a long time researching this issue, but due to my limited knowledge I have not been able to find a proper solution, and I would greatly appreciate help.
(Please tell me if more information about my project is required)