0

I've been recently working on a small demo expression grammar in Xtext:

The Grammar

grammar org.example.expressions.Expressions with org.eclipse.xtext.common.Terminals

generate expressions "http://www.example.org/expressions/Expressions"

ExpressionsModel:
    elements+=AbstractElement*;

AbstractElement:
    Variable | EvalExpression;

Variable:
    'var' name=ID '=' expression=Expression;

EvalExpression:
    'eval' expression=Expression;

/*-----------------------------------EXPRESSIONS-----------------------------------*/
//Note that expressions on the same precedence get grouped into a single rule, but not necessarily into the same AST node.
//Examples of this pattern are compound rules such as the AddOrSubExpression, or the ComparisonOrInExpression

Expression:
    ConditionalExpression;

ConditionalExpression returns Expression:
    OrExpression ({ConditionalExpression.condition=current} '?' trueExp=OrExpression ':' falseExp=OrExpression)*;

OrExpression returns Expression:
    AndExpression ({BinaryExpression.left=current} operator='||' right=AndExpression)*;

AndExpression returns Expression:
    BinOrExpression ({BinaryExpression.left=current} operator='&&' right=BinOrExpression)*;

BinOrExpression returns Expression:
    BinXorExpression ({BinaryExpression.left=current} operator='|' right=BinXorExpression)*;

BinXorExpression returns Expression:
    BinAndExpression ({BinaryExpression.left=current} operator='^' right=BinAndExpression)*;

BinAndExpression returns Expression:
    EqualityExpression ({BinaryExpression.left=current} operator='&' right=EqualityExpression)*;

EqualityExpression returns Expression:
    ComparisonOrInExpression ({BinaryExpression.left=current} operator=('==' | '!=') right=ComparisonOrInExpression)*;

ComparisonOrInExpression returns Expression:
    BitShiftExpression (({BinaryExpression.left=current} operator=('<' | '>' | '<=' | '>=') right=BitShiftExpression) // ComparisonExpression
    | ({InExpression.left=current} 'in' (right=BitShiftExpression | ('[' openRangeList+=OpenRangeValue (','
    openRangeList+=OpenRangeValue)* ']'))) // InExpression
    )*;

OpenRangeValue:
    lowBound=Expression ('..' highBound=Expression)?;

BitShiftExpression returns Expression:
    AddOrSubExpression ({BinaryExpression.left=current} operator=('<<' | '>>') right=AddOrSubExpression)*;

AddOrSubExpression returns Expression:
    MulOrDivExpression ({BinaryExpression.left=current} operator=('+' | '-') right=MulOrDivExpression)*;

MulOrDivExpression returns Expression:
    PowExpression ({BinaryExpression.left=current} operator=('*' | '/' | '%') right=PowExpression)*;

PowExpression returns Expression:
    BitSliceExpression ({BinaryExpression.left=current} operator='**' right=BitSliceExpression)*;

BitSliceExpression returns Expression:
    FieldOperationExpression ({BitSliceExpression.operand=current} '[' highBound=Expression ':' lowBound=Expression ']')*;

FieldOperationExpression returns Expression:
    UnaryExpression (({CollectionAccess.operand=current} '[' index=Expression ']') // CollectionAccessExpression
    | ({FunctionCallExpression.targetObject=current} '.' name=ID '(' (params+=Expression (','
    params+=Expression)*)? ')') // FieldAccessFunction
    | ({FieldAccessExpression.targetObject=current} '.' fieldIdentifier=ID))*; // FieldAccessExpression

UnaryExpression returns Expression:
    PrimaryExpression
    | {UnaryExpression} operator=UnaryOperator operand=PrimaryExpression;

PrimaryExpression returns Expression:
    '(' Expression ')'
    | FunctionCallExpression
    | CompileHasExpression
    | StaticRefrencePathExpression
    | {Expression} NullRefrenceExperession
    | StringConstantExpression
    | BoolConstantExpression
    | IntConstantExpression;

FunctionCallExpression:
    name=ID '(' (params+=Expression (',' params+=Expression)*)? ')';

CompileHasExpression:
    'compile' 'has' staticRefPath=StaticRefrencePathExpression;

StaticRefrencePathExpression:
    (globalScope?='::')? (identifiers+=ID ('::' identifiers+=ID)*);

NullRefrenceExperession:
    'null';

IntConstantExpression:
    value=INT;

StringConstantExpression:
    value=STRING;

BoolConstantExpression:
    value=BoolValue;

/*-----------------------------------ENUM_TERMINALS-----------------------------------*/
enum UnaryOperator:
    ARITH_NEGATE='-' | BOOL_NEGATE='!' | BIN_NEGATE='~' | BIN_AND='&' | BIN_OR='|' | BIN_XOR='^';

enum BoolValue:
    TRUE='true' | FALSE='false';

Everything here seemed to work well, but when I added the BitSliceExpression and tried to process the grammar, I've received the following error:

The Error:

error(211): ../org.example.expressions/src-gen/org/example/expressions/parser/antlr/internal/InternalExpressions.g:1626:3: [fatal] rule ruleFieldOperationExpression has non-LL(*) decision due to recursive rule invocations reachable from alts 1,4.  Resolve by left-factoring or using syntactic predicates or using backtrack=true option.

After a bit of testing, I reached the conclusion that this problem is caused by ambiguity between BitSliceExpression And CollectionAccessExpression

//BitSliceExpression example:
[5:3]
//CollectionAccessExpression example:
[5]

I have very little knowledge of Xtext, Antlr3, and parsing algorithms in general, but I can only assume that since Expression is a recursive rule, the parser is unable too look past it and check for the semicolon literal, which is the distinguishing mark between BitSliceExpression And CollectionAccessExpression at runtime, and thus an ambiguity is born.

I have spent a long time researching this issue, but due to my limited knowledge I have not been able to find a proper solution, and I would greatly appreciate help.

(Please tell me if more information about my project is required)

Tomer Vax
  • 11
  • 4
  • The problem is that the rule you added conflicts with FieldOperationExpression, which is what the error message says, not CollectionAccessExpression. The problem is that FieldOperationExpression has an alt for `'[' Expression ']'`, but you also define that in BitSliceExpression, so it can't distinguish between the two. One solution is to just change the FieldOperationExpression rule: `FieldOperationExpression: UnaryExpression ( '[' Expression ( ':' Expression )? ']' | '.' RULE_ID '(' ( Expression ( ',' Expression )* )? ')' | '.' RULE_ID )* ;` (ignoring the tree construction code). – kaby76 Sep 07 '21 at 23:31
  • https://www.eclipse.org/Xtext/documentation/307_special_languages.html#expressions – mvmn Sep 07 '21 at 23:35
  • Actually, the replacement rule I gave works, but my explanation was incorrect. The conflict is in the common `'[' Expression` sentential strings between BitSliceExpression and FieldOperationExpression. Your grammar would work just fine in Antlr4, however--I tested it. – kaby76 Sep 08 '21 at 01:04
  • I have tried this method, and it worked, but creating relevant AST branches is very important in my project. Also, I much prefer to use Xtext (Which uses Antlr3), because of all the IDE features it provides. Is there any other way to solve this issue? – Tomer Vax Sep 08 '21 at 07:13
  • Keep with XText for now and just modify the AST construction for each alt of the rule. – kaby76 Sep 08 '21 at 14:35
  • That sounds good, but I'm new to Xtext and I don't know how to implement that. can you show me? – Tomer Vax Sep 08 '21 at 14:48

0 Answers0