Xtext 2.10 (AbstractFormatter2): Indentation for single or multiline comments is not always correct

Question

Problem Description:

The default indentation for single or multiline comments does not always work properly. The following issues are being faced.

• The indentation for single line comments never worked if there wasn’t already at-least a space or a tab in-front of them.
• The indentation for single or multi-line comments never worked if they were part of the IHiddenRegion for which a decrease of indentation was set. The current understanding is that the formatter for the respective IHiddenRegion will increase the indentation for the comments contained in it and will then set an overall decrease of indentation for the upcoming strictly alternating IHiddenRegion(s) or IsemanticRegion(s).

Important Note:

Kindly direct us to an already existing eclipse Xtext ticket, if this is a known issue. Or any stackoverflow threads that explain why this is happening. And what could possible be coded wrong within the formatters based on IFormmatter2. Else please read further, would appreciate useful hints, solutions that could help us reach to a solution.

Further Explanation:

By "default indentation" its meant that:

• The methods applyHiddenRegionFormatting, createCommentReplacer or any other related methods to manipulate comments are not overridden.

• The terminal rules for SL_COMMENT and ML_COMMENT are left unchanged and are as under (taken from Terminals.xtext):

  terminal ML_COMMENT : '/*' -> '*/'; terminal SL_COMMENT : '//' !('\n'|'\r')* ('\r'? '\n')?;

• There are also no other known changes made to manipulate comments in any form.

The current usage/behaviour understanding of AbstractFormatter2 w.r.t. indentation is as follows, please correct if its incorrect.

Given an EObject the following ways to set an indentation collectively for text of contained IHiddenRegion(s) and ISemanticRegion(s) are available:

1. EObject.interior().indent[]
2. Interior.(ISemanticRegion_Start, ISemanticRegion_End).indent[], where ISemanticRegion_Start & ISemanticRegion_End were taken for the keywords '{' and '}' respectively contained within the EObject.
3. Document.set(IHiddenRegion_Start, IHiddenRegion_End).indent[], where IHiddenRegion_Start was the next hidden region of ISemanticRegion_Start (as explained above) and IHiddenRegion_End was the previous hidden region of ISemanticRegion_End (as explained above).

For '2' and '3' above the current understanding is that the indentation will be increased for all the strictly alternating IHiddenRegion(s) and ISemanticRegion(s) that will be in between two ISemanticRegions (in case of '2') or between two IHiddenRegions (in case of '3', the two start and end boundary IHiddenRegions will be included) respectively.

The indentation of the comments showed exactly the same behaviour when using any of the above approaches. Please also advise if there is always a need to override certain methods to handle formatting of comments correctly. Or the default implementations provided for handling comments are sufficient.

As an example, the behaviour of the the formatters from our projects and the one from domain model example (org.eclipse.xtext.example.domainmodel) for the indentation of comments was identical. Hence providing below an example DSL with comments from the domain model example so that its easy to relate to it.

Example domain model DSL with comments:

package p1 {
    /*
     * ml one 
     */
    // sl 11
// sl 12
    package p2 {
    }
    // sl two
    package p3 {
    }
    // sl three
    /*
     * ml two 
     */
    /*
     * ml three 
     */
    package p4 {
    }
// sl four
/*
 * ml four 
 */
/*
 * ml five 
 */
}

In the example above comment "sl 12" is an example of the incorrect indentation case (1). And comments "sl four" and "ml four" are examples of case (2). As explained in the very beginning. Comment "sl 11" is an example of the right indentation of a single line comment when there was at-least a space or a tab in-front of it. The indentation of multi line comments works correctly also when there is no space or tab in-front of them.

Additional Points:

• It was also tested if it makes a difference to use grammar access for aquiring access to the required ISemanticRegions, i.e. EObject.regionFor.ruleCall() instead of Eobject.regionFor.keyword(), but the results were the same.
• Also in the domain model example the formatter extends XbaseFormatter while the formatter in our projects extends AbstractFormatter2, but the results of both formatters is also identical w.r.t. comments. The resulting DSL(s) and comments contained in them are in English language.

Thanks in advance.

Answer 1

you may try this workaround

class DomainmodelFormatter extends XbaseFormatter {

    override ITextReplacer createCommentReplacer(IComment comment) {
    var EObject grammarElement=comment.getGrammarElement() 
    if (grammarElement instanceof AbstractRule) {
        var String ruleName=((grammarElement as AbstractRule)).getName() 
        if (ruleName.startsWith("ML")) return new MultilineCommentReplacer(comment,Character.valueOf('*').charValue) 
        if (ruleName.startsWith("SL")) {
            if (comment.getLineRegions().get(0).getIndentation().getLength() > 0) 
                return new SinglelineDocCommentReplacer(comment,"//")  
            else 
                return new SinglelineCommentReplacer(comment,"//") {

                    override configureWhitespace(WhitespaceReplacer leading, WhitespaceReplacer trailing) {
                        // do nothing
                    }

                    override createReplacements(ITextReplacerContext context) {
                        return context
                    }

                }
            }
        }
        var String elementName=new GrammarElementTitleSwitch().showQualified().showRule().doSwitch(grammarElement) 
        throw new IllegalStateException('''No «ITextReplacer.getSimpleName()» configured for «elementName»''')
    }
}