VSCode Syntax Highlighting Problem: positive lookbehind does not work with whitespace

Question

I am working on a custom language syntax highlighting extension for VSCode. I am running into a problem where lookbehinds are behaving strangely when used next to whitespaces.

Example of code I want to highlight:

variableName :=thisValueShouldHighlight;
variableName := thisValueShouldHighlight;
variableName := thisValueShouldAlsoHighlight,

A sample of the code I am attempting to use (in tpl.tmLanguage.json):

"end_variable_assignment": {
    "comment": "Covers ending assignment of a value to a variable. IN PROGRESS",
    "match": "(?<=:=)\\s*(\\w+)(;|,)$",
    "name": "punctuation.accessor.tpl",
    "captures": {
        "1": {
            "name": "entity.name.type.tpl"
        },
        "2": {
            "name": "punctuation.accessor.tpl"
        }
    }
}

This pattern is included with (in another pattern, which is present throughout the file):

{
    "include": "#end_variable_assignment"
}

The sticking point is the \\s* part of the regular expression. You should be able to do var:=value or var := value or any number of spaces between := and value, it doesn't matter how many. However, I get this when I try to use it:

variableName :=thisHighlightsProperly;
variableName := thisFails;

I have also tried just \\s and \\s+, and neither work. I've tried stupid examples to make sure my logic works by using some placeholder, like (?<=:=)#(\\w+)..., and then tested it with var:=#value, which does work... but it never works with spaces.

I don't understand why this is a problem, because I've used \\s* elsewhere without any issues (just different circumstances). I don't believe it's a property problem because it works with everything except whitespace.

Additional info: I have tested this on RegexCoach and Regex101.com, and it works there.

For testing purposes, I have also included a testing example of the code that should highlight here: https://drive.google.com/open?id=1yoDXVxW3LFYjejW1wps8ENWUQ4iCal9w

This is the most minimal example I can provide of the code:

tpl 1.15 module Pattern_Module_Name;
pattern Pattern_name 1.0
    triggers
        on si := SoftwareInstace created, confirmed where name matches "(?i)SomeRegex";
    end triggers;
    body
        // RE: STACKOVERFLOW PROBLEM
        // As you can see, these aren't highlighting properly

        var_name :=thisShouldHighlight;
        var_name := thisShouldHighlightButDoesnt;
        var_name := thisShouldHighlightButDoesnt,
    end body;
end pattern;

All of my custom language code can be found on my GitHub: https://github.com/cdpautsch/vscode-tpl

score 1 · Answer 1 · answered May 13 '19 at 00:09

So thanks to @Gama11, I was able to figure out what the issue was. Another pattern was interfering with the first, but not in a way that was immediately obvious to me.

The pattern in question is meant to be the end part of a variable assignment (actually assigning a value), so it doesn't apply when using functions or strings (as intended). It should only apply to assigning numbers and the value of other variables to the variable in question. The beginning part of the variable assignment (taking a variable name and assigning something to it) is done with a separate block.

Upon first look, I didn't think it was a problem because there is another pattern already included (var_assign) which matches to :=. The problem with the begin_variable_assignment pattern is that it matched to \\:\\=\\s*. The trailing \\s* is the problem. Removing that causes the problem to work again.

In short, the following matches in separate patterns can all coexist peaceably:

":="

"^\\s*(\\w+)\\s*(\\:\\=)"

"(?<=:=)\\s*(\\w+)(;|,)$"

But this will NOT be compatible:

"^\\s*(\\w+)\\s*(\\:\\=)\\s*"

All this is purely by observation/testing. Clearly the issue is with overlaps, possibly with variable-length matches.

VSCode Syntax Highlighting Problem: positive lookbehind does not work with whitespace

1 Answers1