How can I get a list of variable names from a lark-parser Tree?

Question

I am using python 3.8.5 and lark-parser 0.11.2. I have a question about Visitors.

I have a grammar for my needs and Lark is working great. I have a case where, under some conditions, I want to evaluate a returned parse tree and scan it to get a, possibly empty, list of variable names appearing in the tree.

A sample expression is:

count + num_items

The parse tree from the expression is:

Tree('add', [Tree('variable', [Token('VARIABLE', 'count')]), Tree('variable', [Token('VARIABLE', 'num_items')])])

I figured that I would write a Visitor class that would scann the tree for variables and store them in an internal list:

from lark import Visitor, v_args
@v_args(inline=True)
class FindVariables(Visitor):
    def __init__(self):
        super().__init__()
        self.variable_list = []

    def variable(self, var):
        try:
            self.variable_list.append(var)
        except Exception as e:
            raise

I am trying to use it as:

fv = FindVariables()
fv2 = fv.visit(parse_result)
for var in fv.variable_list:
    ...

The issue I have is that when fv = FindVariables() is executed I get a TypeError exception:

f() missing 1 required positional argument: 'self'

If I change the call above to:

fv = FindVariables().visit(parse_result)

the statement runs but fv does not "see" variable_list.

I am probably misusing the Visitor class. Is there a best/better way to approach this?

score 0 · Answer 1 · answered Mar 17 '21 at 15:29

Well, I am answering my question but I am not sure that it is the answer.

I changed Visitor to Transformer in the code block in the question and it just worked.

I am glad that I have a solution but it feels like Visitor should have been the right tool here. Still happy to find out if I am misusing the lib here and if there is a better way.

How can I get a list of variable names from a lark-parser Tree?

1 Answers1