How to recursively read many include files with lark?

Question

For this config file format:

# comments here
# next an empty commment line
#

include "parseconf/dir_with_many_files"

Thing1
 "objname" {
    InnerThing "inner_thing_name"
        {
            IP_Address = "10.12.14.1" Hostname = "abc.fred.com"
       }
}

Thing2 # comment
 "objname" {
    InnerThing "inner_thing_name" #a comment
        {
            IP_Address = "10.12.14.1"
            Hostname = "abc.fred.com"  # comments here
       }

    # comment
}

The include statement, if pointing to a directory, needs to read all .conf files in that directory.

I have the following lark syntax:

start: (value|COMMENT)*

value: name (COMMENT)* string (COMMENT)* object
    | assignment
    | include

include: "include" string

object: "{" (COMMENT)* (value)* "}" (COMMENT)*

assignment: name "=" string (COMMENT)*
    | object

name: /[A-Za-z_][A-Za-z_0-9]*/

COMMENT: "#" /[^\n]*/ _NEWLINE
_NEWLINE: "\n"

string: ESCAPED_STRING

%import common.ESCAPED_STRING
%import common.SIGNED_NUMBER
%import common.WS

%ignore WS

The tree is built with output including

  value
    include
      string    "parseconf/dir_with_many_files"

Based on this and the comments below, I'm trying to handle the includes with a Transformer, like this:

#!/usr/bin/env python3
from lark import Lark, Transformer
from pprint import pprint
class ExpandIncludes(Transformer):
    def include(self, item):
        path = item[0]
        filedir = strip_quotes(path.children[0].value)
        # how to return back a tree from parsed files?

        return item

def strip_quotes(s):
    if s.startswith('"') and s.endswith('"'):
        return s[1:-1]

def parse_file(conf_grammar, conf_file):
    # Create the parser with Lark, using the Earley algorithm
    conf_parser = Lark(conf_grammar, parser='earley')
    with open(conf_file, 'r') as f:
        test_conf = f.read()
        tree = conf_parser.parse(test_conf)
        ExpandIncludes().transform(tree)
    return tree

if __name__ == '__main__':
    with open('parseconf/try.lark') as f:
        conf_grammar = f.read()
        tree = parse_file(conf_grammar, 'parseconf/test.conf')
        print(tree.pretty())

This gives filedir which I can read and parse. How do you return contents back into the tree?