Parsing: functional inheritance for class methods in Python

Question

For some context I am trying to write a parser for a search language. At the moment I have a bunch of class methods for accepting a token given that it satisfies a certain criterion, however it seems that their general structure/format can be encapsulated with the (newly written) method _accept_generic.

class Parser(object):
    ...
    def now(self):
        # now = self._tokens[self._pos] if not self.end() else None
        # print(now, self._pos)
        return self._tokens[self._pos] if not self.end() else None

    def end(self):
        return self._pos == self._length
    
    def step(self):
        self._pos += 1

    def _accept_generic(self, criterion):
        if not self.end():
            if criterion:
                self.step()
                return True
        return False
    
    def _accept(self, char):
        return self._accept_generic(self.now() == char)
        # if not self.end():
        #     if self.now() == char:
        #         self.step()
        #         return True
        # return False
        
    def _accept_re(self, regex):
        return self._accept_generic(regex.match(self.now()))
        # if not self.end():
        #     if regex.match(self.now()):
        #         self.step()
        #         return True
        # return False
    
    def _accept_any_res(self, *regexes):
        return self._accept_generic(any([r.match(self.now()) for r in regexes]))
        # if any([r.match(self.now()) for r in regexes]):
        #     self.step()
        #     return True
        # return False

    def _accept_all_res(self, *regexes):
        return self._accept_generic(all([r.match(self.now()) for r in regexes]))
        # if all([r.match(self.now()) for r in regexes]):
        #     self.step()
        #     return True
        # return False

The problem with the current code is that it will throw errors (since the criterion is being evaluated in the function argument rather than within the if statement if not self.end()). Is there any way using e.g. functools to allow the functions to inherit the generic's structure, with the ability to specify new arguments in the child functions' code, and not have to write the same code blocks repeatedly? functools.partialmethods doesn't really do what I'm looking for.

just as a note on your code, I suggest making use of Python's standard methods of iteration (given that you are iterating over input to parse it). The use of `yield` to make a generator and the standard exception `stopiteration` (instead of your `self.end`) are worth considering — jrmylow, Oct 02 '20 at 08:03
Will the `criterion` you want always depend in some way on the `self.now()` result? Is that an intentional aspect of the design? — Karl Knechtel, Oct 02 '20 at 11:49
The criterion will depend on what kind of thing I am trying to parse the current token as, so yes. Outside of regexes and matching text however, no, the generic is just meant to save me some rewriting. — James Harrison, Oct 02 '20 at 12:05

score 2 · Accepted Answer · answered Oct 02 '20 at 07:56

2

You can make the criterion a function and pass it to _accept_generic:

def _accept(self, char):
        return self._accept_generic(lambda c=char: self.now() == c)

Then call in _accept_generic:

def _accept_generic(self, criterion):
        if not self.end():
            if criterion():
                self.step()
                return True
        return False

answered Oct 02 '20 at 07:56

Wups

2,489
1
6
17

Tried this but now it seems to be raising some new bugs? Notably when I track where self.now() is, it seems to skip tokens. – James Harrison Oct 02 '20 at 08:28
This seems to be a different bug to me. We don't see what `self.now()` does, so I can't comment on that. It's unlikely that the lambda function has something to do with it. – Wups Oct 02 '20 at 08:45
Updated to include what `self.now()` does – James Harrison Oct 02 '20 at 11:39
Never mind, turns out I just forgot to put the function braces after `criterion`. – James Harrison Oct 02 '20 at 12:01

Parsing: functional inheritance for class methods in Python

1 Answers1