Lazy Evaluation: forward operations to deferred value

Question

I have implemented classes for lazily evaluating configurations that are dumped to JSON. No problem there, just extend the encoder to actively evaluate the classes using a specific protocol (fixed method/property).

class DeferredCall(object):
  """Call that is evaluated lazyly"""
  def __init__(self, func, *func_args, **func_kwargs):
    self.func = func
    self.func_args = func_args
    self.func_kwargs = func_kwargs

  def resolve(self):  # called by JSON encoder
    return self.func(*self.func_args, **self.func_kwargs)

 a = DeferredCall(lambda: 1)
 a # gives <[module].DeferredCall at 0x1e99410>
 a.resolve() # gives 1

Now, with great power comes users that want more power. Namely, doing operations with the classes directly instead of the values they stand for. According to the python data model, this should be as simple as implementing the magic methods, such as __add__, __len__ etc.

Adding

def __add__(self, other):
    return self.resolve() + other

or

def __add__(self, other):
    return self.resolve().__add__(other)

will correctly give me a + 3 == 4.

Implementing all magic methods a bit too much, though. So I tried using __getattr__

def __getattr__(self, item):
  return getattr(self.resolve(), item)

which works for a.__mul__(3) == 3 but blow up for a * 3 == 3 with TypeError: unsupported operand type(s) for *: 'DeferredCall' and 'int'.

So is there any other way to forward operators to the wrapped values? Ideally without venturing to programmatically writing code or the hassle of __getattribute__.

As I understand it, the way Python looks up magic method attributes won't allow you to implement them dynamically like this - for `x + y` if `x.__add__` isn't defined `y.__radd__(x)` gets called, hence the error you're seeing. Also, note that you aren't dealing with cases where `other` is *also* a `DeferredCall`. — jonrsharpe, Oct 28 '15 at 14:30
@jonrsharpe Technically, `x.__add__` is defined - I can use it directly. As far as I understand the docs, both objects being `DeferredCall`s should work as the left hand side will resolve to `x.resolve().__add__(other)`, which will fail but trigger `other.resolve().__add__(x.resolve())` since x has already been resolved in the call to `__add__`. Obviously, at least the first is not correct somehow, but I haven't found any information on how operators resolve attributes. — MisterMiyagi, Oct 28 '15 at 14:40
You can access the method directly, but you can demonstrate for yourself that `x + y` will not call `__getattr[ibute]__` on `x`. — jonrsharpe, Oct 28 '15 at 14:42

MisterMiyagi · Answer 1 · 2016-09-20T08:36:38.930

Posting my solution in case anybody else needs it. Most "builtin" actions such as * or len will not use __getattr[ibute]__, by design.

I've settled for creating the methods programmatically

class DeferredCall(object):
  def __init__(self, func, *func_args, **func_kwargs):
    self.func = func
    self.func_args = func_args
    self.func_kwargs = func_kwargs

  def resolve(self):
    return self.func(*self.func_args, **self.func_kwargs)

  # magic must be resolved explicitly
  # self other - resolve in reflected order to allow other to resolve as well
  for so_magic in [("lt", "gt"), ("le", "ge"), ("eq", "eq"), ("ne", "ne"), ("add", "radd"), ("sub", "rsub"), ("mul", "rmul"), ("div", "rdiv"), ("truediv", "rtruediv"), ("floordiv", "rfloordiv"), ("mod", "rmod"), ("divmod", "rdivmod"), ("pow", "rpow"), ("lshift", "rlshift"), ("rshift", "rrshift"), ("and", "rand"), ("xor", "rxor"), ("or", "ror")]:
    for func_name, refl_name in [(so_magic[0], so_magic[1]), (so_magic[1], so_magic[0])]:
      exec("def __%(fname)s__(self, other):\n\ttry:\n\t\tres = other.__%(rname)s__(self.resolve())\n\t\tif res == NotImplemented:\n\t\t\traise AttributeError\n\texcept AttributeError:\n\t\tres = self.resolve().__%(fname)s__(other)\n\treturn res" % {"fname": func_name, "rname": refl_name})

  # regular magic - immutable only
  for magic in ("str", "nonzero", "unicode", "getattr", "call", "len", "getitem", "missing", "iter", "reversed", "contains", "getslice", "neg", "pos", "abs", "invert", "complex", "int", "long", "float", "oct", "hex", "index"):
    exec("def __%(fname)s__(self, *args, **kwargs):\n\treturn self.resolve().__%(fname)s__(*args, **kwargs)" % {"fname": magic})

Basically, the magic methods must be split into two categories: self-contained ones and contextual ones.

The self-contained ones are created straightforward, resolving the call and executing the magic method. For example, len is resolved as:

def __len__(self, *args, **kwargs):
  return self.resolve().__len__(*args, **kwargs)

The contextual ones must invert the call, e.g. "greater than" other actually checks if other is "smaller than" self. This is required if both objects are deferred calls, allowing other to resolve itself as well; otherwise, a TypeError would be raised by many methods. The direct evaluation is only used if other does not have the inverted version.

def __gt__(self, other):
  try:
    res = other.__lt__(self.resolve())
    if res == NotImplemented:
      raise AttributeError
  except AttributeError:
    res = self.resolve().__gt__(other)
  return res

Some of the calls could probably be implemented more efficiently, since python uses some tricks (which is precisely where my problem comes from in the first place). For example, multiplying could exploit commutativity:

def __mul__(self, other):
  """self * other"""
  return other * self.resolve()

Lazy Evaluation: forward operations to deferred value

1 Answers1