0

This seems like something that might not be possible, but I'm trying to implement something like this:

a = 0
with before():
  a += 1

do_thing(a) # does thing with a, whose value is now 1
do_thing(a) # does thing with a, whose value is now 2

So I want something that can take the block within that with statement, save that block somewhere and have it be called prior to each do_thing function call, while using that scope.

Another option is something like this:

@before
def callback():
  a += 1

Rather than a with statement. Either option, I suppose, is fine with me, although the with statement is preferred.

There is something that is supposed to do what I want, I think, here, but I get an error when I actually try it.

Eugene Bulkin
  • 1,068
  • 1
  • 13
  • 14
  • Why is the with statement preferred? Its meaning is nothing like a with statement's normal meaning, so even if you managed to hack it up somehow it would be horribly unreadable... – abarnert Oct 16 '13 at 06:07
  • 2
    Meanwhile, either way, how is the interpreter supposed to know that your `before` callback is supposed to be attached to some `do_thing` function defined at some remote place? Is it supposed to magically attach to whatever function gets called first after you define it? – abarnert Oct 16 '13 at 06:09
  • @abarnert These are going to be part of a class. The do_thing function is not a random function, it's one that will be defined later to specifically look at the function that the before block put in. What I want ideally is to have the before callback be stored in a list somewhere with the corresponding locals/globals. And the with statement is preferred because it removes a bit of the extraneous stuff that goes with defining a function. Basically I want some sort of simple anonymous function functionality. I don't mean it for use as a real with statement. – Eugene Bulkin Oct 16 '13 at 14:52
  • If you really think that `with` is a good way to define anonymous functions, you really should be using a different language. Python is designed around strong idioms that are 100% consistent, and that's what makes it readable. If you want a language that's designed to let you twist the syntax into knots, Python is _terrible_ at that, while there are other languages (e.g., almost anything derived from Lisp) that are great at it. – abarnert Oct 16 '13 at 18:34
  • I'm not trying to do it because I want to use anonymous functions and the only language I know is Python. I'm doing it because I want to implement something *close* to anonymous functions in Python. I *don't* think it's a good way to do it. But it's the way with the least syntactical noise. – Eugene Bulkin Oct 17 '13 at 01:26
  • Are you saying that you want the `with` block to *define* the function? That is, at the time that you write the `with` statement, `before` does not exist? That is not possible, since in a `with X` statement, `X` is evaluated when the `with` statement is executed, so it already has to exist. – BrenBarn Oct 17 '13 at 05:39
  • Also, when you do write `do_thing`, how do you want `do_thing` to know that it is supposed to execute `before`? Is it supposed to look for something specifically named `before`? I don't get from your example how you intend to encode the information about the link between `before` and `do_thing` (i.e., how either knows that the other exists). – BrenBarn Oct 17 '13 at 05:41
  • @EugeneBulkin: You've got a weird notion of "syntactical noise". In a normal function definition, everything is significant and relevant—the `def` or `lambda` says that you're defining a function. In your weird thing, the `with` is meaningless (if not misleading)—it says you're entering a context, which is not part of what you're trying to do, it's just a side-effect of the strange way you're trying to do it. – abarnert Oct 17 '13 at 17:50
  • @BiRico: I assumed the OP already knew about `lambda`, and knew that `lambda` can't be used with statements (like `a += 1`), and didn't bother to explain that. But you could be right that he doesn't; I shouldn't have just assumed that. – abarnert Oct 17 '13 at 17:51
  • @abarnert you were right in your assumption. I know how Python works. Obviously lambdas aren't useful here because they don't have enough room to do things. – Eugene Bulkin Oct 18 '13 at 03:05
  • @BrenBarn the plan was to, in the class which has before and do_thing as methods/decorators/whatever, store the functions in a property, and the do_thing function would look at that property and call those callbacks. – Eugene Bulkin Oct 18 '13 at 03:06
  • And look, I'm aware that this is not the proper usage for the with statement. You don't need to keep reminding me, I know it's not right. I'm using it because of the idea that it starts a block, since there's no other way to just start a block. – Eugene Bulkin Oct 18 '13 at 03:08
  • @EugeneBulkin: The thing is that `with` doesn't "just start a block". It starts a `with` block. There isn't any such thing as "just a block" in Python. Also, re your other comment: store what functions in a property? How does the class know which functions to store as callbacks and which are the ones that will call those callbacks? – BrenBarn Oct 18 '13 at 03:11
  • @BrenBarn: It's more than that; "start a block" has no semantic content at all in Python. Block structure isn't represented in any way at the bytecode level (as `dis.dis('if True: print(1)')` should demonstrate). This isn't true for most other Algol-family languages (e.g., C and its descendants, where a block is a new lexical scope), but it is for Python (which of course means there's no closure for a block, unless that block is a function or class definition). – abarnert Oct 18 '13 at 08:15
  • @EugeneBulkin: Can you at least comment on my answer, so I have some idea on how much of your question (if any) has been answered? – abarnert Oct 18 '13 at 08:16
  • @BrenBarn The class has a list called beforeCallbacks. When you use the before block, it would store the functions in that list. and then a function that is supposed to call them will call them from that list. – Eugene Bulkin Oct 18 '13 at 18:02

1 Answers1

1

You can create a decorator that stores functions in a list, to be attached to another function by another decorator. That seems to be what you think is the hard part of your problem, but it's trivial:

before_funcs = []
def before(func):
    before_funcs.append(func)
    return func

def attach_befores(func):
    @functools.wraps(func)
    def newfunc(*args, **kwargs):
        for before_func in before_funcs:
            before_func()
        return func(*args, **kwargs)
    return newfunc

So, now you can do this:

a = 0

@before
def callback():
    global a
    a += 1

@before
def another():
    global a
    a *= 2

@attach_befores
def do_thing(i):
    print(i)

Notice that you need the global a there, because the function isn't valid otherwise.


And now, you can call it:

do_thing(a)
do_thing(a)
do_thing(a)
do_thing(a)

However, it's not going to give you the result you wanted—in particular, changing the global a will not change the argument passed to the real do_thing function. Why? Because function arguments are evaluated before the function is called. So, rebinding a after the argument has already been evaluated does you no good. It will, of course, still change the argument passed to the next call. So, the output will be:

0
2
6
14

If you just want to modify the argument passed to the function, you don't need all this mucking about with globals. Just have the before function modify the argument, and have the decorator-applier pass the arguments through each before function before passing them to the real function.

Or, alternatively, if you want to modify the globals used by the function, have the function actually use those globals instead of taking parameters.

Or, alternatively, if you want to mutate the value in place, make it something mutable, like a list, and make the before functions mutate the value instead of just rebinding the global to a different value.

But what you're asking for is a decorator that can reach up to the calling frame, figure out what expressions were evaluated to get the arguments, and force them to be re-evaluated. That's just silly.


If you really, really wanted to do that, the only way to do it would be to capture and interpret the bytecode in sys._getframe(1).f_code.

At least in CPython 2.7, you will get some sequence of codes that pushes your decorated function onto the stack (a simple LOAD_NAME or LOAD_NAME in the typical case, but not necessarily), then a sequence of codes to evaluate the expressions, then a CALL_FUNCTION/CALL_FUNCTION_VAR/etc. So, you can walk backward, simulating the operations, until you've found the one that pushed your function onto the stack. (I'm not sure how to do this in a foolproof way, but it ought to be doable. Then, build a new code object that just pushes your function with a LOAD_CONST and repeats all the operations after it (and then returns the value). Then wrap that code in a function with the exact same environment as the caller, then call that new function and return its value, instead of calling the wrapped function directly.

Here's an example:

def call_do_thing(b):
    global a
    b += a
    return do_thing(a * b)

The psuedo-bytecode is:

LOAD_FAST b
LOAD_GLOBAL a
INPLACE_ADD
STORE_FAST b
LOAD_GLOBAL do_thing
LOAD_GLOBAL a
LOAD_FAST b
BINARY_MULTIPLY
CALL_FUNCTION 1
RETURN_VALUE

In this case, finding the function call is easy, because it used a LOAD_GLOBAL. So, we just need to take all the ops from there to the RETURN_VALUE and wrap them up in a new function to call instead of the one we've been given, and a will be re-evaluated in the new globals.

abarnert
  • 354,177
  • 51
  • 601
  • 671
  • It's being used for a test suite. So of course it wants to reach up into the previous frame. I don't see how that makes it silly, it just makes it difficult. That said, it does appear that the only way to do so would be through messing about with the bytecode, which is pretty inefficient and sort of overkill. At this stage it seems that this is not possible with the way Python works. – Eugene Bulkin Oct 18 '13 at 18:04
  • @EugeneBulkin: Messing with bytecode isn't particularly inefficient, just complicated. But I still don't see why you need to do it. If you just want to intercept and transform arguments, that's trivial with a decorator. The only reason this is hard is because you're trying to intercept the evaluation of the arguments before the call site, which is impossible with a decorator. – abarnert Oct 19 '13 at 20:02
  • 1
    Meanwhile, if you want to change the language semantics, have you looked at MacroPy? – abarnert Oct 19 '13 at 20:04
  • Hmm, MacroPy actually looks pretty interesting. I might look into that. – Eugene Bulkin Oct 20 '13 at 16:57