is it possible to simplify loops and multiple if statement checks for speed?

Question

I've got a list of several instances of the same python object that I want to loop through and perform 3 of the methods on; however, there are situations I want to select which of the 3 to run (in any combination).

I could do something like this:

do_method_1 = True
do_method_2 = True
do_method_3 = True

for item in list_of_items:
    if do_method_1:
        item.method_1()
    if do_method_2:
        item.method_2()
    if do_method_2:
        item.method_2()

It's easy to read but, I'm doing each check multiple times for each item in the loop and slowing things down a little. I could also flip it by doing each check and then loop through the items for each True/False check... but then I'm potentially looping through everything multiple times and slowing things down that way.

I'm curious if there's an easy way to only evaluate the checks and the loop once. I've currently got something like this:

do_method_1 = True
do_method_2 = True
do_method_3 = True

if do_method_1:
    if do_method_2:
        if do_method_3:
            for item in list_of_items:
                item.method_1()
                item.method_2()
                item.method_3()
        else:
            for item in list_of_items:
                item.method_1()
                item.method_2()
    elif do_method_3:
        for item in list_of_items:
            item.method_1()
            item.method_3()
    else:
        for item in list_of_items:
            item.method_1()
else:
    if do_method_2:
        if do_method_3:
            for item in list_of_items:
                item.method_2()
                item.method_3()
        else:
            for item in list_of_items:
                item.method_2()
    elif do_method_3:
        for item in list_of_items:
            item.method_3()
    else:
        print('not doing anything...')

This does work out so that the loop is only executed once, and I think each check would technically only get evaluated once, but it's a lot of repeat code clutter and would become even more of a headache to write/read if a 4th method was added to the list of possibilities. So, is there another way to write this out that's 'cleaner' to do the loop only once and do each check only once for speed?

Thanks!

Have you profiled your code to determine that this is a bottleneck? — C_Z_, Jan 22 '20 at 20:58
In addition to what @C_Z_ asked, can you share something which is closer to your actual code? — AMC, Jan 22 '20 at 21:42

score 1 · Answer 1 · answered Jan 22 '20 at 20:53

1

Make a list of the methods to call, and loop over the list.

methods = ['method_1', 'method_2', 'method_3']
for item in list_of_items:
    for m in methods:
        getattr(item, m)()

Then you can change the contents of methods to reflect the situation.

See Call a Python method by name

answered Jan 22 '20 at 20:53

Barmar

741,623
53
500
612

An alternative - not that it's needed in this case (or makes much difference) is to allow Python to do the `getattr` machinery: `for m in (item.method_1, item.method_2, item.method_3): m()` – Jon Clements Jan 22 '20 at 21:01
@JonClements But where is the conditional logic there? It needs to be something he can control with a variable. – Barmar Jan 22 '20 at 21:02
Moved inside the loop with `methods` containing attributes directly rather than by name... so presumably the same way however `methods` is somehow built... – Jon Clements Jan 22 '20 at 21:05
I'm still not seeing your point, unless you're suggesting Alex Skalozub's method wit a list like `[Item.method_1, Item.method_2, ...]`. – Barmar Jan 22 '20 at 21:37

score 1 · Answer 2 · answered Jan 22 '20 at 21:23

If items in list_of_items all have the same (and known) type, using a list of methods instead of list of names would be twice as faster, and virtually the same as a plain loop with ifs. This could matter for a large number of items.

import time

class Item:
    def method_1(self):
        pass
    def method_2(self):
        pass
    def method_3(self):
        pass

list_of_items = []  
for i in range(100000):
    list_of_items.append(Item())

do_method_1 = True
do_method_2 = True
do_method_3 = True

def test1():
    for item in list_of_items:
        if do_method_1: item.method_1()
        if do_method_2: item.method_2()
        if do_method_2: item.method_2()

def test2():
    methods = []
    if do_method_1: methods.append('method_1')
    if do_method_2: methods.append('method_2')
    if do_method_3: methods.append('method_3')

    for item in list_of_items:
        for m in methods:
            getattr(item, m)()

def test3():
    methods = []
    if do_method_1: methods.append(Item.method_1)
    if do_method_2: methods.append(Item.method_2)
    if do_method_3: methods.append(Item.method_3)

    for item in list_of_items:
        for m in methods:
            m(item)

print('testing plain ifs')
start = time.time()
test1()
print(time.time() - start)

print('testing method names')
start = time.time()
test2()
print(time.time() - start)

print('testing method ptrs')
start = time.time()
test3()
print(time.time() - start)

Outputs:

testing plain ifs
0.022666215896606445
testing method names
0.04316902160644531
testing method ptrs
0.021532058715820312

Note this would only work if all 3 methods have the same number of arguments. Once the arguments are different, the initial implementation with ifs is the way to go.

Thank you for taking a look at these different options, and for the timing tests - it helped in finding the solution I was looking for! — silent_sight, Jan 23 '20 at 16:21

score 1 · Accepted Answer · answered Jan 22 '20 at 21:39

1

Before the loop, decide if each call will be a methodcaller object or a do-nothing function.

from operator import methodcaller

def noop(obj):
    pass

do_method_1 = True
do_method_2 = True
do_method_3 = True

m1 = methodcaller('method_1') if do_method_1 else noop
m2 = methodcaller('method_2') if do_method_2 else noop
m3 = methodcaller('method_3') if do_method_3 else noop    


for item in list_of_items:
    m1(item)
    m2(item)
    m3(item)

Similar to Barmar's answer, you can create a list of the methodcaller objets to use and iterate over it, eliminating the cost of calling noop.

answered Jan 22 '20 at 21:39

chepner

497,756
71
530
681

But note that performance-wise it is only a bit faster than Barmar's `getattr()` suggestion, and is slower than original plain `if`s in the loop. Dynamic method calling is expensive! – Alex Skalozub Jan 22 '20 at 23:45
This was really helpful - I ended up doing something very similar to this... I used Item.method_x (similar to what was in test3 from Alex Skalozub's answer) instead of methodcaller. I liked how this didn't have any nested loops and checks, and the noop call was really not much of a hit (if any), thanks a lot! – silent_sight Jan 22 '20 at 23:46

is it possible to simplify loops and multiple if statement checks for speed?

3 Answers3