python speedup a simple function

Question

I try to find a simple way to "speed up" simple functions for a big script so I googled for it and found 3 ways to do that. but it seems the time they need is always the same. so what I am doing wrong testing them?

file1:

from concurrent.futures import ThreadPoolExecutor as PoolExecutor
from threading import Thread
import time
import os
import math
#https://dev.to/rhymes/how-to-make-python-code-concurrent-with-3-lines-of-code-2fpe
def benchmark():
    start = time.time()
    for i in range (0, 40000000):
        x = math.sqrt(i)
        print(x)
    end = time.time()
    print('time', end - start)
with PoolExecutor(max_workers=3) as executor:
    for _ in executor.map((benchmark())):
        pass

file2:

#the basic way
from threading import Thread
import time
import os
import math

def calc():
    start = time.time()

    for i in range (0, 40000000):
        x = math.sqrt(i)
        print(x)

    end = time.time()
    print('time', end - start)

calc()

file3:

import asyncio
import uvloop
import time
import math
#https://github.com/magicstack/uvloop
async def main():
    start = time.time()
    for i in range (0, 40000000):
        x = math.sqrt(i)
        print(x)
    end = time.time()
    print('time', end - start)
uvloop.install()
asyncio.run(main())

every file needs about 180-200 sec

so i 'can't see' a difference.

The way you have written your code, none of these change how your code is run. All of them just execute the code regularly and then pass the result (`None`) to the executor /loop. You can't see a difference because practically there is none. — MisterMiyagi, Jul 27 '19 at 21:53

user4815162342 · Answer 1 · 2019-07-28T08:08:58.473

I googled for it and found 3 ways to [speed up a function], but it seems the time they need is always the same. so what I am doing wrong testing them?

You seemed to have found strategies to speed up some code by parallelizing it, but you failed to implement them correctly. First, the speedup is supposed to come from running multiple instances of the function in parallel, and the code snippets make no attempt to do that. Then, there are other problems.

In the first example, you pass the result benchmark() to executor.map, which means all of benchmark() is immediately executed to completion, thus effectively disabling parallelization. (Also, executor.map is supposed to receive an iterable, not None, and this code must have printed a traceback not shown in the question.) The correct way would be something like:

# run the benchmark 5 times in parallel - if that takes less
# than 5x of a single benchmark, you've got a speedup
with ThreadPoolExecutor(max_workers=5) as executor:
    for _ in range(5):
        executor.submit(benchmark)

For this to actually produce a speedup, you should try to use ProcessPoolExecutor, which runs its tasks in separate processes and is therefore unaffected by the GIL.

The second code snippet never actually creates or runs a thread, it just executes the function in the main thread, so it's unclear how that's supposed to speed things up.

The last snippet doesn't await anything, so the async def works just like an ordinary function. Note that asyncio is an async framework based on switching between tasks blocked on IO, and as such can never speed CPU-bound calculations.

python speedup a simple function

1 Answers1