Python how to optimize map function to handle huge data without reaching 100% CPU

Question

I am trying to iterate over range of large numbers and perform a small calculation. But my CPU is reaching 100% until the execution is done. Can someone suggest me how to optimize this or let me know if any better approach can be done. My hardware is windows10,4core CPU,8GB RAM.

from functools import partial


def using_map_function(number, name):
    print("number square is {} and repo_name is {}".format(number * number, name))

def using_range(numbers, name):
    for number in numbers:
        print("number square is {} and repo_name is {}".format(number * number, name))

if __name__ == '__main__':
    # mapFunc = partial(using_map_function, name="Hello")
    # map(mapFunc, range(500000))
    using_range(numbers=range(500000),name="Hello")

what's wrong with 100% of cpu being used? .... to slow it down, you can add a sleep in your loop. — Corey Goldberg, Jul 31 '19 at 18:41
I thought that introducing sleep will increase the script execution time, so I didn't try with sleep. — Auto-learner, Jul 31 '19 at 18:43
you are basically asking "how do I slow my program down without slowing it down" — Corey Goldberg, Jul 31 '19 at 18:45
my aim is to reduce the cpu usage without reducing the execution time — Auto-learner, Jul 31 '19 at 18:47
if you want to do some other tasks while running this large query, you can try to limit the CPU usage for Python — Verse He, Jul 31 '19 at 23:48

score 0 · Answer 1 · answered Jul 31 '19 at 18:40

0

Using the print() function wastes a lot of time for processing, you can try to remove them when you're sure your code is running well. You can also have a lot at this post in order to try limitating the CPU usage of your python script.

answered Jul 31 '19 at 18:40

cocool97

1,201
1
10
22

Python how to optimize map function to handle huge data without reaching 100% CPU

1 Answers1