Performance when passing huge list as argument in recursive function?

Question

I am using Python and I have a recursive function that takes a huge list as one of the arguments:

# Current implementation
def MyFunction(arg1, arg2, my_huge_list)
    ...
    ...
    MyFunction(new_arg1, new_arg2, my_huge_list)

As you can see above, MyFunction is called recursively using the same list my_huge_list; this doesn't change, unlike the other arguments. And, again, this list is huge. A friend of mine suggested that I could treat my_huge_list as a global variable to improve the performance, as otherwise this huge list may be copied over and over in every iteration.

# Friend's suggestion

MyHugeList=[a,b,c, ...and many many other elements... ]

def MyFunction(arg1, arg2)
    global MyHugeList
    ...
    ...
    MyFunction(new_arg1, new_arg2)

Does using a global variable as shown above improve the performance of the algorithm over the original version? My program runs for weeks, so even a slight improvement may be valuable in the long run.

Function arguments are not copied in Python. Your list won't be copied for the recursive call. — BrenBarn, Oct 04 '13 at 17:59

score 12 · Accepted Answer · answered Oct 04 '13 at 18:05

12

The list will be passed by reference, so it doesn't take any longer to transfer a 1-item list vs. a 100000 item list:

def null(x): return x
longlist = range(100000)
shortlist = range(1)
longerlist = range(1000000)

%timeit null(shortlist)
10000000 loops, best of 3: 124 ns per loop

%timeit null(longlist)
10000000 loops, best of 3: 137 ns per loop

%timeit null(longerlist)
10000000 loops, best of 3: 125 ns per loop

The longer lists have 100k and 1M entries in them, and yet don't take significantly long to pass as arguments than shorter lists.

there may be other ways to improve performance; this probably isn't one of them.

answered Oct 04 '13 at 18:05

Corley Brigman

11,633
5
33
40

Note that your example is dependent on which version of python you are using. In a newer version `range(100000)` is a generator, not a list of of numbers. This doesn't change the fact that the list _will_ be passed by reference anyways, it just makes your example not as strong. – Hooked Oct 04 '13 at 19:16
definitely true; still stuck on 2.6 here :) i suppose a safer way would be something like ['x'] * 100000, which should work in anything. – Corley Brigman Oct 04 '13 at 19:20

Gill Bates · Answer 2 · 2013-10-04T18:10:35.050

No, arguments in Python are passed by reference.
More precisely - variables in Python is just a pointers that stores memory-addresses of actual data. So when Pythons variable-pointer passed to a function - it passed by its value - address pointing to actual data, it means that, variables passed to functions by value and value of variables are references to objects.

More on that topic:

Performance when passing huge list as argument in recursive function?

2 Answers2

Linked

Related