python how to memoize a method

Question

Say I a method to create a dictionary from the given parameters:

def newDict(a,b,c,d): # in reality this method is a bit more complex, I've just shortened for the sake of simplicity
    return { "x": a,
             "y": b,
             "z": c,
             "t": d }

And I have another method that calls newDict method each time it is executed. Therefore, at the end, when I look at my cProfiler I see something like this:

17874 calls (17868 primitive) 0.076 CPU seconds

and of course, my newDict method is called 1785 times. Now, my question is whether I can memorize the newDict method so that I reduce the call times? (Just to make sure, the variables change almost in every call, though I'm not sure if it has an effect on memorizing the function)

Sub Question: I believe that 17k calls are too much, and the code is not efficient. But by looking at the stats can you also please state whether this is a normal result or I have too many calls and the code is slow?

Answer 1

1. You mean memoize not memorize.
2. If the values are almost always different, memoizing won't help, it will slow things down.
3. Without seeing your full code, and knowing what it's supposed to do, how can we know if 17k calls is a lot or the little?

Answer 2

If by memorizing you mean memoizing, use functools.lru_cache. It's a function decorator

Answer 3

The purpose of memoizing is to save a result of an operation that was expensive to perform so that it can be provided a second, third, etc., time without having to repeat the operation and repeatedly incur the expense.

Memoizing is normally applied to a function that (a) performs an expensive operation, (b) always produces the same result given the same arguments, and (c) has no side effects on the program state.

Memoizing is typically implemented within such a function by 'saving' the result along with the values of the arguments that produced that result. This is a special form of the general concept of a cache. Each time the function is called, the function checks its memo cache to see if it has already determined the result that is appropriate for the current values of the arguments. If the cache contains the result, it can be returned without the need to recompute it.

Your function appears to be intended to create a new dict each time it is called. There does not appear to be a sensible way to memoize this function: you always want a new dict returned to the caller so that its use of the dict it receives does not interfere with some other call to the function.

The only way I can visualize using memoizing would be if (1) the computation of one or more of the values placed into the result are expensive (in which case I would probably define a function that computes the value and memoize that function) or (2) the newDict function is intended to return the same collection of values given a particular set of argument values. In the latter case I would not use a dict but would instead use a non-modifiable object (e.g., a class like a dict but with protections against modifying its contents).

Regarding your subquestion, the questions you need to ask are (1) is the number of times newDict is being called appropriate and (2) can the execution time of each execution of newDict be reduced. These are two separate and independent questions that need to be individually addressed as appropriate.

BTW your function definition has a typo in it -- the return should not have a 'd' between the return keyword and the open brace.