4

I've been testing some optimisations to a piece of code (specifically, whether elif n in [2,3] is faster than elif n == 2 or n == 3) and noticed something strange.

Using timeit.default_timer I did several runs of each version of the function, and the first run was always significantly slower than subsequent ones (values starting off around 0.01 of a second that trailed off to consistently around 0.003).

Is python doing something behind the scenes to optimise the code on later runs? This isn't really a problem by any means, but I'd be interested to know what is happening (if anything)

twigonometry
  • 166
  • 1
  • 9
  • It's likely due to some kind initialization tha takes place the first time something is accessed (at the OS or hardware level) This is why I almost always use `min(timeit.repeat(...)` when doing timings. – martineau Dec 23 '19 at 15:51
  • You're supposed to use `timeit.timeit`, not run things manually and call `timeit.default_timer` yourself. – user2357112 Dec 23 '19 at 15:52

3 Answers3

2

There's no such general optimization on CPython, the reference Python implementation. There are a variety of more specific things that could be happening, but we can't tell what.

Marcos's answer suggests it's pyc file creation, but that's not how timeit works, even if you call timeit.default_timer yourself (which you shouldn't - you should be using timeit.timeit or timeit.repeat or other such mechanisms).

pyc files are created when a module is imported that does not have a pyc file, or whose pyc file is out of date. They are not created for timeit snippets, and even if your timed code comes from an imported module, typical timeit usage patterns will import the module before timing starts.

You're calling timeit.default_timer instead of letting timeit handle things the way it's designed to work, but even then, any pyc file creation is unlikely to happen within the timed code.

PyPy, an alternate Python implementation, uses JIT compilation, but you'd probably know if you were on PyPy.

Numba, a library used to accelerate numeric computation, has its own JIT mechanisms, which could also cause speedup after the first run. It's easier to depend on Numba without noticing than to run on PyPy without noticing.

Memory allocation might happen faster on subsequent runs, depending on what types you're using and details of how they interact with the memory management system, as well as how your malloc behaves. For example, there might be free lists with more memory blocks on them after the first run.

There are other possibilities, but ultimately, we can't really tell what's happening.

user2357112
  • 260,549
  • 28
  • 431
  • 505
0

One major consideration is that the over heads for imports may only impact the first round in a timeit call.

Consider this example:

from timeit import timeit

for _ in range(5):
    print(timeit('import requests', number=1))

The output will be something like:

0.1009
1.8307e-05
1.6907e-05
1.6800e-05
1.6817e-05

Little back ground on imports, first time an import is encountered there's a bit of work done to add it to the namespace, subsequent imports of the same module are pretty much a no-op. The results are indicating that 'requests' is loaded for the first timeit call. Printing globals() before and after timeit calls confirms this. Inserting 'import requests' at the top of the module will cause the first result to be on par with the others, again confirming that theory but not always a practical solution.

Increasing the number of rounds will reduce the impact of the first rounds but it's probably still going to be significant. In this case, number=100000 gave 0.12 (0.10 + 1.70e-5*100000) for the first call and 0.017 (1.70e-5*100000) for the others.

For what it's worth, I have a throw away number=1 call to timeit and then get on with it.

Guy Gangemi
  • 1,533
  • 1
  • 13
  • 25
-1

Yes, python caches the pyc file after the first run, so if the code doesn't change it will run faster in next iterations because it won't need to compile it to byte code again. Remember python is an interpreted language, it's just skipping an interpretation step.

marcos
  • 4,473
  • 1
  • 10
  • 24
  • That's a thing Python does, but it doesn't apply to `timeit`. – user2357112 Dec 23 '19 at 15:40
  • So does timeit account for this in a similar way that it accounts for garbage collection etc? – twigonometry Dec 23 '19 at 15:43
  • mmm, it's very hard to tell how garbage collection impact on your readings. @twigonometry – marcos Dec 23 '19 at 15:45
  • Sorry, I don't mean just in my code - I just meant generally is it one of the things it 'avoids' timing (but you clarified this in your answer anyway when you said it starts timing after the pyc file is imported) – twigonometry Dec 23 '19 at 15:58