0

I am considering profiling a Python application in production (such as a Django website). I've found many options that self-describe as lightweight and demonstrate how they are used (including cProfile, vmprof, yappi and DTrace/SystemTap) but I am struggling to get a sense of what kind of negative performance impact I can expect while the profiler is in operation. This would be an important factor in identifying whether a particular tool is suitable for profiling in production.

Is it possible to anticipate the performance impact of a particular profiler, without experimentation in a test environment?

lofidevops
  • 15,528
  • 14
  • 79
  • 119

1 Answers1

0

A profiler will not in itself speed up you program what it does is tell you where the program spends most time. Suppose that you profiler tells you that the program is in func1 1% of the time and in func2 80% of the time. If you make func1 run twice as fast you will speed up your progrram by about .5%. If you make func2 run twice as fast you will speed up you program by 40%. Spend your time and effort on func2.

How do you make functions faster? There is no simple answer, but there are lots of sites that talk about this (search for Python Performance). What sort of performance increase can you expect? Again there is no simple answer. My rule of thumb is a factor of two increase, but your mileage will probably vary?

William
  • 324
  • 1
  • 8
  • Using a profiler will of course make the program run slower. But profilers go to a lot of trouble to discount their own activities when they report on the program being profiled. Unless you have a really long-running program, do you really care if the profiler makes it run 10% or 50% or 100% slower? Once you have the profiling results you will be switching the profiling off anyway. As in all such things, TANSTAAFL. Profilers that report more detail can be expected to take longer doing it. – BoarGules Feb 10 '22 at 22:08
  • @William thanks, I've updated the question to clarify that I don't expect the profiler to magically speed things up, rather that I expect the application to slow down while the profiler is running – lofidevops Feb 10 '22 at 22:15
  • @BoarGules I agree in principle; I'm imagining, for example, a Django website running in production with variable load, maybe I should put that in the question to make it more concrete – lofidevops Feb 10 '22 at 22:16