Profiling for a program using python-eve server

Question

I am running a backend python-eve server with multiple functions being called to provide one service. I want to do profiling for this python backend server. I want to find out which among the multiple functionalities is taking time for execution. I have heard and used cprofiler but for a server that is continuously running, how do I do profiling? Moreover, I am using Pycharm IDE to work with the python code. So, it will be beneficial if there's a way I can do profiling using Pycharm.

Answer 1

While I do not have direct experience with python-eve, I wrote pprofile (pypi) mainly to use on Zope (also a long-running process).

The basic idea (at least on processes using worker threads, like Zope) is to start pprofile in statistic mode, let it collect samples for a while (how long heavily depends on how busy the process is and the level of detail you want to capture in the profiling result), and finally to build a profiling result archive, which in my case contains both the profiling result and all executed source code (so I'm extra-sure I'm annotating the correct version of the code).

I have so far not needed to do extra-long profiling session, like do one query to start profiling, then later another query to stop and fetch the result - or even to keep the result server-side and browse it somehow, and how to do this will likely heavily depend on server details.

You can find the extra customisation for Zope (allowing to fetch the python source out of Zope's Python Scripts, TAL expressions, and beyond pure python profiling it also collects object database loading durations, and even SQL queries run, to provide a broader picture) in pprofile.zope, just to give you an idea of what can be done.