1

My research requires processing memory traces of applications. For C/C++ programs, this is easy using Intel's PIN library. However, as suggested here Use Intel Pin to instrument Python scripts, I may need to instrument the Python runtime itself, which I'm not sure will represent the true memory behavior of a given python script due to some overheads(If this is not the case, please comment). Some of the existing python memory profilers only talk about the runtime memory "usage" in terms of the heap space usage, etc.

I ended up making an executable from my python script using PyInstaller and running my PINTool over it. However, I'm not sure if this is the right approach.

Is there any way(any library or any hack into the python runtime) that may help in getting the memory traces accessed by the python scripts?

Harsh Kumar
  • 97
  • 6
  • #1 A python script must be run by a python interpreter (using tools such as Pyinstaller doesn't change that fact: they all embed the interpreter and extract the needed scripts). Ofc, instrumenting the interpreter will lead to a lot of instrumented code that is not specific to your script (mostly the interpreter initialization), but besides that you're actually instrumenting the behavior of the interpreter while it's running your script. – Neitsa Apr 20 '21 at 13:33
  • #2 If you'd like to instrument only the behavior of the interpreter while running your script try find in which interpreter's function the script starts its execution and in which the script actually reach its end. transpose that in PIN (if you see the start function, then start instrumenting, etc.). If you want to push further the level of detail, try to understand how and where the interpreter JIT the python code to native code and instrument only the JITed code. – Neitsa Apr 20 '21 at 13:37
  • Thanks for your comment. Regarding PyInstaller, however, I have noticed one thing. When I run my PinTool over the binary package, it runs faster and the output generated is orders of magnitude less as compared to when I pass the python interpreter along with the file name. – Harsh Kumar Apr 22 '21 at 06:26

0 Answers0