Questions tagged [cprofile]

cProfile is a built-in Python module that describes the run time performance of a program, providing a variety of statistics.

cProfile is a built-in module suitable for profiling Python programs. It's designed as a faster drop-in replacement for the profile implementation. cProfile is implemented as C-Extension which provides less overhead than the pure Python implementation of profile, but it may not be available in all Python implementations.

It can be used in a Python file:

import cProfile
cProfile.run('insert_your_code_to_profile_here')

But also from the command-line, to profile a script:

python -m cProfile [-o output_file] [-s sort_order] (-m module | myscript.py)

Sources:

230 questions
20
votes
1 answer

Understanding Pycharm's profiler's results vs. cProfile results and how to get more detail on standard library functions

I am working on driving down the execution time on a program I've refactored, and I'm having trouble understanding the profiler output in PyCharm and how it relates to the output I would get if I run cProfile directly. (My output is shown below,…
Troy D
  • 381
  • 2
  • 4
  • 17
18
votes
2 answers

Cannot import cProfile in Python 3

I am trying to import the cProfile module into Python 3.3.0, but I got the following error: Traceback (most recent call last): File "", line 1, in import cProfile File "/.../cProfile_try.py", line 12, in
alittleboy
  • 10,616
  • 23
  • 67
  • 107
17
votes
1 answer

cProfile adds significant overhead when calling numba jit functions

Compare a pure Python no-op function with a no-op function decorated with @numba.jit, that is: import numba @numba.njit def boring_numba(): pass def call_numba(x): for t in range(x): boring_numba() def boring_normal(): …
Christopher
  • 315
  • 1
  • 8
16
votes
3 answers

What is this cProfile result telling me I need to fix?

I would like to improve the performance of a Python script and have been using cProfile to generate a performance report: python -m cProfile -o chrX.prof ./bgchr.py ...args... I opened this chrX.prof file with Python's pstats and printed out the…
Alex Reynolds
  • 95,983
  • 54
  • 240
  • 345
15
votes
3 answers

Which is the relationship between CPU time measured by Python profiler and, real, user and sys time?

Using the python built-in profiler with a script runninng in one processor (and no multithreading) time python -m cProfile myscript.py the CPU time reported by the profiler is 345.710 CPU seconds 24184348 function calls (24183732 primitive calls)…
corto
  • 287
  • 5
  • 9
14
votes
1 answer

PyPy significantly slower than CPython

I've been testing a cacheing system of my making. Its purpose is to speed up a Django web application. It stores everything in-memory. According to cProfile most of the time in my tests is spent inside QuerySet._clone() which turns out to be…
julx
  • 8,694
  • 6
  • 47
  • 86
13
votes
2 answers

Profiling a system with extensively reused decorators

Our code base has a few decorators that are used extensively. When I create a runtime profile, a large part of the call graph looks like an hour glass; many functions call one function (the decorator), which then calls many functions. This is a…
bukzor
  • 37,539
  • 11
  • 77
  • 111
13
votes
2 answers

Bad Marshal error -- runsnake

I ran cProfile on a python 3 script, worked nicely, then tried to visualize it using runsnake. Howvever, I got an empty screen and the error 'bad marshal data'. I removed .pyc file but that did not work either. The code I used to install runsnake…
kat
  • 163
  • 7
13
votes
5 answers

How to use cProfile with nosetest --with-profile?

nosetest --with-profile --profile-stats-file output The output can't read by runsnake, because nosetest uses hotshot, if I want to generate a file that can be read with runsnake, I need to convert it so: st =…
Virako
  • 650
  • 10
  • 18
12
votes
1 answer

cProfile causes pickling error when running multiprocessing Python code

I have a Python script that runs well when I run it normally: $ python script.py I am attempting to profile the code using the cProfile module: $ python -m cProfile -o script.prof script.py When I launch the above command I get…
James Adams
  • 8,448
  • 21
  • 89
  • 148
12
votes
1 answer

How to improve the code performance if the most time-consuming part comes to posix.waitpid?

I use cprofile to profile a python program. The most time-consuming part turns out to be posix.waitpid. What can I do to improve the code performance? See the screenshot below for the one line of my cprofile result
zell
  • 9,830
  • 10
  • 62
  • 115
12
votes
1 answer

Profile a python script using cProfile into an external file

I am new to python programming.I have a python script and I am trying to profile it using cProfile command. I typed the following python -m cProfile -o readings.txt my_script.py It generated readings.txt. But when I try to open the file using any…
user3217603
  • 123
  • 1
  • 4
11
votes
1 answer

What is correct way to use cProfile with asyncio code?

I'm trying to determine how to correctly use cProfile and pstats with asyncio code. I am profiling my script by running cProfile.run('loop.run_until_complete(main())', 'profile.stats'). After using pstats to sort by SortKeys.TIME, I get the…
J. Taylor
  • 4,567
  • 3
  • 35
  • 55
11
votes
3 answers

Find bottleneck of flask application

I wrote a flask application. I found it very slow when I deployed it in a remote server. So, I did some profiling practices with it. Please take a look at the pictures below: The code I use to profiling is: #coding: utf-8 from…
holys
  • 13,869
  • 15
  • 45
  • 50
10
votes
0 answers

Profiling Python in a docker container is very slow (cProfile and pyinstrument)

I'm trying to profile some very simple code (using both cProfile and pyinstrument). The code is: sum(1 for e in range(1533939)) When running the code without the profiler active, it is very quick (~85ms). However when attempting to run the same…
Edan Maor
  • 9,772
  • 17
  • 62
  • 92
1
2
3
15 16