Stack Overflow
Stack Overflow

Questions tagged [cprofile]

cProfile is a built-in Python module that describes the run time performance of a program, providing a variety of statistics.

cProfile is a built-in module suitable for profiling Python programs. It's designed as a faster drop-in replacement for the profile implementation. cProfile is implemented as C-Extension which provides less overhead than the pure Python implementation of profile, but it may not be available in all Python implementations.

It can be used in a Python file:

import cProfile
cProfile.run('insert_your_code_to_profile_here')

But also from the command-line, to profile a script:

python -m cProfile [-o output_file] [-s sort_order] (-m module | myscript.py)

Sources:

230 questions
1
vote
1 answer

Profile a celery task using cProfile

I've a django app which runs some asynchronous tasks in the background with celery. I'd like to profile the tasks running inside the celery worker. I found these questions but they weren't very useful. How do I profile a specific celery task in a…
Judy T Raj
  • 1,755
  • 3
  • 27
  • 41
1
vote
1 answer

Randomly selected file take longer to load with numpy.load than sequential ones

Context While training a neural network I realized the time spent per batch increased when I increased the size of my dataset (without changing the batch size). The important part is, I need to fetch 20 .npy files per data point, this number doesn't…
Ivan
  • 34,531
  • 8
  • 55
  • 100
1
vote
1 answer

Profile Python cProfile vs unix time

I am profiling a python code ; why does it spend more time in the user space ? user@terminal$ time python main.py 1964 function calls in 0.003 CPU seconds Ordered by: standard name ncalls tottime percall cumtime percall…
vkris
  • 2,095
  • 7
  • 22
  • 30
1
vote
2 answers

How to profile a method with arguments in a loop?

I have a class Foo and it implements a method compute. I would like to see how long compute takes and where it spends most of its time when executing. To resolve the first problem, I used timeit.default_timer in a loop and averaged: import numpy as…
user8087992
1
vote
1 answer

Whats the correct way to pass a function to cProfile in python?

I'm trying to benchmark my code using the cProfile and pstats libraries. This is the code I've got so far: profile = cProfile.Profile() profile.runcall(gillespie_tau_leaping(propensity_calc, popul_num, LHS, stoch_rate, popul_num_all, tao_all,…
Mike5298
  • 377
  • 1
  • 13
1
vote
1 answer

Why does cProfile only run through the code once?

On the flip side, timeit runs through the code 1,000,000 times to get a reasonable asymptotic comparison to other code. cProfile only runs through the code once, and with only 3 decimal places in the results (0.000), it is not enough to get the full…
NoName
  • 9,824
  • 5
  • 32
  • 52
1
vote
0 answers

cProfile - How can I find specific accesses of thread lock variables?

I am profiling my program with python -m cProfile script.py. I am using multiprocessing and it seems that the program is taking a lot of time accessing thread lock variables, because the top of cProfile is: method 'acquire' of '_thread.lock'…
Skinish
  • 65
  • 2
  • 10
1
vote
0 answers

How to run cprofile with already a -m existing in command line?

Usually if you have a script foo.py you can easily profile it as follows: python -m cProfile foo.py However let's imagine I am already running foo as a module with -m: python -m path.foo In this case is there a clean way to profile this execution?…
Evan Pu
  • 2,099
  • 5
  • 21
  • 36
1
vote
0 answers

See top results cProfile

I have a main() function and I am running cProfile.run("main()",sort="cumtime") However, since I have a very large script, I can only see the bottom results and not actually do any profiling. Is there a way to filter for the top bottleneck results…
python_enthusiast
  • 896
  • 2
  • 7
  • 26
1
vote
0 answers

cProfile not showing any (real) results--all 000's--in Python

In running cProfile on anything (for example a mergeSort) I'm getting all 000's in the runtimes, and key lines/vars/methods not listed/tested in the process. Only seems to test under methods, internals. Please advise. below are my results for a…
Zach Oakes
  • 185
  • 14
1
vote
1 answer

How to calculate the average result of several cProfile results?

Instead of only running the profile one time like this: import cProfile def do_heavy_lifting(): for i in range(100): print('hello') profiller =…
Evandro Coan
  • 8,560
  • 11
  • 83
  • 144
1
vote
1 answer

cProfile taking a lot of memory

I am attempting to profile my project in python, but I am running out of memory. My project itself is fairly memory intensive, but even half-size runs are dieing with "MemoryError" when run under cProfile. Doing smaller runs is not a good option,…
Matthew Scouten
  • 15,303
  • 9
  • 33
  • 50
1
vote
1 answer

Determine origin of __call__ in cProfile when wrapping a function's __call__ method

I have a C++ library that I am wrapping and exposing via python. For various reasons I need to overload the __call__ of the functions when exposing them via python. A minimal example is below using time.sleep to mimic functions with various compute…
Alexander McFarlane
  • 10,643
  • 9
  • 59
  • 100
1
vote
1 answer

Django + PostgreSQL - 1000 inserts/sec, how to speed up?

I did some diagnosis and found that on htop: python save_to_db.py takes 86% of the CPU postgres: mydb mydb localhost idle in transaction takes 16% of the CPU. My code for save_to_db.py looks something like: import datetime import django import…
OneRaynyDay
  • 3,658
  • 2
  • 23
  • 56
1
vote
0 answers

How can I profile a python script with many decorated methods?

I'm using snakeviz/cProfile to view the profile data of a slow running python script. However a lot of the longer running methods are wrapped with decorators and so I don't see the name of the function/method in the call but instead I see 'wrapped'…
JSharm
  • 1,117
  • 12
  • 11
1 2 3
…
15 16