Questions tagged [cprofile]

cProfile is a built-in Python module that describes the run time performance of a program, providing a variety of statistics.

cProfile is a built-in module suitable for profiling Python programs. It's designed as a faster drop-in replacement for the profile implementation. cProfile is implemented as C-Extension which provides less overhead than the pure Python implementation of profile, but it may not be available in all Python implementations.

It can be used in a Python file:

import cProfile
cProfile.run('insert_your_code_to_profile_here')

But also from the command-line, to profile a script:

python -m cProfile [-o output_file] [-s sort_order] (-m module | myscript.py)

Sources:

230 questions
1
vote
0 answers

Profiling Django: what is {posix.write} function doing?

Profiling Django app to figure out slow functions. I just added some middleware to track function calls, following this blog: http://agiliq.com/blog/2015/07/profiling-django-middlewares/ and I see that the entry of cProfile stats for {posix.write}…
MrE
  • 19,584
  • 12
  • 87
  • 105
1
vote
1 answer

How to know if a python programming is running under a profiler?

Is there any methods that can be used to judge whether a Python script is currently running under profiler? I have a program with GUI. I want to skip the GUI part when running under profiler so that the result is more accurate.
iuradz
  • 1,128
  • 1
  • 12
  • 25
1
vote
0 answers

How to profile python script run as "named module"?

How to profile a python module runs by command python -m foo.bar? Seems that python -m foo.bar -m cProfile -o test.out is totally weird, and doesn't work here.
Wizmann
  • 839
  • 1
  • 9
  • 14
1
vote
1 answer

How to output a .cprofile file from script

I'm using the cProfiler in my script. I'd like to output the .cprofile file with results. I want to get the output file that is not human readable and later open it in snakeviz. Is that possible? I cannot use the python -m cProfile -o program.prof…
Kowalski Paweł
  • 594
  • 1
  • 6
  • 27
1
vote
0 answers

Multiple kernels in Spyder - Python3

I need to get source code of about 4 thousand of web pages, and extract a few numbers from it. I achieve this using urllib and .split(), store it in a dataframe and export to csv. After running cProfile: ncalls tottime percall cumtime percall…
Mathbreaker
  • 33
  • 1
  • 1
  • 6
1
vote
2 answers

Problems trying to use cProfile

I am trying to run the code below at Python 2.7 GUI: python -m cProfile -s time abc.py However here is the error I have: >>> python -m cProfile -s time abc.py >>> ^ >>> SyntaxError: invalid syntax Any idea how can I solve it?
Alex Luk
  • 41
  • 9
1
vote
1 answer

Correct Way To Use cProfile

I want to profile some functions that I've written. I'm looking at the docs for cProfile, and I don't understand what (if any) the difference between the following two pieces of code is: import cProfile profile = cProfile.Profile() result =…
Batman
  • 8,571
  • 7
  • 41
  • 80
1
vote
0 answers

Python multithreaded profiling for method that uses background process workers

I'm trying to find the best way to profile a method that uses redis queue to set a number of jobs for running workers, waits till they're done, and returns. I tried using cProfile, but it just profiles what's going on in the main thread. Is there a…
zoopz
  • 1,099
  • 2
  • 9
  • 10
1
vote
0 answers

cProfile shows significant tottime in function which only exclusively calls an external Cython function

I have a function that calls a function from a compiled Cython library multiply_vec as follows: def ScorePair(self): return multiply_vec.multiply_v2(self.x,self.y) When running cProfile I get the following output: ncalls tottime percall …
hirantal91
  • 11
  • 1
1
vote
0 answers

What can I improve considering these cProfile results?

I created an advanced snake game in python/pygame. The problem is that the game has gotten to the point that when I put the new version on our family computer so everyone can play it, it runs too slowly. I ran the gameplay with cprofile and got the…
Thomas Hauser
  • 55
  • 1
  • 10
1
vote
1 answer

cProfile on a function that uses Argument Parser

I am trying to run cProfile on a Python script that uses an ArgumentParser to parse arguments given from the Command Line. Amongst the arguments, there are two file names that are used by the script for data input. I was wondering whether I am able…
user2273009
  • 21
  • 1
  • 3
1
vote
1 answer

Why is this lambda involving a list.index() call so slow?

Using cProfile: ncalls tottime percall cumtime percall filename:lineno(function) 1 0.000 0.000 17.834 17.834 :1() 1 0.007 0.007 17.834 17.834 basher.py:5551(_refresh) 1 0.000 …
Mr_and_Mrs_D
  • 32,208
  • 39
  • 178
  • 361
1
vote
1 answer

Profiling a Python program that calls parallel external programs

Similar to this question on profiling multiprocessing code... I need to profile my codebase, and I'd planned to use cProfile. This program is essentially a genetic algorithm that runs an evaluation function in parallel. The kicker is that this…
erik
  • 3,810
  • 6
  • 32
  • 63
1
vote
1 answer

Python profile for multiple inputs and average result

I have several different inputs to a Python script, which I invoke in the form: ./myscript.py myfile1 ./myscript.py myfile2 ./myscript.py myfile3 ... I can profile the code on a per-function basis for any one input file using python -m cProfile -s…
Douglas B. Staple
  • 10,510
  • 8
  • 31
  • 58
1
vote
1 answer

String searching with ord is slower than with 'in' method

I have two version of some simple text parser (it validates login correctness): rgx = re.compile(r"^[a-zA-Z][a-zA-Z0-9.-]{0,18}[a-zA-Z0-9]$") def rchecker(login): return bool(rgx.match(login)) max_len = 20 def occhecker(login): …
senior_pimiento
  • 702
  • 1
  • 5
  • 15