7

This code:

import numpy as np
import cProfile

shp = (1000,1000)
a = np.ones(shp)
o = np.zeros(shp)

def main():
    np.divide(a,1,o)
    for i in xrange(20):
        np.multiply(a,2,o)
        np.add(a,1,o)

cProfile.run('main()')

prints only:

         3 function calls in 0.269 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    0.269    0.269 <string>:1(<module>)
        1    0.269    0.269    0.269    0.269 testprof.py:8(main)
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Prof
iler' objects}

Can I get cProfile to work with numpy to tell me how many calls it makes to the varous np.* calls and how much time it spends on each?

edit

It is too cumbersome to wrap each of the numpy functions individually as hpaulj suggests, so I'm trying something like this to temporarily wrap many or all of the functions of interest:

def wrapper(f, fn):
    def ff(*args, **kwargs):
        return f(*args, **kwargs)
    ff.__name__ = fn
    ff.func_name = fn
    return ff

for fn in 'divide add multiply'.split():
    f = getattr(np, fn)
    setattr(np, fn, wrapper(f, fn))

but cProfile still refers to all them as ff

Paul
  • 42,322
  • 15
  • 106
  • 123

1 Answers1

1

How about wrapping the relevant calls in Python functions?

def mul(*args):
    np.multiply(*args)
def add(*args):
    np.add(*args)

def main():
    np.divide(a,1,o)
    for i in xrange(20):
        mul(a,2,o)
        add(a,1,o)

That's basically the idea in this SO thread about improving profiling granularity - it profiles function calls, not lines.

Does effective Cython cProfiling imply writing many sub functions?

Community
  • 1
  • 1
hpaulj
  • 221,503
  • 14
  • 230
  • 353
  • points, but this would be far too cumbersome for my real-life profiling. I've attempted some automatic wrapping, but am failing (see edit if you want) I even tried this: http://code.activestate.com/recipes/81535-allowing-the-python-profiler-to-profile-c-modules/ but somethings amiss there too. – Paul Dec 07 '13 at 00:55