45

I have a simple task: in addition to measuring the time it takes to execute a chunk of code in Python, I need to measure the amount of memory a given chunk of code needs.

IPython has a nice utility called timeit which works like this:

In [10]: timeit 3 + 3
10000000 loops, best of 3: 24 ns per loop

What I'm looking for is something like this:

In [10]: memit 3 + 3
10000000 loops, best of 3: 303 bytes per loop

I'm aware that this probably does not come built in with IPython—but I like the timeit-memit analogy.

Erik Kaplun
  • 37,128
  • 15
  • 99
  • 111
  • 1
    I'll take a look, thanks; the solution there looks quite simple. And good point about "best"—but timeit also gives you the best, and you might say you're really (sometimes) interested in the worst time. Re. IPython magic: I've found http://blog.vene.ro/2012/06/30/quick-memory-usage-benchmarking-in-ipython/ – Erik Kaplun Sep 30 '13 at 11:18
  • The question that this question has been marked as duplicate of is about memory profilers in Python in general—this one here asks about IPython; so I don't see how it's really a duplicate... The other question has no mentions of `timeit`, `memit` or even "ipython" whatsoever. – Erik Kaplun Oct 01 '13 at 09:08

1 Answers1

62

In fact, it already exists, as part of the pragmatically named memory_profiler package:

In [2]: %memit np.zeros(1e7)
maximum of 3: 76.402344 MB per loop

More info at https://github.com/pythonprofilers/memory_profiler#ipython-integration

Edit: To use this, you first need to load it as an IPython extension:

%load_ext memory_profiler

To make IPython always load the memory_profiler extension upon startup, add it to the c.InteractiveShellApp.extensions list in your profile's ipython_config.py:

$ grep -C2 c.InteractiveShellApp.extensions ~/.ipython/profile_default/ipython_config.py
 # A list of dotted module names of IPython extensions to load.
 #
 c.InteractiveShellApp.extensions = [
   'autoreload',
   'memory_profiler',
Motsah
  • 5
  • 2
Thomas K
  • 39,200
  • 7
  • 84
  • 86
  • 3
    having done `pip install memory_profiler`, `In [2]: %memit 3` gives me `ERROR: Line magic function `%memit` not found`. – Erik Kaplun Oct 01 '13 at 09:10
  • 2
    There's an IPython specific command to load an extension. I've added it to the answer. – Thomas K Oct 01 '13 at 21:42
  • 6
    I also needed to do `pip install memory_profiler` prior to `%load_ext memory_profiler` working in my Jupyter Notebook. – kevin_theinfinityfund Jul 03 '20 at 06:36
  • If the config file doesn’t already exist, run `ipython profile create` in a terminal. – ash Dec 29 '21 at 19:03
  • It's important to note that I believe %%memit only measures the memory difference (increase) from before the cell contents was run, and afterwards. Therefore it does not actually measure the peak memory usage during the execution of the cell contents – Finn Andersen Apr 12 '23 at 14:42