how to store ipython magic output into variable

Question

I am a python and Ipython beginner. This may be a trivial question. It is probably duplicated with other questions. However I do not know what key words I should search.

I have already known how to interactive with shell.

For example:

In [1]: a = !ls
In [2]: a
        ...same ls result as shell...
In [3]: type(a)
Out[3]: IPython.utils.text.SList

However, how to interactive with Ipython magic?

For example

In [1]: a = %history -t 
        ...Ipython result...
In [2]: a
In [3]: type(a)
Out[3]: NoneType

Answer 1

For the history command, specifically, the simplest solution is

In [243]: history -t -f history.txt
In [244]: with open('history.txt') as f:
   .....:     HIST = [l.strip() for l in f]
   .....:     

In [245]: len(HIST)
Out[245]: 258

In [246]: HIST[-1]
Out[246]: "get_ipython().magic(u'history -t -f history.txt')"

In [247]: 

Basically, dump it to a file and read it back in.

This may seem a kludge, but I suspect it comes from the nature of IPython. It isn't actually an interpreter, but instead is a command line shell for the underlying interpreter. My suspicion is that the magic commands are handled inside IPython and do not go through the normal path of passing the command to the interpreter, capturing the output, and storing it in the command history as Out[n]. So it is not available for recall and assignment.

The alternative is that get_ipython().magic simply returns None.

Either way, the screen output d=for %history is not available. You have to dump it to a file.

It seems to vary per magic command. alias, for example, does return the screen output

In [288]: a=%alias
Total number of aliases: 17

In [289]: a
Out[289]: 
[('cat', 'cat'),
 ('clear', 'clear'),
 ('cp', 'cp'),
 ('ldir', 'ls -F -G -l %l | grep /$'),
 ('less', 'less'),
 ('lf', 'ls -F -l -G %l | grep ^-'),
 ('lk', 'ls -F -l -G %l | grep ^l'),
 ('ll', 'ls -F -l -G'),
 ('ls', 'ls -F -G'),
 ('lx', 'ls -F -l -G %l | grep ^-..x'),
 ('man', 'man'),
 ('mkdir', 'mkdir'),
 ('more', 'more'),
 ('mv', 'mv'),
 ('rm', 'rm'),
 ('rmdir', 'rmdir'),
 (u'show', u'echo')]

In [290]: 

Answer 2

Im working on an ipython reload project and want to have a quick way to select from previous %run statements. My solution was the following.

import os

histvar = os.popen("ipython -c 'history -g'").read()

#regex match / do stuff here

Answer 3

with line magic, you can use result = %lsmagic to get result into variable; with cell magic, thanks to ipython, you can use _ to get result, for example:

%%some_magics
balabala
balabala

a = _

Answer 4

At least for the %history command, the output is written to stdout, so by redirecting that to a StringIO, you can capture the output without any temporary file, like this:

@register_line_magic
def get_magic_out(command):
    ipy = get_ipython()
    out = io.StringIO()

    with redirect_stdout(out):
        ipy.magic(command)

    return out.getvalue()

Gist: get_magic_out.py

Which you can then use like this:

In [1]: import get_magic_out as _ # We don't actually use the module, because of the `@register_line_magic` decorator

In [2]: x = %get_magic_out history

In [3]: x
Out[3]: 'import get_magic_out\nx = %get_magic_out history\n'

Answer 5

I know this is a really old question but this feature is quite poorly documented so I thought I'd at least try to document how I did it here and save some other people the same troubles I had. TLDR: use @needs_local_scope and update the local_ns dictionary with your values.

For a more detailed solution, I wanted to return dynamic variables where each was a pandas dataframe. In the end, this is what my working Juypter notebook looks like:

In  [1]: %load_ext faker_maker

In  [2]: %%build_fake_dataframes
         names
         ----
         first_name
         last_name

In  [3]: names
Out [3]:
            first_name    last_name
         0        Fred        Smith
         1      George         Wood

To get the above to work, I have another file called faker_maker.py with the following content:

from IPython.core.magic import (Magics, magics_class, register_cell_magic,
                                line_cell_magic, needs_local_scope)
import pandas as pd

class FakeDataFrameBuilder():
    dataframes = {}

    def __init__(text):
        ...

    def parse():
        ...

@magics_class
class AutoMagics(Magics):
    @needs_local_scope
    @line_cell_magic
    def build_fake_dataframes(self, line, cell, local_ns=None):
        cls = FakeDataFrameBuilder(cell)
        cls.parse()
        for name, df in cls.dataframes.items():
            local_ns[name] = df

def load_ipython_extension(ipython):
    ipython.register_magics(AutoMagics)

I also have this important tip:

• Every time you change your *.py file, be sure to restart your kernal in Jupyter. This is needed so the %load_ext can reload your file.

Answer 6

Q: How to store/capture ipython magic output into variable?

A: With %%capture magic (to capture whatever the cell outputs, whether is another magic or not)

Example for your case:

A simpler example:

Code:

Generating cell

%%capture other_stuff_captured
!date
print("hello there!")

Any other cell

str(other_stuff_captured)
print("---------------------")
print(str(other_stuff_captured))

Also, as 李东泽 suggested, for non-magic commands, the output of the last one is on _ (+Info)