53

I have two different Jupyter notebooks, running on the same server. What I would like to do is to access some (only a few of them) of the variables of one notebook through the other notebook (I have to compare if the two different versions of the algorithm give the same results, basically). Is there a way to do this?

Thanks

user22710
  • 633
  • 1
  • 5
  • 5

3 Answers3

80

Between 2 jupyter notebook, you can use the %store command.

In the first jupyter notebook:

data = 'string or data-table to pass'
%store data
del data # This will DELETE the data from the memory of the first notebook

In the second jupyter notebook:

%store -r data
data

You can find more information at here.

Kane Chew
  • 3,693
  • 4
  • 12
  • 24
chabir
  • 2,108
  • 1
  • 11
  • 13
  • is the first del going to delete the content of the variables in the first nb or just the %store? – giammi56 Nov 03 '20 at 13:53
  • it will delete the variable in the first nb not in %store but you can always recover it in the first nb using %store -r data – chabir Nov 04 '20 at 14:19
  • 1
    `%store data` is using additional memory. Would be great if the second notebook can just point to the memory that was already used by the first notebook. – kjsr7 Jan 15 '21 at 14:11
  • @JayaKommuru: there is maybe another way but if the first notebook save the data as additional memory, you can close the notebook (which will kill the first allocated memory for the variable) and later, open a different notebook and retrieve the variable on the "additional" memory which is often really handy! – chabir Feb 01 '21 at 23:35
  • 1
    is it obligatory to delete the variable in the first notebook? – BND Feb 09 '21 at 15:02
  • 1
    @BND, as far I know and tested, it is not required. – chabir Feb 09 '21 at 20:29
  • 1
    After 30 mins of search this was by far the easiest way to do it for a jupyter notbook. Which is why I normally only use jupyter for data analytics. Thank you – JQTs Aug 18 '21 at 22:32
10

If you only need something quick'n dirty, you can use the pickle module to make the data persistent (save it to a file) and then have it picked up by your other notebook. For example:

import pickle

a = ['test value','test value 2','test value 3']

# Choose a file name
file_name = "sharedfile"

# Open the file for writing
with open(file_name,'wb') as my_file_obj:
    pickle.dump(a,my_file_obj)   

# The file you have just saved can be opened in a different session
# (or iPython notebook) and the contents will be preserved.

# Now select the (same) file to open (e.g. in another notebook)
file_name = "sharedfile"
# Open the file for reading
file_object = open(file_Name,'r')  
# load the object from the file into var b
b = pickle.load(file_object)  

print(b)
>>> ['test value','test value 2','test value 3']
alfredoc
  • 126
  • 6
2

You can use same magic commands to do this.The Cell magic: %%cache in the IPython notebook can be used to cache results and outputs of long-lasting computations in a persistent pickle file. Useful when some computations in a notebook are long and you want to easily save the results in a file.

To use it in your notebook, you need to install the module ipycache first as this Cell magic command is not a built-in magic command.

then load the module in your notebook:

%load_ext ipycache

Then, create a cell with:

%%cache mycache.pkl var1 var2
var1 = 1  # you can put any code you want at there,
var2 = 2  # just make sure this cell is not empty.

When you execute this cell the first time, the code is executed, and the variables var1 and var2 are saved in mycache.pkl in the current directory along with the outputs. Rich display outputs are only saved if you use the development version of IPython. When you execute this cell again, the code is skipped, the variables are loaded from the file and injected into the namespace, and the outputs are restored in the notebook.

Alternatively use $file_name instead of mycache.pkl, where file_name is a variable holding the path to the file used for caching.

Use the --force or -f option to force the cell's execution and overwrite the file.

Use the --read or -r option to prevent the cell's execution and always load the variables from the cache. An exception is raised if the file does not exist.

ref: The github repository of ipycache and the example notebook

hxysayhi
  • 1,888
  • 18
  • 25