14

I have Jupyter running in a tmux session on an ec2 instance. I have very long-running cells, but when I close my browser or laptop lid, the notebook no longer writes output cells (and may crash the python kernel).

This is how I launch labs on my remote instance:

jupyter lab --ip=0.0.0.0 --port=5002 --no-browser --allow-root

I'm looking for a solution to run a notebook indefinitely without losing data and without having to keep my local computer on.

  • I don't want to use a VNC or X-windows forwarding (too slow)
  • I don't want to rewrite my code into python scripts (need work only in jupyter labs)

There has got to be a solution out there!

Update:

The 'nohup' solution below doesn't work:

enter image description here

After running this cell and closing the browser, upon reopening there is no output:

enter image description here

M.R.
  • 1,053
  • 2
  • 13
  • 30

2 Answers2

9

EDIT (after clarification):

You can use some Jupyter magic to continue running the cell after you close the browser or laptop, and then print the output after you return. Here's how that's done:

%%capture stored_output

import time
time.sleep(30)
print("Hi")

After returning, run the following:

stored_output.show()
# Hi

ORIGINAL:

You need to launch the notebook with nohup.

nohup jupyter notebook &

Adding the '&' is only needed to return the shell. The notebook will be running in the background and it's process ID won't be killed when you close your SSH connection.

desertnaut
  • 57,590
  • 26
  • 140
  • 166
samredai
  • 662
  • 3
  • 8
3

I don't think what you want is possible because when your laptop goes into sleep mode after closing the lid, the notebook client in your browser will stop interacting with the server, even if you managed to keep the SSH connection alive. You could change your OS settings to prevent your system from sleeping upon lid closing, but that would be no different than keeping your machine on and using the battery.

The approach I use is:

  • Start your remote Jupyter server in screen or tmux so the server process and Python kernel stay running if the SSH connection drops.
  • Write your long running cells in a way that your outputs either will not be written to stdout or will be logged into a file in the server.

When you awake your machine and restart the SSH connection, the cell will have finished running and you can inspect the results in another cell or look at your logs directly.

foglerit
  • 7,792
  • 8
  • 44
  • 64