0

Running this code cell

%%bash
touch /var/tmp/hello_world
ls -l /var/tmp/hello_world

inside a Google Cloud Datalab (i.e. Jupyter) notebook leaves me wondering: where does the file actually reside.

  • It is not stored as /var/tmp/hello_world locally where I run the browser.
  • It is apparently not stored as /var/tmp/hello_world on the VM instance that hosts my instance of Datalab (i.e. Jupyter). I double-checked with ssh.
  • It is apparently not stored inside the notebook's .ipynb file. (That file is apparently stored in /mnt/disks/datalab-pd/content/ on the cited VM instance.)

So where does the file actually reside?

Drux
  • 11,992
  • 13
  • 66
  • 116

1 Answers1

2

This is a location within the Datalab container running on the Datalab VM. Hope that helps.

Anything you want to hold on to should probably be written to /content which is mapped to the PD mounted on the host VM (a data disk, separate from the boot disk). For example notebooks.

For actual data, the PD might not be a great location, esp. if its large. For that, you should look to GCS.

Nikhil Kothari
  • 5,215
  • 2
  • 22
  • 28
  • First I don't think you can ssh into the container since it isn't running sshd. I think /tmp is also mounted, and /var/tmp might use that (not 100% sure). You can see how the docker container is launched: https://github.com/googledatalab/datalab/blob/master/tools/cli/commands/create.py#L213 – Nikhil Kothari Dec 27 '17 at 03:14
  • Thx, I get it now: So if I `ssh` into the Datalab VM and then enter the Datalab [container](https://github.com/googledatalab/datalab/blob/master/tools/cli/commands/create.py#L213) with `docker exec -it datalab bash`, this is where I can see the file with `ls -l /var/tmp/hello_world`. Your point about PD is also well taken, this was just an experiment. – Drux Dec 27 '17 at 03:27