3

I'm very new to GCP/AI Platform/Git/etc., so please bear with me.

I am trying to connect an AI Platform Notebook & VM to a Google Cloud Source Repository. I am not the only one at my organization who will be working on the code in the repo, so it's important that multiple people maintain access to it. My goal is to be able to instantiate a Notebook/VM in AI Platform Notebooks, fork/clone the repo, make changes as needed, and push back to my fork of the repo (eventually merging with master branch down). I would also like to have my VM persist, spinning it up and down as needed, pulling and pushing from/to the repo at will.

Surprisingly, I have found no documentation on connecting AI Platform with Cloud Source Repositories.

When I try to clone a repo in CSR, it asks for SSH credentials but I am not very familiar with that process. I was able to generate an RSA key pair from the SSH command line for my particular VM instance, but I don't know what to do from there (most importantly, I want to be sure that multiple people can fork/clone the master repo).

Thanks in advance for any help you can provide.

magnawhale
  • 257
  • 3
  • 11
  • Have you tried registering the VM's public key in Cloud Source Repos -> more -> manage ssh keys -> register? Public key is one ending with ".pub" in ".ssh" folder on VM. – Emil Gi Mar 30 '20 at 09:19

2 Answers2

5

Just an update, I found a method that works (I'm sure there are alternatives):

HOW TO CLONE a Cloud Source Repository into a GCP AI Platform Notebook instance:

  1. Instantiate an AI Platform Notebook as you normally would, then Open JupyterLab
  2. In another window, open Cloud Source Repositories, click on All Repositories, click on the name of the repo you wish to clone.
  3. Click on the + Clone button in the top-right corner, then click on How to setup?
  4. In the pop-up, click on Manually generated credentials, then click Generate and store your Git credentials
  5. In the new window, copy the second code block (bottom of page, in blue box)
  6. Back in your JupyterLab instance, open a terminal window and paste the code block in the terminal, hit ENTER
  7. in the same terminal, run git clone https://source.developers.google.com/p/<REST_OF_REPO_LOCATION_HERE> , being sure to use your repo's location
  8. You’re done! You can now use git like you normally would from the terminal, or even from the Git UI by going to Git > Git Interface
magnawhale
  • 257
  • 3
  • 11
3

Surprisingly, I have found no documentation on connecting AI Platform with Cloud Source Repositories

According to the official documentation Working with notebooks:

When you run datalab create VM-instance-name for the first time, it adds a datalab-notebooks Cloud Source Repository in the project (referred to, below, as the "cloud remote repo"). This is a remote repository for the /content/datalab/notebooks git repository created in the docker container running in your Cloud Datalab VM instance (referred to, below, as the "Cloud Datalab VM repo"). You can browse the cloud remote repo from the Google Cloud Console Repositories page.

You can use git or ungit to manage the notebooks in the Cloud Datalab VM repo.

EDIT:

I created a Notebook instance (I am the owner of my project) and I was able to clone one repository from my project using gcloud sdk:

gcloud source repos list
gcloud source repos clone your-repository
cd your-repository

Quickstart: Create a repository

Community
  • 1
  • 1
marian.vladoi
  • 7,663
  • 1
  • 15
  • 29
  • That's the documentation for Datalab which, after reading, doesn't seem to function the same way as AI Platform Notebooks. If I'm misreading this, please let me know. Also, for my case I need to clone a pre-existing repo, not create a new one when I create a new notebook. The closest thing I've found to what I need is documentation for connecting AI Platform Notebooks to GitHub ( https://cloud.google.com/ai-platform/notebooks/docs/save-to-github ) but strangely not Google's own version control system. – magnawhale Mar 24 '20 at 17:30
  • `gcloud source repos clone` works in Vertex AI Notebooks as well, and is far easier than the currently top-voted answer that suggests using `Manually generated credentials` – jb_ml_eng Jan 19 '23 at 01:39