Google Cloud Build Trigger - Problem Fetching Submodules

Question

I have source code hosted in Google Cloud Source Repositories. It has a single git submodule which is also hosted in Google Cloud Source Repositories (in the same GCP project). The .gitmodules file looks something like this:

[submodule "src/my-repo"]
    path = src/my-repo
    url = ssh://source.developers.google.com:2022/p/my-project/r/my-repo

I have a Google Cloud Build trigger configured, but the build is failing because the git submodule is not present (it seems that it's an ongoing shortcoming that a clone of a Cloud Source Repositories repo doesn't init and update git submodules).

I added a step to the cloudbuild.yaml file to init and update submodules, but I get a Host key verification failed error. I did something like this,

cloudbuild.yaml

steps:
  - name: 'gcr.io/cloud-builders/git'
    args: ['submodule', 'update', '--init']
  ...

I can confirm that the default service account is being used for the trigger. And that service account does have permissions for the submodule's repo.

I would welcome any suggestions on how to debug this problem.

Cloning in general doesn't clone submodules (it requires an explicit "do clone the submodules" argument and presumably GCP never add that argument), so your approach is probably the right one. The "host key verification failed" indicates you're using ssh to do the cloning and that you need to teach ssh to accept, or maybe ignore, the host key for the submodules to clone. One way to accept it is to use `ssh-keyscan` first to generate the known_hosts file; one way to ignore it is to use the ssh options (from ssh config or via `-o`). — torek, Dec 22 '21 at 05:58
Making Git pass options *to* ssh requires using `git config` or similar so if you can configure ssh directly, or run ssh-keyscan early, that's probably better. Depending on how much you want (or don't-want) to trust the host you'll clone the submodules *from*, you could include the host key in the superproject repository, rather than running ssh-keyscan. — torek, Dec 22 '21 at 05:58
You may have a look at a [Stackoverflow case](https://stackoverflow.com/questions/59180893/google-cloud-build-cant-update-submodules) which is also having a similar issue. Let me know if that helps! — Mousumi Roy, Dec 22 '21 at 10:03

score 0 · Accepted Answer · answered Jan 31 '22 at 23:59

I solved this by rewriting the url of the submodule so that it uses HTTPS instead of SSH, based on the answer from this question. Thanks, Mousumi Roy.

Specifically, my .gitmodules file was unchanged

[submodule "src/my-repo"]
    path = src/my-repo
    url = ssh://source.developers.google.com:2022/p/my-project/r/my-repo

but the first step in my cloudbuild.yaml was changed to

steps:
  - name: 'gcr.io/cloud-builders/git'
    entrypoint: 'bash'
    args:
    - -c
    - |
      git config -f .gitmodules submodule.src/my-repo.url https://source.developers.google.com/p/my-project/r/my-repo
      git submodule update --init

Somewhat magically, the submodule could then be checked out. I think that it's the Cloud Build service account that's being used to authenticate against the repo by default, but I don't know how that authentication is being performed in the container when the build is running.

Google Cloud Build Trigger - Problem Fetching Submodules

1 Answers1