4

With git config --global url.<base>.insteadOf <url>, one can switch, in particular, the protocol used to clone submodules of a given git module.

I would like to switch from https to ssh credentials. My problem is that - by the current git settings, the clone is actually performed as https://username:password@example.com/repo.git, and I do not know the user/password in advance (but I do know the domain/repo url itself).

From what I have tried, wildcards as

git config --global url.'https://*:*@' 'ssh://@' do not work, and just putting url.'https://' 'ssh://' does not work, since that leaves me with git@username:password@example.com, which is - of course - not what I want.

Unfortunately, I also do not know the current state git is configured as, and I cannot control the current git installation, so I would need a fix that works regardless of what username/password git is currently configured to use when cloning over https.


A bit of context where this bizarre situation actually occurs:

I have a private repository hosted on gitlab.com that uses a private submodule where I would like to setup a GitLab runner. By default, this of course does not have access to the private submodule, so I followed https://docs.gitlab.com/ee/ci/ssh_keys/ to set up ssh keys within the build environment of the runner. I do not clone recursively, but in my ci build script, I first add the ssh key and then update the submodules manually via git submodule update --init --rebase.

So far, this has worked well, but since my last commit - for some reason I do not know - the runner switched to trying to clone the submodule via https, which fails of course. By experimenting around, I figured out that

  • The cloning method chosen by the GitLab runner depends on the commit of the submodule that is defined in the main repository
  • All commits that existed so far in the submodule can be specified in the main repository and will be cloned correctly.
  • The other files in the main repository apparently do not affect the cloning method.
  • Any new commit that I make in the submodule and check out in the main repository results in https being used to clone the submodule, which fails.
  • This applies even to new, empty commits

To emphasize the last property, I have a situation where the runner works fine (cloning over ssh), and after doing

  • adding empty commit to submodule, pushing to origin
  • update submodule in main repository to use that commit
  • push main repository to origin

the runner changes to using https again, which fails.

This problem seems to be similar to Gitlab-runner fails to init a submodule using ssh-agent or Docker GitLab Runner cant load submodules, but since these did not have too much feedback, I decided to risk the x y problem and ask for a workaround directly, since I have not much hope that the gitlab side of this can be fixed, and I should just force git to use ssh directly within my script.

Apparently, by default the runner uses the https protocol with a runner-specific token issued by gitlab, so that it can clone the (main) repository in the beginning, and in the cases my pipeline fails, this token is also tried when cloning the submodules, and currently, I do not know how to configure git to force it to use ssh for cloning the submodules, instead of this gitlab-issuse https credentials.


A bit of context

  • 1
    Wildcards (or regex or anything like that) are not allowed here. Probably there should be some way to do this, but there isn't now. Consider making a contribution to the Git project. – torek Mar 28 '22 at 21:31
  • Trying to do something similar and have yet to find a solution. My remote has more complex SSH and HTTPS paths so there is no way to simply replace https:// with something like git@something. – Crustyeyelids May 03 '22 at 16:59
  • Are you sure you do not know the user and passwort in advanced? there are many predefined variables like `GITLAB_USER_LOGIN` and `CI_JOB_TOKEN`. I do it the other way around in my pipeline and use `git config --global url."https://gitlab-ci-token:${CI_JOB_TOKEN}@git.".insteadOf ssh://git@git.` - maybe yours should work the other way around!? For Reference: https://docs.gitlab.com/ee/ci/variables/predefined_variables.html – lexXxel May 30 '23 at 12:38

0 Answers0