GitLab pull submodules inside CI

Question

I have a GitLab project that utilises GitLab CI. The project also uses submodules, both the project and it's submodules are under the same GitLab account.

Here is my .gitmodules file

[submodule "proto_contracts"]
    path = proto_contracts
    url = https://gitlab.com/areller/proto_contracts.git

I also have this piece in the .gitlab-ci.yml file

variables:
  GIT_SUBMODULE_STRATEGY: recursive

However, when i run the CI I get this error

fatal: could not read Username for 'https://gitlab.com': No such device or address

Both the project and the submodules are in a private repository so you would expect to be prompted for authentication, but as I've mentioned, the project and the submodule are under the same account and one of the runner's jobs is to clone the original repository

So it's odd that it's unable to reach the submodule Is there a way around it?

@StefanvanGastel The only relevant part is the part where I set the `GIT_SUBMODULE_STRATEGY` variable, everything else is just building the code and running. I know for a fact that it fails at the cloning part (where it's supposed to recursively clone the submodules) — areller, Feb 17 '18 at 20:49
Do you have git clone ... in .gitlab-ci.yml? If yes, you should not — ilia, Feb 18 '18 at 08:11
@ilia No, the gitlab runner is responsible for cloning, all I do is set the `GIT_SUBMODULE_STRATEGY` variable to `recursive` to make the cloning pull the submodules as well. — areller, Feb 18 '18 at 08:13
This answer might be useful for you: https://stackoverflow.com/questions/58019082/how-do-i-pass-credentials-to-pull-a-submodule-in-a-gitlab-ci-script/67843350#67843350 — rubicks, Jun 07 '21 at 16:43

score 28 · Accepted Answer · edited Jul 08 '19 at 07:33

28

You must use relative URLs for submodules. Update your .gitmodules as follow:

    [submodule "proto_contracts"]
        path = proto_contracts
        url = ../../areller/proto_contracts.git

Further reading: Using Git submodules with GitLab CI | GitLab Docs

edited Jul 08 '19 at 07:33

MrRolling

2,145
1
24
33

answered Feb 18 '18 at 15:54

ilia

1,082
11
19

Wow, I've somehow missed it while reading the documentation, thank you! – areller Feb 18 '18 at 16:58
1

How do you avoid this interfering with development and deployment outside of gitlab? – naught101 Jan 16 '19 at 01:38
1

Oh, never mind. That's something git understands. Awesome :) – naught101 Jan 16 '19 at 01:41
1

I tried this and for me it did not make a difference. – mirk Jul 31 '19 at 16:19
3

Okay! But, if a have relative git path, then how could I work with submodules locally? – Konstantin Chuykov Jan 12 '22 at 13:15
You can use relative paths in your `.gitmodules` file and it also will work locally. It's outlined in GitLab docs: When your submodule is on the same GitLab server, you should use relative URLs in your .gitmodules file. Then you can clone with HTTPS in all your CI/CD jobs. You can also use SSH for all your local checkouts (https://docs.gitlab.com/ee/ci/git_submodules.html). – Psylone Mar 20 '23 at 09:27
This does not take into consideration the possibility of using submodules outside of local instance and also repositories that require credentials. – rbaleksandar May 07 '23 at 13:53

GitLab pull submodules inside CI

1 Answers1

Linked