Batch import repositories to gitlab (self-hosted)

Question

I have about 50 git repositories that I want to import into a GitLab instance. They are not in GitHub/GitLab/whatever, but just sitting on my hard disk as a result of a SVN git conversion.

What would be a good way to import them all?

I am thinking about writing a command line script or small program in Java/Python/... but I am not sure whether I should use the Gitlab API directly, use glab or if there is already something build in that makes this task easier.

Note that this is a bit harder in GitHub, but if you need a GitHub way to do it, see the command line `gh` command ([tag:github-cli]). There is no guaranteed generic-Git answer since automating repository creation is hosting site dependent. — torek, Mar 12 '22 at 06:47

sytech · Accepted Answer · 2022-03-12T03:48:20.133

Your best bet is to just use a simple script for this. The good news is that you can create projects simply by pushing them to a namespace to which you have appropriate permissions to create repositories.

For example, you can create a new project like this:

git remote add origin ssh://git@gitlab.example.com/mynamespace/my-newproject.git
git push -u origin --all

You'll see your GitLab instance answer back with the following message indicating the project has been created:

remote:
remote: The private project mynamespace/my-newprojct was successfully created.
remote:
remote: To configure the remote, run:
remote:   git remote add origin git@gitlab.example.com:mynamespace/my-newprojct.git
remote:
remote: To view the project, visit:
remote:   https://gitlab.example.com/mynamespace/my-newproject
remote:

Assuming you have a valid (and ideally passwordless) SSH configuration, the repository names are safe to use for a project slug (no spaces, special characters, etc) and a flat structure containing your converted repositories like:

/path/to/converted/repos
├── my-great-repo1
├── my-great-repo2

In Python, something like this should work...

import os
import subprocess
from typing import Generator, Tuple

# change these values
GITLAB_NAMESPACE = "gitlab.example.com/mynamespace"
CONVERTED_REPOS_ROOT = "/path/to/converted/repos"


def iter_git_repos(root) -> Generator[Tuple[str, str], None, None]:
    for name in os.listdir(root):
        path = os.path.join(root, name)
        if not os.path.isdir(path):
            continue  # skip files
        git_directory = os.path.join(path, ".git")
        if not os.path.exists(git_directory):
            continue  # skip directories that are not git repos
        yield name, path


for name, path in iter_git_repos(CONVERTED_REPOS_ROOT):
    os.chdir(path)
    origin_url = f"ssh://git@{GITLAB_NAMESPACE}/{name}.git"

    # set the remote (use `set-url` instead of `add` if an origin already exists)
    subprocess.run(["git", "remote", "add", "origin", origin_url], check=True)
    # push all branches
    subprocess.run(["git", "push", "-u", "origin", "--all"], check=True)
    # push all tags
    subprocess.run(["git", "push", "-u", "origin", "--tags"], check=True)

score 1 · Answer 2 · answered Mar 22 '22 at 13:03

In the end, I used gitlab-rake to import the repositories:

https://docs.gitlab.com/ee/raketasks/import.html

I copied the repositories on the server into a dedicated directory, changed the owner to the git user and group, and ran:

gitlab-rake gitlab:import:repos["/var/opt/gitlab/git-data/repository-import-$(date "+%Y-%m-%d")"]

Batch import repositories to gitlab (self-hosted)

2 Answers2