Migrating a Gerrit project into a different instance of Gerrit

Question

I have MainGerritServer which hosts many projects.
Also, I have RogueGerritServer which also hosts many projects.

I'm looking for a way to move a ProjectA from RogueGerritServer to MainGerritServer while preserving both Git commit history as well as the Gerrit review history.

For example, I want to import Android project's Gerrit history onto my own Gerrit server, so that when I work on a forked version of Android, I can look up the Gerrit history on my local server. What's the best way to do this?

If it were a simple Git installation, I would simply clone ProjectA into my computer then push to MainGerritServer. Does a Gerrit based project work the same way?

I'm concerned because Gerrit installation uses a database on the background, and I'm not sure whether I need to also migrate the information in the database. I saw many threads about taking a database dump and moving it to a brand new server. However, I'm trying to move just one project so taking a DB dump doesn't seem appropriate. The closest answer I saw was this which is still not exactly what I'm looking for.

I would appreciate any help, especially if you can show me whether I'm misunderstanding how to approach this.

Thank you

Answer 1

I've been moving a lot of git projects around lately and although I've mostly been moving from Gerrit (due to department fiat, Gerrit's a fine choice IMO) this should work the same and is relatively easy. Moving the Gerrit history, however will be more difficult.

Here's how to move a repo:

1. Clone the project from the current source.
2. Create the new gerrit project.

3. In your local clone of the project you want to move, add the new project location as a new remote:

   git remote add NewGerrit ssh://NewGerritServer:29418/NewProject

4. push to the new remote. To push the full history do:

   git push --all NewGerrit

You can also set it to automatically track the new repo by adding a -u flag:

git push --all -u NewGerrit

Migrating the Gerrit database stuff is a much bigger job and honestly I'd suggest you simply forgo it unless absolutely necessary. You can get an idea of what's involved by accessing the database directly through Gerrit's own (terrible) query language, GSQL. It's a lot like mysql except more temperamental and lacking a lot of features.

ssh -p 24981 HOSTNAME gerrit gsql

This will give you a gsql prompt. Try this:

gerrit> SHOW TABLES;

The caps, at least in the version of Gerrit I have access to, are important. (Mine is admittedly outdated.) This will show you around 29 tables and your project history is spread throughout them.

According to this posting from Shawn Pearce (Gerrit project lead) you would need to copy the database to a new db server then manually remove all other projects using SQL. You could then dump the remaining data and import it to the other Gerrit server. NOTE THE CAVEAT: there could be a collision of change_id values. He does suggest a work around for this however:

You could find out the max change_id of the incoming set, go manually bump the destination server's change_id_seq to reserve sufficient id space, then bump all of the old change_ids by some base value so they are in a unique space in the destination... and finally load the rows to the destination.

So it can be done. It is a pain in the hind quarters.

Answer 2

There is also now an importer plugin that can be used to migrate a git repository and all reviews from one server to another. It works with Gerrit version 2.11 and later.

More documentation:

• About importer plugin
• Announcement email

The importer plugin is downloadable from the (unofficial) CI server: master version, stable-2.12 version, stable-2.11 version.

Answer 3

The importer plugin introduced in the answer by David Pursehouse seems to be the way to go. However, as I struggled getting the binary (it doesn't seem to be downloadable, one have to build it yourself there are download links, but I wasn't aware of them at the time of writing this) as well as using it, thought I added my experiences on doing this.

1. Download, compile and install Buck. Buck is a build system created by facebook. It needs linux/unix system, won't work on Windows.

Steps:

$ sudo apt-get ant -y
$ git clone https://github.com/facebook/buck.git
$ cd buck
$ ant

(ref. installation instructions)

2. Download and compile importer

Steps:

$ git clone https://gerrit.googlesource.com/plugins/importer
$ git clone https://gerrit.googlesource.com/bucklets
$ cd importer
$ ln -s ../bucklets . && ln -s bucklets/buckversion .buckversion && ln -s bucklets/watchmanconfig .watchmanconfig

# check latest stable version
$ git branch -a
* master
  remotes/origin/HEAD -> origin/master
  remotes/origin/master
  remotes/origin/stable-2.11

# and use that
# why? well, snapshot version depends on snapshot apis, and you get into dependency hell with those...
$ git checkout stable-2.11

$ ../buck/bin/buck build plugin

Binary file will be buck-out/gen/importer.jar.

(ref. build instructions)

3. Copy the generated .jar to your $gerrit_site/plugins (create the directory if it does not exist) or, if you need to do it remotely, do ssh -i /path/to/your/ssh/key -p 29418 youraccont@localhost gerrit plugin install -n importer.jar /path/to/importer.jar. The latter will need plugins.allowRemoteAdmin enabled.

4. Restart your gerrit instance with $gerrit_site/bin/gerrit.sh restart

5. Plugin should be visible in the web ui

After that, the process is as follows:

1. do the initial import of the project
2. test on the target Gerrit that everything is okay
3. inform the project team about the project move and disallow further modifications of the project in source Gerrit server (e.g. by permissions or by setting the project state to read-only)
4. resume the project import to get all modifications which have been done after the initial import
5. complete the import and if needed make project in the target Gerrit server writable
6. inform the project team that they can now work on the project in the target Gerrit server
7. reconfigure any third-party tools (such as Jenkins) to work against the project in the target Gerrit server
8. [optionally] delete the project in the source Gerrit server using the delete-project plugin

Doing an initial import first and resuming the import later has the advantage that the downtime for the project team can be kept minimal. The initial project import may take some time, but the resume should be fast since it only needs to transfer the delta since the initial (last) import.

(ref: About.md)

Personally, I haven't been able to get the import process to work. It seams that during the import phase, after hours of work-in-progress, at least on our network we get greeted with java.io.IOException: Unexpected response code for GET on /config/server/version : 504 meaning network timeout before this ever gets to some state. This happens even with small repos.

Answer 4

I was able to get the importer plugin to work, but also got 504 errors continuously. The plugin asks for 500 changes at a time (read from GlobalCapability.DEFAULT_MAX_QUERY_LIMIT, which is hard-coded to 500). I couldn't successfully build the stable-2.13 branch of the plugin, so I just retried and re-tried the import until it finally succeed.