Git repository on S3 (as "origin", not as backup)

Question

I recently tried the S3 remote provider of JGit, which works like a charm.

EDIT: Statement regarding transmission of whole repos removed, see comments. This does not change the main question I have here, however.

So now I'm thinking about using this productively, which leads me to the following questions:

Do you use a Git repository hosted on S3 (not just for backup, but for collaboration)?
Is it possible to encrypt the repository? I know about the default DES encryption of JGit, but I'd rather not rely on ancient encryption algorithms.
What are the pros and cons of such a solution?

Thanks for your opinions!

Git always transfers whole repos, or at least branches with full history. That's a design feature. — Fred Foo, Jan 05 '12 at 15:05
git always compress objects, thus "big repository" is quite relative: If there are many commits, but they contain always few changes, the repository doesn't growth as much as you think. — KingCrunch, Jan 05 '12 at 15:08
Ok, I didn't know. I edited the question accordingly. My main question remains. — theDmi, Jan 07 '12 at 15:54
Really? Nobody is using S3 as Git repo? It sounded like a good idea :-) — theDmi, Jan 23 '12 at 21:16
Also let everyone know your OS, it could be more specific in this situation. — Eugene, Jan 31 '12 at 05:54
@Janaka: I didn't take this approach in the end. We're using a Git hosting provider now. — theDmi, Nov 27 '12 at 07:53

score 3 · Answer 1 · answered Jan 31 '12 at 05:53

So the idea of this, is mount your S3 bucket as drive on your local system.

If you prefer UI way, I would recommend Transmit application by Panic INC. http://panic.com/transmit/ ($34 at the moment of January 2012)

You can read more about it here: http://www.40tech.com/2011/05/02/how-to-mount-inexpensive-amazon-s3-storage-as-a-local-drive-using-transmit-mac/

As for console way (compatible with both mac and linux) take a look into:

http://code.google.com/p/macfuse/ (Snow Leopard)
http://osxfuse.github.com/ (Lion)
http://code.google.com/p/s3fs/ (*nix)

Next step: Go into your mounted s3 bucket:

cd /mnt/yourS3bucket

and init repository:

mkdir myRepository && cd myRepository
git init

Below are answer to your questions:

Do you use a Git repository hosted on S3 (not just for backup, but
for collaboration)?

1) First time we used it like that, but later we moved it into EBS block inside of running EC2, with automatic backups to S3 and automatic restores (if EC2 or EBS crash or both). Its just much faster.

Is it possible to encrypt the repository? I know about the default DES encryption of JGit, but I'd rather not rely on ancient encryption algorithms.

2) If you want just encrypt whole repository, think about encrypting of file system containers. And here you're free to choose, based on OS you're using. Again, you'will be dealing with productivity of such way: http://www.askdavetaylor.com/password_protect_encrypt_mac_os_x_folder.html Anyway for this way it will be only single encryption so, all your users will need to know it. Another solution, is SSH, based access to that EC2 instance, with separated user privileges. Also do not forget about separate IAM account for your S3 bucket for different situations. So you can always log back and detect changes by your users.

What are the pros and cons of such a solution?

3) So, in a short: it was good idea, but I'd rather advice build it over SSH and use S3 as single-file-backup solution, not multiple-files-repository-backup way.

Best, Eugene

Git repository on S3 (as "origin", not as backup)

1 Answers1