0

Assume an SVN setup that consists of a single repo with nested project folders, something like this:

\
 - trunk
 - - projectA
 - - - trunk
 - - - - ...
 - - - branches
 - - - - ...
 - - projectB
 - - - trunk
 - - - - ...
 - - - branches
 - - - - ...
 - ...

The whole repo is about 40GB on disk, with 17605 commits at last count. I now need to extract a single project from the repo to setup on a separate SVN server, which I understand is possible only by using svnadmin dump on the entire repository and then using svndumpfilter to isolate the project that I need. I fully expect that this will take a really long time to complete the initial dump. Is there a good formula for calculating exactly how long it will take and how much disk space will be required for the final dump file? Also, I've heard that the dump operation uses 100% CPU while it runs. Is that true?

Alternately, is there a better way to go about this given the size of the repo? (Other than just doing an export and losing the revision history.)

Chris Bloom
  • 124
  • 1
  • 7
  • 1
    Why wouldn't it use 100% cpu when it is running. If the processor isn't doing anything else, and you are running a program, then you want to use all your resources to make it complete faster. You could set the priority of your dump if other important services are running on the system. On some *nix there is a command `nice` to start a process with a lower priority. – Zoredache Sep 08 '11 at 17:54
  • I know this is an old post but I'm hoping you get notified. I attempted your solution, Shane and it worked well until the rinse and repeat part. I created the second dump file but when I loaded that it failed. SVN didn't like that I was trying to load dmp.2 into a repo that wasn't empty. Is there a switch I need to force it? – Devilsbane Feb 18 '15 at 15:20

2 Answers2

1

Is there a good formula for calculating exactly how long it will take and how much disk space will be required for the final dump file?

To avoid the disk I/O, you can pipe the svndumpfilter result to netcat.

On the old SVN server:

svnadmin dump /path/to/your/repo/ | svndumpfilter --drop-empty-revs --renumber-revs include single_project | nc -l 2302

and on the new SVN server:

svnadmin create single_project
nc IP_address 2302 | svnadmin load single_project

I've tested with my repo (4GB, ~12000 revisions), it takes ~12 minutes to complete.

PS: You can also use gzip to compress the data and ionice to run this with low priority.

quanta
  • 51,413
  • 19
  • 159
  • 217
  • I should have mentioned that the source machine is Windows. I don't think I will have command line access to the new machine as it is a hosted SaaS repo. – Chris Bloom Dec 08 '11 at 18:05
  • Thanks for providing this answer. I'm marking this as the answer because theoretically I think it would have done exactly what I needed even though the project scope changed and I didn't need to do this after all. – Chris Bloom Jan 10 '12 at 15:24
0

That'll be pretty massive without the --deltas option, which you can't use when you're going to filter it. The time and space needed will, of course, depend on your server's performance and the contents of the repo.

If you need to do a dry run to see the space usage, just send it to /dev/null - and as @Zoredache mentioned, set the process's niceness as needed.

A couple recommendations:

  • Send it straight to a compression engine from stdout; they compress well.
  • If the size usage is too much to handle (which I suspect it might be), do it in parts.
    • dump the first thousand revisions (svnadmin dump -r 1:1000),
    • filter them,
    • load into the new repo,
    • delete the first dump, make a new one, rinse and repeat.
Shane Madden
  • 114,520
  • 13
  • 181
  • 251
  • Thanks for this comment. It sounds logical, but the client has since asked for a simple export of trunk and they will be starting a new Git repo with it instead, so I won't be able to test your suggestion. – Chris Bloom Dec 08 '11 at 18:04