29

I have a large Subversion repository with nearly 15 GB of data spread across ~500,000 files. Now I need to check out this repository to a remote host which would take days to complete.

The host I'm checking out to already has a complete copy of the data in the repository. But seeing as the files weren't checked out directly from the repository, they do not constitute a working copy (no ".svn" folders).

I'd like to avoid copying all this data across the network, especially when it already exists on the target host. Is there a trick I can use that will turn a preexisting directory into a working copy without replacing the local files with identical copies from the repository?

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
weston
  • 2,878
  • 3
  • 24
  • 23
  • Just out of curiosity: What are you keeping in that repository? I've never heard of a repo this large. – sleske May 14 '09 at 09:41
  • 2
    Raw audio, mostly spoken word. – weston May 14 '09 at 17:09
  • What this sounds like is "how to get the .svn directories back after cleaning them out" or "I did an export that took days. But I wanted the .svn files, too..." – lilbyrdie Jun 22 '11 at 14:23
  • couldn't you use `rsync` with `--ignore-existing`, and duplicate the repository (ie `.svn` dirs)? (not sure if this world work, just thinking..) – Anil Jan 27 '14 at 22:15

10 Answers10

27

As of SVN 1.7 (but not before) you can do this easily with:

svn co --force http://path/to/repo

This will respect the local copy as existing, and you'll see the "E" for existing before each file name in the output: E some/existing/file

If the file is different (new or modified) from the repository it will handle that gracefully too according to the book:

Prior to version 1.7, Subversion would complain by default if you try to check out a directory atop an existing directory which contains files or subdirectories that the checkout itself would have created. Subversion 1.7 handles this situation differently, allowing the checkout to proceed but marking any obstructing objects as tree conflicts. Use the --force option to override this safeguard. When you check out with the --force option, any unversioned file in the checkout target tree which ordinarily would obstruct the checkout will still become versioned, but Subversion will preserve its contents as-is. If those contents differ from the repository file at that path (which was downloaded as part of the checkout), the file will appear to have local modifications—the changes required to transform the versioned file you checked out into the unversioned file you had before checking out—when the checkout completes.

Also note that SVN 1.7 can lead to situations where this a more common problem (perhaps motivating the solution). I hit this problem when moving a sub directory to a new location on disk. In pre-1.7 that would have moved the .svn directory with it and it would have stood alone just fine. In 1.7 the directory became effectively unversioned. But svn co --force saved the day.

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
Rhubarb
  • 34,705
  • 2
  • 49
  • 38
  • However, this add, to the local directory, files which were in the repository (if they are not found locally). How could I avoid this ? and tell svn : this is the new working copy, deal with it ! – yota Mar 08 '16 at 13:53
3

There is the relocate command: http://svnbook.red-bean.com/en/1.1/re27.html

EDIT: If the local files aren't linked to a repository then you could create a local repository, import the files into it and then use the relocate command.

Alternatively if you have physical access to both machines you can you check out the repository locally and then copy the files to the remote machine via a external HD.

Jonathan Parker
  • 6,705
  • 3
  • 43
  • 54
  • 1
    I don't see how you could use relocate here. relocate only updates metada in the .svn folders. There are no .svn folders according to OP. – Wim Coenen May 14 '09 at 00:34
  • 2
    I doubt this will work, because you can only relocate to a repository with same repository UID – Peter Parker May 14 '09 at 12:41
  • 2
    The documentation I linked to above says: If the working copy still reflects the same repository directory, but the location of the repository itself has changed, then use svn switch --relocate. – Jonathan Parker May 17 '09 at 23:25
1

The following command is to delete all .svn directories.

chmod -R 0755 project_dir
find /project_dir -type d -name .svn -exec rm -rf '{}' +

If you already have a checkout version, you can try to copy those .svn folders by replacing rm with cp. I didn't try though.

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
Wen
  • 19
  • 1
1

svn co --force https://PATH/TO/REPO/ .

Where the . at the end assumes you are already inside the directory which you want to turn into a working SVN copy.

For instances, if you wanted to make your public_html directory a working svn copy of a repository:

cd /home/username/public_html; svn co --force https://PATH/TO/REPO/ .

udondan
  • 57,263
  • 20
  • 190
  • 175
BornSupercharged
  • 111
  • 6
1

You could try using rsync if you already have a working copy under svn control somewhere else on the network.

  • This isn't such a bad idea but, as mentioned in another comment, the .svn/text-base folder contains pristine copies of all your repo files, so rsync would end up copying all the data over anyway. You could possibly tell rsync to ignore replicating "text-base" and then script something to populate the "text-base" once the .svn folders are synced. – weston May 14 '09 at 18:41
0

Do a checkout on the server, creating a working copy in local (local to the server), then rsync that working copy to the remote system over the existing directory structure.

Use Subversion 1.7, that way there's no .svn with pristine copies of the files.

Diego Fernández Durán
  • 2,502
  • 23
  • 32
0

I had a working repository on my local computer that got all of its .svn folders deleted when Eclipse crashed.

The only way I was able to connect it to the remote SVN repository was to follow these steps from a blog I found (Recovering a broken Subversion working copy):

# Backup your project in case you run into trouble
cp -Rp /path/to/project /temporary/location

# Strip out the old .svn folders (if any)
find /path/to/project -name .svn -print0 | xargs -0 rm -rf

# Check out a clean copy
svn co http://repo/location /temporary/location2

# Move the .svn folders from the clean copy into the correct relative
# place in the broken copy
cd /temporary/location2
find . -name .svn -print0 | xargs -0 -I {} mv '{}' '/path/to/project/{}'

# Remove the clean copy
rm -rf /temporary/location2
Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
Lucky Dog
  • 51
  • 6
0

None of the native svn commands will checksum existing files for matches and not download them.

What protocol are you using to access the repository? If it is https, that may be your problem. Try the native svn protocol (using svnserve), or svn+ssh. Or perhaps even do a checkout via a file:// URL on the server hosting the svn repo, andthen use rsync to transfer across the network.

So long as you are not paying for the bandwidth per-byte, it might just make sense to let "svn co --force" run under nice or (START /LOW on Windows) and not waste your own time. It will not make anything on the local filesystem unavailable during the checkout process.

Finally, I can't figure out why your checkout is so slow... we have 500K file repositories that checkout in ~6 minutes via https on a gigabit LAN. Granted all the files are much smaller (1 GB total). How far away from the server are you in terms of latency?

rmalayter
  • 519
  • 6
  • 12
0

You could just checkout the repository locally, then transfer only the .svn directories (careful, they contain copies of the workspace files, obviously you don't want to copy those). That should work, as you would have the exact files of a correct working copy.

Of course you'd have to write some kind of script to transfer the .svn files. On a Unix system you could do it with find and friends.

sleske
  • 81,358
  • 34
  • 189
  • 227
  • -1 If he has around 500,000 files imagine to move all the .svn folders 'carefully'? – victor hugo May 14 '09 at 00:56
  • 1
    Of course you would use some script. The "careful" was to warn of a potential problem when writing such a script. – sleske May 14 '09 at 09:39
  • I considered this, but as Victor pointed out this could be pretty tedious with such a large repo. Also, the ".svn/text-base" folders of the working copy contain pristine copies of all files in the repo, so even copying just the .svn folders will still require a complete copy of the data. – weston May 14 '09 at 18:27
  • Yes, of course, that's why I mentioned a script. I'm not expecting anyone to copy it all by hand :-/. And of course you wouldn't copy the stuff under text-base, you'd only create the emtpy directories '.svn/text-base', then copy the existing files from the destination into text-base. Yes, that might require a bit of programming, but it still beats recopying 15 GiB. – sleske May 14 '09 at 23:17
-1

I don't believe there's a solution without transferring those 15 GBs to the destination. It would probably be faster and easier to copy the repository there and do a local checkout.

Gleb
  • 2,404
  • 1
  • 13
  • 7
  • Well, funny you suggest this because I tried checking out just a local working copy on the server itself. Even a local checkout takes well over a day to complete and this wouldn't include the time it would take to dump/copy/load into a new repository on the remote host. – weston May 14 '09 at 18:31