3

We have a Subversion setup that makes fairly extensive use of svn:externals (more than 5 external references in one project, and while most are in the same repository, one or two are in a different repository but still on the same server). Currently, the svn:externals properties are set up in a way where they have the full URL of the referenced project (i.e. "https://[server]:[port]/svn/Repository1/Projects/...)

Recently, I have set up an SVN mirror using svnsync which backs up our repositories to an external off-site facility. We need the ability to do read-only checkouts from those mirrored repositories at the remote site, which seems to work fine for the most part except when it tries to pull the externals, it still references our local on-site server.

Not sure if this is the best way to go about it, but I would like to set up some sort of post-commit hook at the remote location which would modify the properties of svn:externals and replace the hostname of our local server here with the hostname of the remote server there. This hook would run after each commit made by svnsync.

Ideally, we would like to avoid modifying our svn:externals properties at the main repository to not include the server name.

Has anyone ran into this issue before? What is the best way to go about it?

Ruslan
  • 2,691
  • 1
  • 19
  • 29

2 Answers2

4

You must understand that svn:externals are a property on a file, and modifying them requires a commit. If you have a mirror, and you modify the svn:external, you're creating a new remote version and breaking your mirror. The revisions will no longer line up.

Although it seemed like a neat idea at the time, svn:externals can be very bad in a project. Imagine a project like this:

http://vegibank.com/svn/trunk/project1

And I have on this directory an svn:external

$ svn pset svn:external "^/trunk/project2/foo foo" .

I do this because the foo directory is a shared set of files for multiple projects.

Now, I create a tag for project1:

$ svn cp http://vegibank.com/svn/trunk/project1 http://vegibank.com/svn/tags/1.2.3

Looks nice -- except the project1/foo directory isn't tagged. It's linked to the trunk of project2/foo.

My assumption with a tag is that the tag never changes, but this isn't true. Work is still being done on project2/foo on trunk, and that changes what my tag represents. If I have a bug in release 1.2.3, and I decide to checkout my tag to see what could be the problem, I'm not getting necessarily what I released in project1/foo -- I'm getting the latest from trunk.

A better way to handle this is to create a release repository, build the code that's common between various projects as some sort of pre-compiled artifact and have your project depend upon that version of that artifact. It ends up being no different than a C program dependent upon a particular version of libz.so or a Java project dependent upon version 1.6 of org.apache.commons.httpd.

This will eliminate the use of svn:externals and simplify your mirroring. You could mirror the release repository as well as the source repository.

If you insist upon using svn:external, don't use the full URL. Instead, use relative URLs.

For example, if you had setup your `svn:external above instead of this:

$ svn pset svn:external "^/trunk/project2/foo foo" .

to do this:

$ svn pset svn:external "../project2/foo foo" .   #property on ^/trunk/project1

Now, if I create a tag like this:

$ svn cp http://vegibank.com/svn/trunk http://vegibank.com/svn/tags/1.2.3

I'm tagging both project1 and project2. Now, my svn:external refers to http://vegibank.com/svn/tags/1.2.3/project2/foo.

What you need is a way to enforce svn:externals in this way, and you can use a pre-commit hook to reject any commit that has a svn:external set that references the trunk or branches directory without either specifying an actual revision.

David W.
  • 105,218
  • 39
  • 216
  • 337
  • Thank you for all the great advice! We actually don't use tags but we run into the exact same issue that you describe with branches. We have a slightly unorthodox branching structure, where we don't merge from trunk into the production branch during launches; instead, we "bless" the trunk branch by copying it as "production", and rename the old production branch to be a backup. There are many issues with this approach, but we are not going to change this process right now because we plan on migrating to git in the next 6-12 months and so doing all the work to make our SVN setup proper is – Ruslan Mar 14 '13 at 16:41
  • relatively unnecessary. We are forced to reconfigure all of the externals each time we do such a release, though (create corresponding branches in all of the dependencies as well and then reconfigure all of the externals across all projects to ensure that the externals are referencing the correct branches). – Ruslan Mar 14 '13 at 16:42
  • I think that relative paths might be exactly what I need, though. I will do more research on that topic and see if our current SVN version is recent enough to support it. Thanks again! – Ruslan Mar 14 '13 at 16:43
  • It may also be worth mentioning that we do not have global branches for all our projects as in your example; in other words, instead of having /svn/trunk/project1, we have /svn/project1/trunk. Hence the need to always reconfigure them. – Ruslan Mar 14 '13 at 16:48
  • 1
    If you're moving to Git, you might as well get rid of the `svn:externals` dependency. Git does not have a feature similar to `svn:externals`. To me, this is a _feature_ of Git because `svn:externals` is really just broken. The idea was good, but implementing it ended up being such a pain that you need to avoid it. – David W. Mar 14 '13 at 16:58
  • You can use `^/` at the beginning of `svn:externals` URLs which will allow projects that share the same server to have `svn:externals` as server independent. That way, when your mirror does its mirroring, the `svn:external` will point to the mirrored server. – David W. Mar 14 '13 at 17:00
  • I think git has something similar called "submodules". But I haven't done more research on that yet... Getting rid of externals would unfortunately require us to change our entire project structure, not something really feasible at the moment :( – Ruslan Mar 14 '13 at 17:01
  • @Ruslan You may not need to change your project structure. Just the way you assemble your code. What language do you program in? C? Java? PHP? Imagine compiling the dependencies into `*.dll`, `*.so`, or `*.jar` file and then including them from a release repo for your build. – David W. Mar 15 '13 at 17:49
  • We do .NET and have the project broken up into many separate class library modules which produce DLLs. Out of the 10 or so modules we have, 3 are actual project references because they almost always need to be present when working on the site. The rest are DLL references in a "Dependencies" folder. The two project references are also components, though, and thus the Site has externals references to them so that they are included with the checkout (otherwise project cannot be opened). The "Dependencies" folder is also an external... So its a bit sticky. Ideally it should all be one solution – Ruslan Mar 20 '13 at 18:11
  • with project references to all components, and that should be all under one folder and SVN project. But that is a pretty big change which is planned for later... – Ruslan Mar 20 '13 at 18:11
  • Back to my original question, though, I did manage to figure it out by modifying your example of using "^/" to just "/", which makes SVN pull from the server root. Should have been a common sense solution that I should have tried first, but it did not cross my mind for some reason. Thanks for all your help! – Ruslan Mar 20 '13 at 18:14
  • "My assumption with a tag is that the tag never changes, but this isn't true." [Use explicit revision numbers](http://tortoisesvn.net/docs/nightly/TortoiseSVN_en/tsvn-dug-externals.html). Problem solved. – cp.engr Oct 03 '15 at 21:19
1

What version of svn are you running? If you're current you can avoid putting the hostname in the svn:externals read the redbook specifically check out starting the external with /

you should strongly consider getting out of the svn:externals morass... especially if the links point to the same repo... you're branching and tagging are harder. I've seen a lot of svn implementations and svn:externals is usually a bad smell.

Also, as a general rule I think it's sub-optimal to have a post-commit hook that changes what the developer meant to do. Though in this case you'd likely be fine.

thekbb
  • 7,668
  • 1
  • 36
  • 61