How do you manage software's local forks?

Question

As a system administrator, I often have to modify programs for my company.

Example:

We're using an web interface to manage our local DNS.

After downloading and extracting the tar.gz file from the internet, I have to do some changes: I added a LDAP connector, changed some template files, adding some custom footer/header, added an additional table in the database, etc.

So many modification become hard to track.

• Is there a way to handle software updates easily with so many modifications?
• How to keep my modifications upon next software release?

Some more informations:

• source of softwares are multiples (github, sourceforge, etc)
• We are more than one sysadmin
• version control systems are welcome

Answer 1

Use version control.

If there is a current version control system in place for development use that.

If there isn't one already in place, think about using git.

Why git? There's as much or as little administrative overhead as you need. You can run a central server, or not. It's up to you. SVN is a good alternative (and widely deployed) but it can be more trouble to maintain.

Regardless of your VCS, you should build a basic directory structure like this:

OPS
|
+ -- Systems
|    |
|    + -- Server2
|         |
|         + -- etc
|              |
|              + -- httpd
|                   |
|                   + httpd.conf
|
+ -- Services
     |
     + -- httpd
          |
          + -- mods
          |
          + -- patches

I built something like this for tracking all of my nagios configurations. It worked perfectly.

Here's a simple example in git, let's say you just installed a web server and want to backup the config.

cd /etc/httpd/
git init
git add .
git commit -a -m "Initial baseline"

Now each time you've changed the file (EVERYTIME you change the file, no matter how trivial). Run this command:

git commit -a -m "Enabled mod_cgi and debugging for ticket 17789"

You are now keeping multiple versions of the /etc/httpd files in the /etc/httpd directory (specifically in the /etc/http/.git directory). This can get out of hand with > 10 git repos, so I'd look into starting a git server.

As an added benefit (of setting up a server), your configuration changes can now be pushed and pulled from any location. You no would no longer have to ssh into a machine to hand edit configuration files, and you can easily run diffs from anywhere. If you work in a large team, you can also track and merge changes.

Here's a few articles to get you started with git:

Git is simpler than you think.

Gitting into git

git main site

Answer 2

The only thing you need is a repeatable build process that takes the source from upstream and applies your patches, in order, until you end up with the result that you want. (This implies that you need to track your own changes in the form of patches, of course.)

This can be something as simple as a shell script that runs the proper commands, or you can go more involved with it and build a native .rpm/.deb/whatever package that handles these steps for you. I generally take the second route, since there's a lot of other benefits to packaging that I like to leverage.

I recommend creating your .patch files incrementally, and keeping them small and related to specific bits of functionality, to reduce the impact of merge conflicts when upstream releases new versions. If you try to do this with one giant godfather-patch that contains every change you've made to the software, it's never going to apply cleanly.

Beyond that, it's just diligence and paying attention to merges that don't work. It wouldn't hurt to write your own regression tests for anything you touch, but that's at your discretion.

Answer 3

Is there a way to handle software updates easily with so many modifications?

Any VCS with good branch-merge facilities. Which to select - depends from a lot of factors and habits

How to keep my modifications upon next software release?

In SVN world it named as "Vendor Branches" - your changes in one branch, upstream in another, you update vendor branch and merge into your (read SVN-book or Google about details)

In DVCS-world you can use different techniques

• branches, as in SVN
• patch&patch-management

Easiest way (for me, after some testing) is mercurial + MQ:

• my changes are set of MQ-patches in MQ-queue, repo is pure clone of upstream
• for every sync with upstream I remove applied patches from repo, pull sources, re-apply patches (with merge-conflict resolving, if needed, and saving edited patches), export tip to unversioned set and copy to final destination