Git: compile master, then merge into dev without recompiling everything

Question

I'm working on a large open source project with rapid development and very long compilation time. My question concerns how to minimise recompilation after getting the newest changes from master.

Say I have

  /--MyDev1
 /
A (=master)
 \
  \--MyDev2

MyDev1 and MyDev2 only touch a few files, so if I have a compiled version of A, MyDev1 or MyDev2, doing a checkout to any of the other and running make is pretty fast.

After a few weeks, master has moved forward a lot, upstream. Several hundred files have changed: I checkout master, pull the newest changes, run make, and grab a few dozen cups of coffee.

Afterwards, I want to do a merge that would result in the world looking something like

  /--MyDev1--------MyDev1'
 /                 /
A----....---------B
 \                 \
  \--MyDev2--------MyDev2'

However, If I simply do:

git checkout MyDev1 git merge master make

Then make will recompile everything, since all the files between A and B have been touched. And drinking that much coffee will make me sick.

What I want is to merge from B, and where git only touch the files changed between A and MyDev1 (and afterwards the same for MyDev2, possibly recompile everything between MyDev1 and MyDev2).

m8mble · Accepted Answer · 2015-09-26T16:54:32.497

I'm not aware of a solution using pure git. (Probably there is even none by design of git.)

But I've got another solution, I'm using frequently instead: I'm rsyncing the code somewhere else (lets call it build area) before each build. The trick is, I'm letting rsync determine which files to sync based on checksum rather than timestamps (flags: -rlpgoDv --checksum --ignore-times). Then I'm only compiling in the build area.

That doesn't speed up things in general, but at least for your

git checkout MyDev1 && git merge master make

example. In that case, rsync only touches your few MyDev1 files. Recompilation should be quick.

Things get slow, when MyDev2 hasn't been merged yet, while MyDev1 has. I'd suggest one build area per branch in this case (but this would essentially require a master compile per branch). Depending on your actual situation, at least one proposal should help a bit.

Update: I've been thinking: What actually changes your timestamps is the git checkout MyDev1, not the merge (assuming no conflicts). In case you can merge in the other direction, you could do (assuming you are on master and just had lots of coffee aka compiled):

git checkout master
git checkout -b tmp_merging_branch master
git merge MyDev1
git branch -d MyDev1
git branch -m MyDev1

This preserves timestamps, as long as the merge doesn't actually change the files. This is ok, as those files should be rebuilt anyway. The nice thing is, you could repeat that for MyDev2: The initial git checkout master then would preserve timestamps for files being equal in master and MyDev1. But it only works, if it is acceptable to produce something like

  / --- MyDev1 --- MyDev1' ------ MyDev1''
 /                   \               \
A --- .... ---------- B --- ... ----- C
 \                   /               /
  \ --- MyDev2 --- MyDev2' ------ MyDev2''

Thanks, that looks clever and useful. It adds an extra layer in the version controlling, though. In my use case, I would probably merge master into all my brances, so your solution should be as quick as possible, compile-wise. I'll wait a bit and see if someone has alternative, esp. pure-git solution, and if not, I'll accept your answer. — jsrn, Sep 24 '15 at 07:23
Thanks, I've accepted your answer which now has two suggestions. I like the latter the most. I have yet to try it out though. Could there be some meta-data in the branch that could get lost? — jsrn, Oct 04 '15 at 11:47

Git: compile master, then merge into dev without recompiling everything

1 Answers1