git filter-branch: moving a folder with --index-filter does not work

Question

Let's say I want to rewrite Git history, moving all the contents of /foo/bar to /baz/quux for all the past commits; running on Windows.

I made a sample repo with two commits in /foo/bar and nothing more:

git clone https://github.com/jakub-g/filter-branch-test

Let's put the script like below one folder up from my Git repo and run it (similar things are mentioned in many places on SO, including https://stackoverflow.com/a/13590229).

#!/bin/bash
PATH_TO_GIT_REPO='./filter-branch-test'
REWRITE_FROM='foo/bar/'
REWRITE_TO='baz/quux/'

cd ${PATH_TO_GIT_REPO} &&
git filter-branch -f --index-filter \
 'git ls-files -s | sed "s-\t${REWRITE_FROM}-\t${REWRITE_TO}-" \
  | GIT_INDEX_FILE=$GIT_INDEX_FILE.new git update-index --index-info \
  && mv $GIT_INDEX_FILE.new $GIT_INDEX_FILE' &&
cd -

When I run it:

bash ./runFilterBranch.sh

The output is:

...
WARNING: Ref 'refs/heads/master' is unchanged

and nothing happened, I still have old paths.

When I run git ls-files -s | sed "s-\t${REWRITE_FROM}-\t${REWRITE_TO}-" in master, I see that sed does the replacements correctly.

What am I missing here?

The obvious problem is that `${REWRITE_FROM}` and `${REWRITE_TO}` won't be expanded in place here as the entire index filter is enclosed in single quotes. Another potential issue is whether `\t` works here (I don't know if it does in the `sed` you are using). — torek, Feb 20 '17 at 23:03
Yep indeed the problem was due to the single quotes preventing the expansion. Looks like it's time to get some sleep! — jakub.g, Feb 20 '17 at 23:04

score 5 · Answer 1 · answered Feb 20 '17 at 23:33

5

By the way, there's an easier way to do this:

git filter-branch --index-filter "
        git read-tree --prefix='$REWRITE_TO'/ \$GIT_COMMIT:'$REWRITE_FROM'
        git rm -r --cached '$REWRITE_FROM'
"

answered Feb 20 '17 at 23:33

jthill

55,082
5
77
137

seems to work as well, though I had to remove the trailing slash in `'$REWRITE_TO'/` (i already have a slash in my variable and here the slash would have been passed after the closing apostrophe), thanks – jakub.g Feb 21 '17 at 20:48
2

An explanation would be nice. – miyalys Nov 04 '19 at 19:18

jakub.g · Accepted Answer · 2017-02-21T21:45:04.987

As @torek mentioned, the problem was a basic issue with variables expansions not taking place inside single quotes.

Just for the record, here's how I updated my script to be reusable and to be able to pass variables from the outside (the variable expansions and escapes are tricky)

In fact I created two bash functions for 2 different use cases, the second one is way faster for certain scenarios.

PATH_TO_GIT_REPO='test-folder'
REWRITE_FROM='foo/bar/'
REWRITE_TO='baz/quux/'

# Rewrite just one folder and keep all the other folder intact.
# Note this will rewrite ALL the commits in the git repo's history
# so it will be slow for big repos.
rewriteFolder(){
    SED_COMMAND='s-\t\"*'${REWRITE_FROM}'-\t'${REWRITE_TO}'-'

    cd ${PATH_TO_GIT_REPO} &&
    git branch -f FILTER_BRANCH_BACKUP &&
    git filter-branch -f --index-filter \
     "git ls-files -s | sed \"$SED_COMMAND\" |
      GIT_INDEX_FILE=\$GIT_INDEX_FILE.new git update-index --index-info &&
      mv \$GIT_INDEX_FILE.new \$GIT_INDEX_FILE" HEAD
}

# Rewrite just one folder and isolate it (remove all the other folders).
# If you have a big number of commits, and only some of them touch ${REWRITE_FROM},
# this will rewrite only this small subset of commits, and discard all the other commits,
# hence it will be much faster than `rewriteFolder`.
isolateAndRewriteFolder(){
    SED_COMMAND='s-\t\"*-\t'${REWRITE_TO}'-'

    cd ${PATH_TO_GIT_REPO} &&
    git branch -f FILTER_BRANCH_BACKUP &&
    echo 'Step 1/2...' &&
    git filter-branch -f --prune-empty --subdirectory-filter ${REWRITE_FROM} &&
    echo 'Step 2/2...' &&
    git filter-branch -f --index-filter \
     "git ls-files -s | sed \"$SED_COMMAND\" |
      GIT_INDEX_FILE=\$GIT_INDEX_FILE.new git update-index --index-info &&
      mv \$GIT_INDEX_FILE.new \$GIT_INDEX_FILE" HEAD    
}

git filter-branch: moving a folder with --index-filter does not work

2 Answers2