1

I want to use filter-branch to set all files to non-executable because most users use Windows. I ran the following command in Git Bash:

git filter-branch --index-filter 'for file in $(git ls-files)
do
  git update-index --chmod=-x -- $file
done' -- HEAD~1..HEAD

However, I got an error message:

Rewrite bc4368aec16cce1c1faa7363dde9ac74ac28da6a (1/1)
error: .gitignore: does not exist and --remove not passed
fatal: Unable to process path .gitignore
error: LICENSE.md: does not exist and --remove not passed
fatal: Unable to process path LICENSE.md
error: README.md: does not exist and --remove not passed
fatal: Unable to process path README.md

Since I am on Windows, I cannot even use --tree-filter with chmod (at least this has not worked for me). When I use --filter-tree instead of --filter-index it works, but I am trying to use --filter-index because it is supposed to be much faster.

Roberto Tyley
  • 24,513
  • 11
  • 72
  • 101
Joseph K. Strauss
  • 4,683
  • 1
  • 23
  • 40

1 Answers1

1

Although your question focuses on git filter-branch to do the rewrite, it's worth considering the BFG instead -although it doesn't do it out of the box, it's a pretty small tweak to add a TreeBlobs cleaner that sets all files non-executable:

https://github.com/rtyley/bfg-repo-cleaner/compare/non-executable

You can build your custom version of the BFG quite easily:

https://github.com/rtyley/bfg-repo-cleaner/blob/master/BUILD.md

Using the non-executable branch above, you can run the BFG without any extra command-line switches, because the cleaner is hardwired in (follow all the usage instructions for the BFG, this is just the core bit):

$ java -jar bfg-custom.jar my-repo.git

This will execute several hundred times faster than git filter-branch (with or without --filter-index) on any large repository - for instance, I did a test run of this custom cleaner on the linux-kernel project (~500k commits), which completed in under 5 minutes.

Full disclosure: I'm the author of the BFG Repo-Cleaner.

Roberto Tyley
  • 24,513
  • 11
  • 72
  • 101
  • I was thinking that it was very inefficient to keep changing the trees for every commit when, for the most part it is the same tree for every commit. I also need to remove bad line endings so that any line that ends in \r, \r\n, or \n\r is changed to \n. I assume that your tool can do this as well? – Joseph K. Strauss Jan 11 '15 at 14:57
  • Is there a full usage instruction for BFG? I only seem to find examples on the usage page, but not actual instructions for all the options. – Joseph K. Strauss Jan 12 '15 at 15:07
  • See my answer here for how to replace Windows newlines with Unix ones: http://stackoverflow.com/a/15730571/438886 . Also, for more detail about usage options, just run the BFG without supplying any arguments - it'll list all options, with a short description of each one. – Roberto Tyley Jan 12 '15 at 22:05
  • I am getting an error when trying to build `[error] (bfg/*:gitDescription) java.io.IOException: Cannot run program "git": CreateProcess error=2, The system cannot find the file specified`. Is it possible just to supply a jar for the branch you created? – Joseph K. Strauss Jan 13 '15 at 03:37
  • The `Cannot run program "git": CreateProcess error=2, The system cannot find the file specified` error sounds weirdly like you don't have `git` on the command line path (I guess because you're using Windows?!?). You can either install git on the path, or remove the line that references it, because it's non-essential. I'm always happy to talk about code and debug issues with my projects, but for custom compiled binaries would charge a fee for my time. – Roberto Tyley Jan 13 '15 at 09:42
  • Yes. I am using Windows, but I figured out that I should try sbt in Git Bash instead of in cmd. This worked, except that it appears that it failed 1/20 tests. `[error] Failed: Total 20, Failed 1, Errors 0, Passed 19 [error] Failed tests: [error] com.madgag.git.bfg.cli.MainSpec [error] (bfg/test:test) sbt.TestsFailedException: Tests unsuccessful`. – Joseph K. Strauss Jan 14 '15 at 03:29
  • I don't have easy access to Windows machines, but I think the BFG _should_ compile on the Windows platform. Could you post the full failing build log to https://github.com/rtyley/bfg-repo-cleaner/issues/new ? CI shows a clean build for that branch: https://travis-ci.org/rtyley/bfg-repo-cleaner/builds/46833625 – Roberto Tyley Jan 14 '15 at 10:01
  • 1
    With the new fix for Windows I had no problem compiling. I was able to clean bad newlines AND executable bit for a repository that had 9,000+ commits in less than 10 minutes. I do not want to even think how long that would have taken with git filter-branch. – Joseph K. Strauss Jan 18 '15 at 16:00