How to delete folder from git history permanently using BFG

Question

I am using BFG to delete two of my folders. They were tracked from the inception of the repo. The reason to delete was those folder contains binary and other txt file that we no longer need. But When I try to delete those two folder, one gets deleted but the other one still ligers around.

I created my dummy repo and did some commits to recreate the problem. I assume myRepo.git is clean repo to begin with.

Function I used to delete folders is : java -jar bfg-1.13.0.jar --no-blob-protection --delete-folders "{system1, system2}" myRepo.git

#!/bin/bash


BUILD(){
  git clone https://github.com/xxxx/myRepo.git
  cd myRepo
  echo "Jpt test1" > jpt1.txt
  echo "Jpt test2" > jpt2.txt
  echo "Jpt test3" > jpt3.txt

  git add jpt1.txt jpt2.txt jpt3.txt
  git commit -m "first commit"
  git push origin master
  ######
  mkdir system1
  cd system1
  mkfile 14m outputfile1.out
  mkfile 14m outputfile2.out
  echo "Jpt test1" > sysjpt1.txt
  echo "Jpt test2" > sysjpt2.txt
  echo "Jpt test3" > sysjpt3.txt
  cd ..
  ######
  mkdir system2
  cd system2
  mkfile 14m outputfile1.out
  mkfile 14m outputfile2.out
  cd ..

  git add system1 system2
  git commit -m "tracking large file"
  git push origin master
  cd ..

  ##### Call function BFG which does BFG stuff. 
  BFG

}

BFG(){
  # run bfg and let git clean history 

  git clone --mirror https://github.com/xxxx/myRepo.git

  java -jar bfg-1.13.0.jar --no-blob-protection --delete-folders "{system1, system2}" myRepo.git 

  cd myRepo.git
  git reflog expire --expire=now --all && git gc --prune=now --aggressive
  git push 

  cd ..
  mkdir test_new
  cd test_new
  git clone https://github.com/xxx/myRepo.git
  cd myRepo
  ls


}

BUILD

When I clone after cleaning and do ls on it. I get jpt1.txt jpt2.txt jpt3.txt system2. See how system2 folder is still there.

Just to clarify, `system2` is still there, but are the files also still in it? Also, I don't like `"{system1, system2}"`. By putting quotes around that, you're passing that string as is to bfg, rather than have the shell expand it. The space after the comma also seems like a bad idea. Have you tried running it in two commands instead, one per folder? — joanis, May 27 '19 at 18:31
Yes the other files are also there. I will try ur suggestion and let you know. — mato, May 27 '19 at 23:29
@joanis So I tried like how you said, by doing them separately and it worked. The problem probably is in `"{system1, system2}"` part of the script, like you said. — mato, May 28 '19 at 00:31
After playing more with the the command `java -jar bfg-1.13.0.jar --no-blob-protection --delete-folders "{system1, system2}" myRepo.git` and re-reading your comment `The space after the comma also seems like a bad idea.`. I tried without space and it seems to work. So I can confirm that the space was the problem. So the final command I ran was `java -jar bfg-1.13.0.jar --no-blob-protection --delete-folders "{system1,system2}" myRepo.git` See how there is now space `"{system1,system2}". — mato, May 28 '19 at 15:15
OK, thanks for letting me know. I'll write it up as an answer then. — joanis, May 28 '19 at 19:49

score 1 · Accepted Answer · answered May 28 '19 at 20:07

As you tested from my suggestions in the comments above, the problem is with the space in "{system1, system2}". When that expression is processed, it expands it to "system1" and " system2", with a space in front of system2, which is not what you want.

You can run the process in two commands, once with system1 and once with system2, or simply remove the space, and things will work.

Interestingly, the expansion of {a,b} appears to be done by bfg itself, not by bash: the quotes tell bash to pass that string literally, so although this looks like bash syntax it actually isn't a bash expansion.

How to delete folder from git history permanently using BFG

1 Answers1