Questions tagged [bfg-repo-cleaner]

BFG Repo-Cleaner is an alternative to Git's native filter-branch tool that's designed to perform branch filtering significantly faster.

The BFG Repo-Cleaner is an alternative to Git's native git filter-branch that's made for cleansing unwanted data (big files, passwords) out of Git repository history. It's designed to be faster and simpler to use, and is written in Scala.

The BFG Repo-Cleaner is even officially endorsed as an alternative in the official Git documentation for filter-branch itself:

git-filter-branch allows you to make complex shell-scripted rewrites of your Git history, but you probably don’t need this flexibility if you’re simply removing unwanted data like large files or passwords. For those operations you may want to consider The BFG Repo-Cleaner, a JVM-based alternative to git-filter-branch, typically at least 10-50x faster for those use-cases, and with quite different characteristics:

  • Any particular version of a file is cleaned exactly once. The BFG, unlike git-filter-branch, does not give you the opportunity to handle a file differently based on where or when it was committed within your history. This constraint gives the core performance benefit of The BFG, and is well-suited to the task of cleansing bad data - you don’t care where the bad data is, you just want it gone.

  • By default The BFG takes full advantage of multi-core machines, cleansing commit file-trees in parallel. git-filter-branch cleans commits sequentially (ie in a single-threaded manner), though it is possible to write filters that include their own parallellism, in the scripts executed against each commit.

  • The command options are much more restrictive than git-filter branch, and dedicated just to the tasks of removing unwanted data- e.g: --strip-blobs-bigger-than 1M.

See also the git-rewrite-history tag on Stack Overflow.

160 questions
3
votes
2 answers

git svn fetch gives "Invalid revision range" "error: 128" after bfg cleanup

I have a Git repo in migration from SVN with git-svn. I used git svn fetch to get the latest commits from SVN. I wanted to push the repo to GitHub, but there were some files in the history larger than 100 MB that I had to remove, so I used bfg repo…
Alex Burdusel
  • 3,015
  • 5
  • 38
  • 49
3
votes
1 answer

How to migrate existing Git-p4 repo to LFS

For this moment I have a Git repo that synced to the main p4 depot and works fine. But its size is far more than 4Gb. And we need to move it to BitBucket, where 2Gb is the maximum limit. What I want to achieve: Put 1 folder and 1 file extension…
orkenstein
  • 2,810
  • 3
  • 24
  • 45
3
votes
1 answer

BFG which repo to use after successful run

I followed the documentation for using BFG repo cleaner I've read "It's best to delete all old clones" but not sure which one and HOW to delete it. I'm pushing everything to bitbucket. I have 2 repos's dirs now /.git /.myproject.git/ Should I just…
Alex Angelico
  • 3,710
  • 8
  • 31
  • 49
3
votes
1 answer

Migrating existing Git repo to new Git LFS enabled repo

My existing Github repo is getting bloated with zip and tar.gz files so I want to migrate it to a new Git LFS enabled repo that is separate from existing Github repo for testing so that existing Github repo is untouched. I came across a wonderful…
p4guru
  • 1,400
  • 2
  • 19
  • 25
2
votes
2 answers

Why can I still see the file after deleting it using BFG

This are the steps I did: Created an empty folder. Mirrored my repository using: git clone --mirror git@bitbucket.org:somespace/myrepo.git Got a list of 10 largest file using the following command: git rev-list --objects --all \ | git cat-file…
Vicky
  • 16,679
  • 54
  • 139
  • 232
2
votes
1 answer

bgf cleaner does not update pull requests

I am using bfg-cleaner to delete some files from github containing senstive information. it works fine except for pull requests. the sensitive date still exists in pull requests . How can I get rid of this?
user2636464
  • 685
  • 7
  • 17
2
votes
2 answers

How to remove binary files from all of the bitbucket git history, when the repo is already very close to 2GB?

Bitbucket git repo has a size limit of 2GB, and now I have one repo ( let's call it bigsize repo) that is already dangerously close to that limit, due to a lot of binary files ( files with extensions of dll and msm). It's so close to the limit that…
Graviton
  • 81,782
  • 146
  • 424
  • 602
2
votes
1 answer

BFG ERROR: BFG aborting: No refs to update - no dirty commits found

I have a file in my GitHub repository, that I would like to remove/rewrite all history related to the file. so i have taken all commits related to the commits kept it in one file. Trying to use BFG approach. since my requirement is want to rewrite…
Ravinder
  • 21
  • 1
  • 2
2
votes
1 answer

How to migrate branches from one repo to another with different histories?

Right now I'm attempting to slim down a bloated repo by deleting git objects associated with long-deleted binary files in source control. I'm using the BFG repo cleaner, which is succeeding at slimming the repo and leaving the current commit…
Answorth
  • 147
  • 1
  • 2
  • 10
2
votes
2 answers

Can't push to >2gb repo after clearing files with BFG

I accidentally pushed some large files that were supposed to be ignored to git and as a result my repo is over 2gb. I'm trying to clear some files out with BFG and have been able to clear around 400mb but when trying to push I get the following…
Joe Scotto
  • 10,936
  • 14
  • 66
  • 136
2
votes
2 answers

Create a new repository from existing of a subdirectory with only relevant files and commits

I have a monorepo with many different packages in it. I'd like to take a package and create a new repo for it. I would like to preserve commits pertaining to the files within the project folder. I've tried using git filter-branch with the…
ThomasReggi
  • 55,053
  • 85
  • 237
  • 424
2
votes
1 answer

BFG restore deleted files in history

BFG was used to delete files from history. During this process a deleted-files.txt was written. A former clone still contains the deleted files... git cat-file -p xxx (xxx as stated in deleted-files.txt) Is there a way to restore the original…
lunanigra
  • 31
  • 4
2
votes
1 answer

Removing nested folders with BFG Repo Cleaner

There is one large directory, called docs, with a lot of nested directories, some of which have 2 more levels of directories inside them, that I want to get rid of entirely. Running java -jar ~/bfg-1.12.15.jar --delete-folders docs actually made the…
kim holder
  • 1,744
  • 3
  • 16
  • 25
2
votes
1 answer

Delete multiple files from multiple branch using bfg repo cleaner

I want to delete multiple files from multiple branches using bfg repo cleaner. Can anyone please suggest way how to do that. I am able to delete single file from single branch using the command bfg --delete-files filename my-repo.git.Please provide…
Lalit Dashora
  • 467
  • 5
  • 16
2
votes
2 answers

Unable to convert git repo to use large file support (lfs) via the bfg tool

I am trying to convert an existing git repo to use git lfs. I am following Atlassian's recipe, using the BFG tool here: https://rtyley.github.io/bfg-repo-cleaner/ The problem is that when I issue the suggested command to do the actual conversion, it…
Spiralis
  • 3,232
  • 2
  • 39
  • 53