Questions tagged [bfg-repo-cleaner]

BFG Repo-Cleaner is an alternative to Git's native filter-branch tool that's designed to perform branch filtering significantly faster.

The BFG Repo-Cleaner is an alternative to Git's native git filter-branch that's made for cleansing unwanted data (big files, passwords) out of Git repository history. It's designed to be faster and simpler to use, and is written in Scala.

The BFG Repo-Cleaner is even officially endorsed as an alternative in the official Git documentation for filter-branch itself:

git-filter-branch allows you to make complex shell-scripted rewrites of your Git history, but you probably don’t need this flexibility if you’re simply removing unwanted data like large files or passwords. For those operations you may want to consider The BFG Repo-Cleaner, a JVM-based alternative to git-filter-branch, typically at least 10-50x faster for those use-cases, and with quite different characteristics:

  • Any particular version of a file is cleaned exactly once. The BFG, unlike git-filter-branch, does not give you the opportunity to handle a file differently based on where or when it was committed within your history. This constraint gives the core performance benefit of The BFG, and is well-suited to the task of cleansing bad data - you don’t care where the bad data is, you just want it gone.

  • By default The BFG takes full advantage of multi-core machines, cleansing commit file-trees in parallel. git-filter-branch cleans commits sequentially (ie in a single-threaded manner), though it is possible to write filters that include their own parallellism, in the scripts executed against each commit.

  • The command options are much more restrictive than git-filter branch, and dedicated just to the tasks of removing unwanted data- e.g: --strip-blobs-bigger-than 1M.

See also the git-rewrite-history tag on Stack Overflow.

160 questions
4
votes
2 answers

Git Repository Only Gets Bigger After Using BFG

We are currently in the process of migrating our SVN repo to GIT (hosted at bitbucket). I used subgit to import all our branches/history into a bare repo i have locally on my (Windows) PC. The repo is quite big (7.42 GB after the import) this is…
Piet
  • 91
  • 1
  • 8
4
votes
1 answer

BitBucket repo larger after using BFG repo cleaner

My BitBucket repo had a whole bunch of large files in that weren't needed. I removed them and then wanted to clear them out from the history to shrink down the repo which had gotten too big. I ran BFG repo cleaner which reported 1755 files found and…
LizW
  • 41
  • 1
4
votes
1 answer

Why is BFG changing my latest commit?

git filter-branch was taking a long time. Happily, I found BFG repo-cleaner. But it is unexpectedly changing the contents of my last commit. $ git clone --mirror example.com:/repo.git $ cd repo.git $ git log HEAD^! commit…
Paul Draper
  • 78,542
  • 46
  • 206
  • 285
3
votes
1 answer

remote: GitLab: Push operation timed out

I'm trying to fix my repository with BFG Repo-Cleaner, by removing some large files that were added to it, using this guide: https://rtyley.github.io/bfg-repo-cleaner/. When I'm trying to push (a very large push), I'm getting: "remote: GitLab: Push…
EladG
  • 63
  • 5
3
votes
1 answer

Delete list of files with BFG-repo-cleaner

We are trying to shrink our git repository to under 500MB due to deployment issues. To achieve that, we have created a new branch where we have moved all old images, videos and fonts to AWS S3. I can easily get the list of files with git diff…
dotnetCarpenter
  • 10,019
  • 6
  • 32
  • 54
3
votes
1 answer

ALL commits are duplicated after BFG (history editing)

I didn't know about the clean way to remove big/sensitive files with BFG, and missed a critical step: git clone --mirror git://example.com/some-big-repo.git Which, when I tried to push to remote, lead to history conflicts, which I naively resolved…
JelenaČuklina
  • 3,574
  • 2
  • 22
  • 35
3
votes
1 answer

What are the parameters for using bfg

I am trying to clean something out of my git repo using bfg. The author's instructions suggest java -jar bfg --replace-text passwords.txt my-repo.git I understood these to mean java -jar bfg.jar --replace-text But all…
Simon H
  • 20,332
  • 14
  • 71
  • 128
3
votes
1 answer

How to delete old commits without affecting history

I need to delete commits made 1 year ago because they contain sensitive data that must be removed. I have used BFG Repo-Cleaner, and I have been able to almost delete everything, but there are some very old commits that are not being removed. I will…
rogervila
  • 974
  • 1
  • 10
  • 27
3
votes
2 answers

BFG Repo Cleaner - Deleted Files Still In Commit Changes

I need to delete configuration files from my web application and from the git history. I've done that using the bfg repo cleaner tool. I went through the steps outlined in the documentation: git clone --mirror url.git java -jar bfg-1.13.0.jar…
tamster008
  • 33
  • 5
3
votes
1 answer

Unable to Push After bfg Cleaner

Followed these steps to remove some content from my repository. However, when I did a git push, I got the below message: Writing objects: 100% (28/28), 6.72 KiB | 1.34 MiB/s, done. Total 28 (delta 18), reused 18 (delta 9) remote:…
Di Yao
  • 31
  • 1
  • 2
3
votes
1 answer

How to rewrite GIT history to track an entire directory with GIT-LFS

I tried a lot of commands but it does not work: java -jar bfg-1.13.0.jar --convert-to-git-lfs 'lib/**' --no-blob-protection java -jar bfg-1.13.0.jar --convert-to-git-lfs 'lib' --no-blob-protection java -jar bfg-1.13.0.jar --convert-to-git-lfs…
Alpes Maritimes
  • 111
  • 1
  • 6
3
votes
1 answer

BFG doesn't update github pull requests

so I ran BFG, following the instructions here. https://rtyley.github.io/bfg-repo-cleaner/ The console output from BFG looked good but, after pushing, when I got back to github, I could still find the sensitive data in pull requests > clsoed > old…
honkskillet
  • 3,007
  • 6
  • 31
  • 47
3
votes
2 answers

Correct Usage of BFG Repo Cleaner

The BFG Repo Cleaner site gives an example of using the tool as follows to clean up a repository: Clone a fresh copy of your repo. $ git clone --mirror git://example.com/some-big-repo.git Run BFG to clean up your repo. $ java -jar bfg.jar…
Bill Greer
  • 3,046
  • 9
  • 49
  • 80
3
votes
1 answer

Why repository size doubles when i copy notes?

I import SVN repository using Subgit, which is an excellent tool doing it fast and supporting custom svn layout. Subgit saves git commit -> svn revision reference in git notes. Every commit has revision number in notes, you can see it with git log.…
Kirill
  • 6,762
  • 4
  • 51
  • 81
3
votes
1 answer

Why can I still see files in GitHub history after cleaning them with the BFG?

I am working on a group project and I want to remove a file from all memory. The content, the file name, everything! I don't want any trace of this left on the Git repo. I have been trying to do this using bfg but I can still find the file on the…
oliversm
  • 1,771
  • 4
  • 22
  • 44
1 2
3
10 11