Questions tagged [bfg-repo-cleaner]

BFG Repo-Cleaner is an alternative to Git's native filter-branch tool that's designed to perform branch filtering significantly faster.

The BFG Repo-Cleaner is an alternative to Git's native git filter-branch that's made for cleansing unwanted data (big files, passwords) out of Git repository history. It's designed to be faster and simpler to use, and is written in Scala.

The BFG Repo-Cleaner is even officially endorsed as an alternative in the official Git documentation for filter-branch itself:

git-filter-branch allows you to make complex shell-scripted rewrites of your Git history, but you probably don’t need this flexibility if you’re simply removing unwanted data like large files or passwords. For those operations you may want to consider The BFG Repo-Cleaner, a JVM-based alternative to git-filter-branch, typically at least 10-50x faster for those use-cases, and with quite different characteristics:

  • Any particular version of a file is cleaned exactly once. The BFG, unlike git-filter-branch, does not give you the opportunity to handle a file differently based on where or when it was committed within your history. This constraint gives the core performance benefit of The BFG, and is well-suited to the task of cleansing bad data - you don’t care where the bad data is, you just want it gone.

  • By default The BFG takes full advantage of multi-core machines, cleansing commit file-trees in parallel. git-filter-branch cleans commits sequentially (ie in a single-threaded manner), though it is possible to write filters that include their own parallellism, in the scripts executed against each commit.

  • The command options are much more restrictive than git-filter branch, and dedicated just to the tasks of removing unwanted data- e.g: --strip-blobs-bigger-than 1M.

See also the git-rewrite-history tag on Stack Overflow.

160 questions
0
votes
1 answer

If credentials are removed from a repo by amending the commit that added them, and subsequent commits are reapplied, is the reflog still tainted?

I've used BFG Repo-Cleaner a couple times to remove credentials that were added dozens/hundreds of commits in the past, and that tool works well, but for smaller-scale situations like the following, I'm wondering if that is necessary. Suppose…
Jordan Bradford
  • 337
  • 2
  • 8
0
votes
0 answers

How to reduce the size of a git repository using bfg

I have a git repository that weighs around 5g. The .git directory weighs 4.5G itself. My goal is to remove all big files from this history in order to don't have 5G to download when doing a clone. To do that I wanted to use the BFG open-source…
bloub
  • 510
  • 1
  • 4
  • 21
0
votes
1 answer

"failed to push some refs" error after using BFG to clean repo

I have a GitHub repository that I am trying to shrink. I've used the BFG repo cleaner to do this, followed by the typical gc commands. This did reduce my repository size (though it's still 3x larger than it should be – but that's another…
Brian Barry
  • 439
  • 2
  • 17
0
votes
1 answer

Why does tool bfg.jar remove files smaller as 1MB from Git with both options "--strip-blobs-bigger-than and --delete-files"?

I'm using both option --strip-blobs-bigger-than 1M --delete-files '*.{jpg,png}' together with bfg.jar tool. But some smaller as 1M files were deleted. Aren't these options working together ?
Gr eek
  • 9
  • 2
0
votes
1 answer

migrating to git lfs by official guide creates bad object error

Following this guide, when I get to the step git push --force I get the following error: ref master:: Error in git rev-list --stdin --objects --not --remotes=origin --: exit status 128 fatal: bad object…
Joey Gough
  • 2,753
  • 2
  • 21
  • 42
0
votes
0 answers

Issues Cleaning Up a Git Repository

I have a private remote git repository, which I have been using for a project for years. In the course of some testing and bug fixing, I inadvertently uploaded some very large files (MySQL dumps, each dump > 100 MB) and 2 directories with test files…
user1045680
  • 815
  • 2
  • 9
  • 19
0
votes
1 answer

BFG Repo Cleaner for items re-introduced to history after a previous successful purge

If an item was removed by BFG cleaner from history, then someone didn't purge their old clone, but instead just deleted the file and then later pushed, will BFG Cleaner still be able to purge that same file again if I use just the filename? Or do I…
MarkTO
  • 247
  • 2
  • 11
0
votes
1 answer

How to remove a deleted file from git history - including rev-list

I have tried filter-branch, and got to the point where it is removed from git log, but still in rev-list. Then I tried bfg repo cleaner and it still is listed in rev-list. Ultimately I need it to be removed from rev-list because the file is missing…
Justin Pihony
  • 66,056
  • 18
  • 147
  • 180
0
votes
1 answer

GIT LFS "failed to push some refs to"

I have a current GIT repository hosted via TFS. We are planning on migrating to LFS to manage some large binaries that are stored via source control. I am cloning the TFS repository to my local PC, and then cloning the clone so that I can…
BFlat
  • 28
  • 6
0
votes
0 answers

bfg to clean large directory in git repo

I am cleaning large git repository using BFG Repo-Cleaner. In particular, I want to remove a directory. I do mirror clone as mentioned in https://rtyley.github.io/bfg-repo-cleaner/ I am using command java -jar bfg-1.13.0.jar --delete-folders FolderA…
cmp
  • 35
  • 6
0
votes
1 answer

BFG Repo-Cleaner - How to delete folders except some folders

How to delete folders except some folders? My command is java -jar ../BFG_Repo-Cleaner/bfg-1.13.0.jar --delete-folders "!(folder1|folder2|folder3)" I want all folders except folder1, folder2 and folder3.
0
votes
3 answers

Migrating unpushed local commits to new repository

We have a problem that our repository size exceeded the allowed limit and we are now blocked to push any local commit. So we used bfg to remove big files and re-write the whole history of the repository then pushed it to a new one, in meanwhile…
Mahmoud Adam
  • 5,772
  • 5
  • 41
  • 62
0
votes
2 answers

Test Git repo size reduction after deleting a branch

Is there any way I can test how the deletion of several branches will affect the size of that git repo. I have a git repo with 10 lived branches and hundreds of feature (short lived branches). The repo is very large and slow to clone. GitLab shows…
ptha
  • 836
  • 1
  • 8
  • 22
0
votes
2 answers

Removing history using BFG Cleaner not working

Removing history of deleted folders using BFG I am using BFG to delete history of deleted folders completely from repo git rev-list --all --objects -- path/to/the/directory/to/delete | git cat-file --batch-check='%(objectname) %(objecttype) %(rest)'…
Rohit
  • 1
  • 1
0
votes
1 answer

Why does my remote repo increase in size after bfg clean up?

Used BFG to clean repo with large files. I tried to follow step from bfg tutorial page. Steps I followed: git clone --mirror myrepo.git java -jar bfg-1.13.0.jar --no-blob-protection --delete-folders "{large_folder}" test_repo.git git reflog expire…
mato
  • 503
  • 8
  • 18