Questions tagged [bfg-repo-cleaner]

BFG Repo-Cleaner is an alternative to Git's native filter-branch tool that's designed to perform branch filtering significantly faster.

The BFG Repo-Cleaner is an alternative to Git's native git filter-branch that's made for cleansing unwanted data (big files, passwords) out of Git repository history. It's designed to be faster and simpler to use, and is written in Scala.

The BFG Repo-Cleaner is even officially endorsed as an alternative in the official Git documentation for filter-branch itself:

git-filter-branch allows you to make complex shell-scripted rewrites of your Git history, but you probably don’t need this flexibility if you’re simply removing unwanted data like large files or passwords. For those operations you may want to consider The BFG Repo-Cleaner, a JVM-based alternative to git-filter-branch, typically at least 10-50x faster for those use-cases, and with quite different characteristics:

  • Any particular version of a file is cleaned exactly once. The BFG, unlike git-filter-branch, does not give you the opportunity to handle a file differently based on where or when it was committed within your history. This constraint gives the core performance benefit of The BFG, and is well-suited to the task of cleansing bad data - you don’t care where the bad data is, you just want it gone.

  • By default The BFG takes full advantage of multi-core machines, cleansing commit file-trees in parallel. git-filter-branch cleans commits sequentially (ie in a single-threaded manner), though it is possible to write filters that include their own parallellism, in the scripts executed against each commit.

  • The command options are much more restrictive than git-filter branch, and dedicated just to the tasks of removing unwanted data- e.g: --strip-blobs-bigger-than 1M.

See also the git-rewrite-history tag on Stack Overflow.

160 questions
2
votes
1 answer

Can BFG repo cleaner perform arbitrary operations on files?

I want to squash images using tools like imageoptim-cli on MacOS from git history to shrink repo size. Can BFG support running some more arbitrary commands? My git filter-branch looked something like this:…
zbeekman
  • 648
  • 7
  • 15
2
votes
1 answer

BFG: mapping from old (pre-cleaning) to new hashes

We use git hashes to point to specific revisions, e.g., in an issue tracking system. When cleaning a repo with BFG, all/many hashes change, so that information is lost. Can BFG, as part of cleaning a repo, produce a mapping between old and new…
Ulrich Scholz
  • 2,039
  • 4
  • 30
  • 51
2
votes
1 answer

git lfs bfg: after that, resolve conflicts how?

We have a repository in which we committed PDF snapshots of reports. I want to try out git lfs, see if it improves the quality of life. I followed the procedures here (https://github.com/rtyley/bfg-repo-cleaner/releases) to use BFG to clean out the…
pauljohn32
  • 2,079
  • 21
  • 28
2
votes
2 answers

How to import git repositories with large files?

Given that GitHub doesn't allow to push files larger than 100 MB, it is not possible to git clone and push a repository with large files into GitHub enterprise. The push fails with a: remote: error: GH001: Large files detected. You may want to try…
Alberto
  • 5,021
  • 4
  • 46
  • 69
2
votes
2 answers

How to remove big (>100MB) file from a GitHub repository and push successfully?

I am in the same situation as described here after having inadvertently adding a big file that I don't want and having done additional commits of other work (not knowing the push would fail) after inadvertently adding the big file: Am I supposed to…
TrailDreaming
  • 507
  • 1
  • 5
  • 16
2
votes
1 answer

Remove folder from git history: BFG tool deletes all branches

I need to delete the folder cache from all branches. I'm using the BFG tool, however it removes all branches but master. Here's what I do. Step 1: Create a mirror from local clone (Verify that there is many branches in my local…
FooBar
  • 15,724
  • 19
  • 82
  • 171
2
votes
1 answer

git: --prune-empty after using bfg duplicates commits

I am using bfg to remove some subdirectories from a (clone of) git repository: java -jar bfg-1.12.12.jar --delete-folders {folder1,folder2,folder3} --no-blob-protection myrepo.git/ git reflog expire --expire=now --all && git gc --prune=now…
fuenfundachtzig
  • 7,952
  • 13
  • 62
  • 87
2
votes
1 answer

Slim down a git repository with bfg

Faced with anarchic add of binary files by a coder, how to slim down a git repository not only removing the problematic files but also their history in the tree. I tried using bfg but as it works on mirrored bare repository I've been faced with…
cmbarbu
  • 4,354
  • 25
  • 45
2
votes
2 answers

How to batch-replace *all* instances (content, filenames and commit messages) of *Foo* to *Bar* in a repo in a single, simple step?

Suppose I have a giant repo for an as-of-yet unpublished software product called "Hammerstein", written by the famous German software company "Apfel" of which I am an employee. One day, "Apfel" spins out the Hammerstein division and sells it to the…
2
votes
0 answers

Anyway around a fresh clone when using BFG Repo-Cleaner?

When using BFG Repo-Cleaner is there any way around not having everyone do a fresh clone? With a large team and multiple branches it is difficult to organize this. I am willing to run bfg multiple times should something be reintroduced as long as I…
David Vasandani
  • 1,840
  • 8
  • 30
  • 53
2
votes
1 answer

Remove unused large files from Git within a range

My repo is forked from an open sourced project, so I don't want to modify the commits before the ForkPoint tag. I've tried the BFG Repo Cleaner but it doesn't allow me to specify a range. I want to Go through the history in ForkPoint..HEAD^ Rewrite…
1
vote
0 answers

Removing large files from branch commit history using BFG Repo Cleaner

I made a pull request on a repository and unfortunately, my branch contained some large files which are now present in my branch's commit history. To address this, I decided to use BFG Repo Cleaner. However, I'm unsure about the correct next…
kklaw
  • 452
  • 2
  • 4
  • 14
1
vote
1 answer

remove permanently a commit on bitbucket

I'm trying to remove usernames from the bitbucket repositories for security reasons I cloned the repo using --mirror, did the following command bfg --replace-text username.txt , changed the directory to the repo ran this command git reflog expire…
Cool_Name
  • 21
  • 3
1
vote
0 answers

using bfg or filter-repo, I want to replace text in specific file

my git repo has above files with directory, service-a app.config service-b app.config I want to replace the text in service-a/app.config, not service-b/app.config using bfg or filter-repo. but those looks don't support it. How can I do…
rura6502
  • 365
  • 2
  • 15
1
vote
1 answer

How to use BFG Repo-Cleaner

I've been advised to use the BFG Repo-Cleaner as my local repo that I want to push contains files too large to push to GitHub. These files (above about 50MB) I don't mind if they get deleted and I accidentally committed them a while back. On the…
Socorro
  • 21
  • 3