1

Git is staging deleted file inside commit which causes large file error

I've tried the following

  1. .zip file inside .gitignore
  2. git rm -r --cached . 2. git add . 3. git commit
  3. I tried git lfs but didn't impact my commit

command: git push origin master -f

git error:

remote: error: GH001: Large files detected. You may want to try Git Large File Storage - https://git-lfs.github.com. remote: error: Trace: 3b5d9c3622d3987730164be886f8a0a4 remote: error: See http://git.io/iEPt8g for more information. remote: error: File resources.zip is 442.41 MB; this exceeds GitHub's file size limit of 100.00 MB To ... ! [remote rejected] master -> master (pre-receive hook declined) error: failed to push some refs to ..

Resources.zip doesn't exist and yet it is still inside my commit after git cache reset. I also ignore .zip files inside my .gitignore file but it keeps staging them.

How do i remove this non-existing file from the staging list?

Andrew Regan
  • 5,087
  • 6
  • 37
  • 73
DutchPrince
  • 333
  • 2
  • 16
  • 1
    A file once added remains in the git commit (not just in index) forever regardless of `.gitignore` content. The latter is used only to filter out unindexed entries to unclutter the file list. Likely you need to rewrite your commits starting from the point where you added the archive, and clearing it from the commit chain. I suggest to use `git rebase -i ` and use `edit` on the first commit in the chain instead of `pickup`. – user3159253 Sep 06 '19 at 11:37
  • Use `git show ` or a visualization tool like `gitk` to understand what files exist in a given commit. – user3159253 Sep 06 '19 at 11:38
  • 1
    Possible duplicate of [How to remove/delete a large file from commit history in Git repository?](https://stackoverflow.com/questions/2100907/how-to-remove-delete-a-large-file-from-commit-history-in-git-repository) – phd Sep 06 '19 at 12:14
  • https://stackoverflow.com/search?q=%5Bgit%5D+remove+large+file+history – phd Sep 06 '19 at 12:14
  • I cant go back to a previous commit or do a pull because there are 3000 changes in recent commit and folder – DutchPrince Sep 06 '19 at 13:15
  • 2
    Every commit contains a full, complete snapshot of every file. So you have some series of commits ending at (say) commit `H`. You add a big file and make a commit `I`. Then you remove the big file and make commit `J` that refers to `I`. Commit `J` doesn't have the file, but commit `I` still does. Then you make`K` that refers to `J`, etc. You're pushing commits I-J-K-L-... and commit `I` has the big file. You *must* replace *every* commit after `I` with a new and improved one so that commit `I`, which has the big file, no longer exists at all. There is no choice, you *must* do this. – torek Sep 06 '19 at 15:15
  • Very useful explanation! so i had to make a new file with the same name and stage it, will try that next time! I fixed my problem by making a new folder, initialize git and pull all files from the repo, then i overwrote those files with my problem folder without the big files in it and then pushed it to the repo. – DutchPrince Sep 06 '19 at 16:49

1 Answers1

1

Quick Solution

In case that the giant file is in your most recent commit, you can use GitHub's help:

  1. Remove the file from index

    git rm --cached giant_file
# Stage our giant file for removal, but leave it on disk

  2. Commit the change using --amend (to edit the last commit) and -CHEAD (to use its commit message)

    git commit --amend -CHEAD
# Amend the previous commit with your change
# Simply making a new commit won't work, as you need
# to remove the file from the unpushed history as well

  3. Push it

    git push

What if the giant file is in an older commit?

In this case, you should use git rebase to edit the large commit. You need to find that commit first. For example, if the output of git log is like the following:

commit 3906488897bb509c5c345f71e2e70b167efae98b
Author: someone
Date:   Sat Sep 7 11:41:40 2019

Fourth commit

commit 70fcefd350840a132fbff37825090ef672ea98e7
Author: a.esmaeilpour <esmaeilpour@iingroups.com>
Date:   Sat Sep 7 11:40:37 2019

Third commit

commit e5b844b8498f899774302c36a8f4b7f1db311afc
Author: a.esmaeilpour <esmaeilpour@iingroups.com>
Date:   Sat Sep 6 09:23:17 2019

Second commit, A file larger than 100MB added

commit 1de3606dcc803224461c23e3e1b15062589b16b4
Author: a.esmaeilpour <esmaeilpour@iingroups.com>
Date:   Sat Sep 5 09:11:06 2019

First commit

and you have added the giant file in the "Second commit", you run the git rebase command with the just EARLIER COMMIT (First commit) as its parameter:

git rebase -i 1de3606dcc803224461c23e3e1b15062589b16b4

(or alternatively use the Second commit's short hash along with ^ i.e. git rebase -i e5b844b^). Then a text editor is opened and you see something like this:

pick e5b844b Second commit, A file larger than 100MB added
pick 70fcefd Third commit
pick 3906488 Fourth commit
#
# Commands:
# p, pick = use commit
# r, reword = use commit, but edit the commit message
# e, edit = use commit, but stop for amending
# s, squash = use commit, but meld into previous commit
# f, fixup = like "squash", but discard this commit's log message
# x, exec = run command (the rest of the line) using shell
# d, drop = remove commit
#
# These lines can be re-ordered; they are executed from top to bottom.
#
# If you remove a line here THAT COMMIT WILL BE LOST.
#
# You are editing the todo file of an ongoing interactive rebase.
# To continue rebase after editing, run:
#     git rebase --continue
#

Replace pick with edit for the commit that must be edited, i.e. Second commit. Save the changes and quit the editor. Now remove the giant file from the index:

git rm --cached giant_file

Save the change to the commit:

git commit --amend --allow-empty

(--allow-empty lets the commit proceeds even if the giant file was the only file in that commit that now has been removed from the commit.) An editor is opened again so you can modify the commit message. After editing the message and closing the editor, run the following command:

git rebase --continue

The commit is successfully edited.

Note: This command rewrites the history and this solution is for the case that some recent commits have not been pushed to a GitHub repository due to being a file greater than 100MB in a commit. If someone has a different scenario and has pushed a commit which now they want to edit it, they should consider that their teammates may already have pulled the previous version of the commit and may face merge conflicts when pulling from the repo after the rewritten commit is pushed.

Now you can push it to GitHub:

git push

Why did not your solution work?

.gitignore does not affect files which already tracked by Git. So the commit which was already in the commit history didn't change. You tried to modify that commit by running git rm -r --cached, but you didn't amend it and by running git add, created another change to be committed. Consequently, the giant file remained in the commit.

Abdollah
  • 4,579
  • 3
  • 29
  • 49
  • 1
    Two comments. I think using `e5b844b8^` is a better target for the commit, because the commit id is the one you're targeting and you're using the `^` to get to the commit before it. Also, there's no mention of `git rebase` rewriting history. If this is somehow shared with others or a fork of a repo that actually lives elsewhere, it could present problems in the future when trying to keep up-to-date, or others try to pull from it. I always encourage people to understand the full impacts of history rewriting and rebase as it can lead to some ugly issues if you don't. – John Szakmeister Sep 07 '19 at 09:39
  • @ John Szakmeister, thank you for your comments. I applied both of them. However, as I mentioned in the answer, the solution works fine for the scenario explained in the question that someone could not have pushed their local commits without any side effects on their teammates. – Abdollah Sep 07 '19 at 10:30