I'd like to add to this.
I often find that for very large commits (for example let's 20GB worth of files)
git lfs will fail partially through a push if there is any interruption in network connectivity at all. It takes a long time to push 20GB of files, so this can easily happen.
I wrote these scripts (both mac and Windows) to push each file individually, so that each upload is atomic. If it does fail, you can at least restart from where you were, and with atomic pushes, it is less likely to fail on an individual push.
Windows (run on Git bash)
#!/bin/sh
# This script is for adding files to Git LFS and then pushing them to a repo.
# It assumes you have already initalized git lfs.
#
# Why do you need this script?
# If you need to push a loooooooot of dependencies to Git LFS.
# If you try to push all your files at once in a single commit, any sort of
# interruption in network connectivity could cause the push to fail and then
# you will have to start all over again. This happened to me several times as I # tried to push 24 GB of files via git lfs to a repository. I wrote this script
# to commit and then push each file individually so I could get past that hurdle
# and move on to other things.
# Cons: this makes the commit history rather long, but after you do it once,
# it should be smooth sailing from there. I've never had this con cause me any
# trouble.
# global var definitions
dir=/c/Projects/path/to/local/deps
branch="my-branch"
# function definitions
function add_file_and_push_to_github() {
branch="my-branch"
echo "in here: $1"
git add "$1"
echo "after add"
git commit -m "add $1"
echo "after commit"
git pull origin "${branch}"
echo "after pull ${branch}"
git push origin "${branch}"
echo "after push to ${branch}"
}
rm -rf .git
rm README.md
git init
git remote add origin https://github.com/sitting-duck/MyDependenciesRepo.git
git pull origin master
git checkout -b "$branch"
git lfs install
git add .gitattributes
echo "before push -u $branch"
git push -u origin "$branch"
echo "after push -u $branch"
git lfs track *.lib
git lfs track *.dll
git pull origin "$branch"
export -f add_file_and_push_to_github
find . -type f -print0 | xargs -0 bash -c 'add_file_and_push_to_github "$@"'
MacOS (run in Terminal app)
#!/bin/sh
# global var definitions
dir=/Projects/path/to/dependencies/
branch="my-branch"
rm -rf .git
rm README.md
git init
git remote add origin https://github.com/sitting-duck/myRepo.git
git pull origin master
git checkout -b "$branch"
git lfs install
git add .gitattributes
echo "before push -u $branch"
git push -u origin "$branch"
echo "after push -u $branch"
git lfs track *.dylib
git pull origin "$branch"
while IFS="" read -r p || [ -n "$p" ]
do
printf '%s\n' "$p"
if [[ p != *".git"* ]]; then
branch="my-branch"
echo "in here: $p"
git add "$p"
echo "after add"
git commit -m "add $p"
echo "after commit"
git pull origin "${branch}"
echo "after pull ${branch}"
git push origin "${branch}"
echo "after push to ${branch}"
fi
done < files.txt
