git - conditionally commit generated artifacts

Question

In my gitlab-ci build, I am generating documentation text files corresponding to the source commits. I would then like to commit these files to the repository so that they will be available for review by developers. The build process also generates executables from source, but those are not added to source control.

I followed the suggestion here to implement the initial commit of the text files: https://forum.gitlab.com/t/how-to-deploy-the-artifact-of-a-ci/38198/3

The relevant lines of my gitlab-ci script are below

- git config --global user.name project_bot
- echo "Committing with token $TOKEN"
- for file in ./*.txt; do echo "*** COMMITTING ${file}"; git add ${file}; git commit ${file} -m "Updated ${file}"; git push --push-option=ci.skip https://project_bot:$TOKEN@our.internal.gitlab/devgroup/project HEAD:${CI_COMMIT_BRANCH}; done

For the initial commit, this went fine, and I received this output (anonymized):

$ for file in ./*.txt; do echo "*** COMMITTING ${file}"; git add ${file}; git commit -m "Updated ${file}"; git push https://project_bot:$TOKEN@our.internal.gitlab/devgroup/project HEAD:${CI_COMMIT_BRANCH}; done
*** COMMITTING ./file1.txt
[detached HEAD 575de7c] Updated ./file1.txt

Subsequent builds, however, fail as shown below:

$ for file in ./*.txt; do echo "*** COMMITTING ${file}"; git add ${file}; git commit -m "Updated ${file}"; git push https://project_bot:$TOKEN@our.internal.gitlab/devgroup/project
*** COMMITTING ./file1.txt
HEAD detached at d2bfb93
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
    modified:   file2.txt
    modified:   file3.txt
Untracked files:
  (use "git add <file>..." to include in what will be committed)
    file1.exe
    file2.exe
    file3.exe
no changes added to commit (use "git add" and/or "git commit -a")
Cleaning up file based variables
00:00
ERROR: Job failed: exit status 1

I am not clear as to why this is failing, but I'm guessing it is because I am attempting to commit a file that has not been changed. Is there a way to only run the commit if the file contents have changed?

You want to commit each file separately!? well, that would be a lot of commits. Anyway - see https://stackoverflow.com/questions/47154114/how-to-reattach-a-detached-head-in-git — KamilCuk, Jun 03 '21 at 16:21
Whatever the reviewer is looking for should probably be codified in a script that the CI can run to verify the output automatically, rather than commit something that doesn't belong in the repository in the first place. — chepner, Jun 03 '21 at 18:28
You generally shouldn't commit build products to your repository. That tends to cause a lot of bloat and needless merge conflicts down the line. It's better to upload them to a suitable server if you need them visible by developers. — bk2204, Jun 03 '21 at 23:57

score 0 · Answer 1 · answered Jun 03 '21 at 18:18

The problem is that if file1.txt does not change in the process, there is nothing to commit. Default behaviour of Git is to fail in this case.

Solution if you only want a commit when the file actually changes: Probe first whether the file has differences and only handle the file if this is the case. The check can be done with git diff --exit-code -s ${file} which omits all the output (-s) and uses exit codes (--exit-code). An exit code of 1 means there were differences and 0 means no differences.

Your script line can look like this (expanded form):

for file in ./*.txt; do
    git diff --exit-code -s ${file};
    if [ "$?" = 1 ]; then
       echo "*** COMMITTING ${file}";
       git add ${file};
       git commit -m "Updated ${file}"
       git push https://project_bot:$TOKEN@our.internal.gitlab/devgroup/project 
    else
       echo "*** SKIPPING   ${file}";
    fi
done

Sidenote: Probably it would be better to move the push out of the loop and only do it once after all changes are commited.

Another option, in case you really always want a commit: Use the --allow-empty option when committing. I do not recommend this approach.

git commit --allow-empty -m "Update ${file}"

git - conditionally commit generated artifacts

1 Answers1