gitlab runner throws "Cleaning up file based variables 00:01 ERROR: Job failed: exit code 1" at the end

Question

Even though all my steps pass successfully , Gitlab CI shows this - "Cleaning up file based variables 00:01 ERROR: Job failed: exit code 1"

and fails the job at the very end . Also interestingly , this only happens for my master branch . It runs successfully on other branches. Has anyone faced this issue and found a resolution ?

    - >
     for dir in $(git log -m -1 --name-only -r --pretty="format:" "$CI_COMMIT_SHA"); do 
     if [[ -f "$dir" ]]; then 
     SERVICE=$(echo "$dir")
     # helm install the service
     fi
     done
    - echo "deployed"

ctwheels · Accepted Answer · 2020-12-23T16:37:14.660

Overview

This drove me crazy and I'm still not sure what the appropriate answer is. I just ran into this issue myself and sunk hours into this issue. I think GitLab messed something up with command substitution (shows a new release yesterday), although I could be wrong about the issue or its timing. It also seems to only occur for some command substitutions and not others, I initially suspected it might be related to outputting to /dev/null, but wasn't going to dive too deep. It always failed immediately after the command substitution was initiated.

My code

I had code similar to yours (reduced version below), tried manipulating it multiple ways, but each use of command substitution yielded the same failure message:

Cleaning up file based variables          00:01
ERROR: Job failed: exit code 1

Attempts I've made include the following:

- folders=$(find .[^.]* * -type d -maxdepth 0 -exec echo {} \; 2>/dev/null)
- >
  while read folder; do
      echo "$folder"
  done <<< "$folders"

And ...

- >
  while read folder; do
      echo "$folder"
  done <<< $(find .[^.]* * -type d -maxdepth 0 -exec echo {} \; 2>/dev/null)

Both those versions succeeded on my local machine, but failed in GitLab (I might have typos in above - please don't scrutinize, it's reduced version of my actual program).

How I fixed it

Rather than using command substitution $(...), I instead opted for process substitution <(...) and it seems to be working without issue.

- >
  while read folder; do
      echo "$folder"
  done < <(find .[^.]* * -type d -maxdepth 0 -exec echo {} \; 2>/dev/null)

I would try to substitute the same in your code if possible:

- >
  while read dir; do
      # the rest goes here
  done < <(git log -m -1 --name-only -r --pretty="format:" "$CI_COMMIT_SHA")

The issue might also be the line inside the if statement (the echo), you can replace that with the following:

read SERVICE < <(echo "$dir")

Again, not exactly sure this will fix the issue for you as I'm still unsure what the cause is, but it resolved my issue. Best of luck.

Thank you for avoiding me from "sunk hours into this issue" :) — Nick, Dec 24 '20 at 15:40

score 3 · Answer 2 · answered Dec 31 '20 at 06:36

3

The error seemed to vanish for me once I removed the script from .gitlab-ci.yml file to another scipt.sh file and called the script.sh file in the gitlab yaml.

answered Dec 31 '20 at 06:36

ssbb191

1,486
3
12
23

Any idea on how to pass environment variables to the script.sh file? – Kshitij Bajracharya May 26 '21 at 03:15
you can define variables in the gitlab ci yaml and while calling the script, pass the variable. ex. /script-to-be-called.sh "$CI_COMMIT_TAG" – ssbb191 May 26 '21 at 05:13

score 2 · Answer 3 · answered Jan 20 '21 at 08:32

We have run into the same issue in GitLab v13.3.6-ee with the following line of the script that we are using for the open new merge request:

COUNTBRANCHES=`echo ${LISTMR} | grep -o "\"source_branch\":\"${CI_COMMIT_REF_NAME}\"" | wc -l`;

and as @ctwheels stated, changing that line into this:

read COUNTBRANCHES < <(echo ${LISTMR} | grep -o "\"source_branch\":\"${CI_COMMIT_REF_NAME}\"" | wc -l);

solved our problem.

score 2 · Answer 4 · edited May 20 '21 at 14:59

2

I had this error when tried to use protected CI/CD variable.

edited May 20 '21 at 14:59

Dharman

30,962
25
85
135

answered May 20 '21 at 14:53

Orest Lenczyk

31
2

1

which one protected var? – andilabs Aug 24 '21 at 22:22

score 2 · Answer 5 · answered Jul 22 '22 at 10:21

In my case I just had a conditionnal command, and if the last condition is false, then gitlab thinks the script has errored (even if it's not the case, because it uses the last line as the return value)

This is what my script looked like, and it would error if the project is using yarn but not npm

      [ -f yarn.lock ] && yarn install --frozen-lockfile --cache .npm && yarn prod
      [ ! -f yarn.lock ] && npm ci --prefer-offline --cache .npm && npm run prod --cache .npm

So the solution is just to make sure the last line returns true

      [ -f yarn.lock ] && yarn install --frozen-lockfile --cache .npm && yarn prod
      [ ! -f yarn.lock ] && npm ci --prefer-offline --cache .npm && npm run prod --cache .npm
      true

score 1 · Answer 6 · answered Jan 22 '21 at 00:25

1

in my case, my script ended with a curl command to an URL that would return a 403 Forbidden and probably hang up

curl -s "$ENV_URL/hello" | grep "hello world"

... if that helps anyone :-)

answered Jan 22 '21 at 00:25

Stefan

747
8
11

score 1 · Answer 7 · answered Nov 03 '21 at 03:40

It was a very specific use case for me (.NetCore), but it will eventually help someone.

In my case, no error was written in the logs and the tests where executed successfully but the job failed with the exist message showed in the question.

I was referencing xunit in my source project (not only in my test project) and I don't know why that causes the ci job to fail (but worked locally showing only a warning : Unable to find testhost.dll. Please publish your test project and retry).

Deleting xunit from my source project (not the test project) resolved the issue.

gitlab runner throws "Cleaning up file based variables 00:01 ERROR: Job failed: exit code 1" at the end

7 Answers7

Overview

My code

How I fixed it

Linked