Why same command fails in GitLab CI?

Question

The following command works perfectly on the terminal but the same command fails in GitLab CI.

echo Hello >> foo.txt; cat foo.txt | grep "test"; [[ $? -eq 0 ]] && echo fail || echo success

return is success but the same command in GitLab CI

$ echo Hello >> foo.txt; cat foo.txt | grep "test"; [[ $? -eq 0 ]] && echo fail || echo success
Cleaning up file based variables
ERROR: Job failed: command terminated with exit code 1

is simply failing. I have no idea why. echo $SHELL return /bin/bash in both.

Answer 1

Source of the issue

The behavior you observe is pretty standard given the "implied" set -e in a CI context.

To be more precise, your code consists in three compound commands:

echo Hello >> foo.txt
cat foo.txt | grep "test"
[[ $? -eq 0 ]] && echo fail || echo success 

And the grep "test" command returns a non-zero exit code (namely, 1). As a result, the script immediately exits and the last line is not executed.

Note that this feature is typical in a CI context, because if some intermediate command fails in a complex script, we'd typically want to get a failure, and avoid running the next commands (which could potentially be "off-topic" given the error).

You can reproduce this locally as well, by writing for example:

bash -e -c "
echo Hello >> foo.txt
cat foo.txt | grep "test"
[[ $? -eq 0 ]] && echo fail || echo success 
"

which is mostly equivalent to:

bash -c "
set -e
echo Hello >> foo.txt
cat foo.txt | grep "test"
[[ $? -eq 0 ]] && echo fail || echo success 
"

Relevant manual page

For more insight:

• on set -e, see man 1 set
• on bash -e, see man 1 bash

How to fix the issue?

You should just adopt another phrasing, avoiding [[ $? -eq 0 ]] a-posteriori tests. So the commands that may return a non-zero exit code without meaning failure should be "protected" by some if:

echo Hello >> foo.txt
if cat foo.txt | grep "test"; then
  echo fail
  false  # if ever you want to "trigger a failure manually" at some point.
else
  echo success
fi

Also, note that grep "test" foo.txt would be more idiomatic than cat foo.txt | grep "test" − which is precisely an instance of UUOC (useless use of cat).

Answer 2

I have no idea why.

Gitlab executes each command one at a time and checks the exit status of each command. When the exit status is not zero, the job is failed.

There is no test string inside foo.txt, so the command cat foo.txt | grep "test" exits with nonzero exit status. Thus the job is failed.