Xcode test failing does not failed Github action pipeline

Question

I have a Github Action pipeline:

name: default

on: [push]

jobs:
  build:

    runs-on: macOS-latest

    steps:
    - uses: actions/checkout@v1
    - name: CocoaPod Install
      run: pod install
    - name: Force xcode 11
      run: sudo xcode-select -switch /Applications/Xcode_11.1.app
    - name: Test
      run: ./pipelines.sh test
    - name: Save report
      uses: actions/upload-artifact@v1
      with:
        name: test_report
        path: ./Test

And a script Shell:

function test
{
    xcodebuild \
       -workspace MyApp.xcworkspace \
       -scheme DEBUG\ -\ MyApp \
       -destination 'platform=iOS Simulator,name=iPhone 11' \
       test
}

My problem is when i run my pipeline with failing test, the pipeline is marked as PASSED, which is a problem...

I have also check with fastlane, failing test does not fail the pipeline.

How can I make my pipeline as FAIL when test does not pass?

Cf screenshot for fastlane:

peterevans · Accepted Answer · 2019-11-05T01:23:04.150

3

You need to return a non-zero value to fail the step. Try adding this to the xcodebuild command.

xcodebuild -... || exit 1

There are some other solutions besides this at the following question. How to get the return value of xcodebuild?

Update: Based on your comments that you would like steps afterwards to complete you can do the following.

Change your script to set an output containing the result of the xcodebuild command.

    xcodebuild \
       -workspace MyApp.xcworkspace \
       -scheme DEBUG\ -\ MyApp \
       -destination 'platform=iOS Simulator,name=iPhone 11' \
       test
    echo "::set-output name=result::$?"

Add an id to the step where this script is executed.

    - name: Test
      id: xcodebuild
      run: ./pipelines.sh test

At the end of your workflow you can check if the tests did not pass and fail the workflow.

      - name: Check Tests Passed
        if: steps.xcodebuild.outputs.result != 0
        run: exit 1

edited Nov 05 '19 at 01:23

answered Nov 04 '19 at 13:57

peterevans

34,297
7
84
83

When i make it ```xcodebuild -... || exit 1```, commands after this line are not executed if i have some – Kevin ABRIOUX Nov 04 '19 at 14:14
Step `Save report` is not executed in the pipeline – Kevin ABRIOUX Nov 04 '19 at 14:16
@KevinABRIOUX Updated the answer with a solution to allow further steps to execute and fail afterwards. – peterevans Nov 05 '19 at 01:24

Xcode test failing does not failed Github action pipeline

1 Answers1