1

I've written a pipeline on Azure that builds our apk, builds the espresso test apk, signs them both with the Keystore that's located in secure storage, then uploads them to BrowserStack automate & executes the test. The issue occurs at the Builds dashboard on Browserstack where it says :

There is signature mismatch between application APK and test-suite APK. Package com.example.appName.test does not have a signature matching the target package com.example.appName. Please fix this, upload your app & test-suite APK and execute the test again.

Here's my.yml file:

pool:
  vmImage: ubuntu-latest

steps:
# download .jks file from secure files for signing
- task: DownloadSecureFile@1
  name: keystore
  displayName: 'Download keystore file'
  inputs:
    secureFile: 'appName.jks'

- script: |
    echo Installing $(keystore.secureFilePath) to the trusted keystore directory...
    sudo chown root:root $(keystore.secureFilePath)
    sudo chmod a+r $(keystore.secureFilePath)
    sudo ln -s -t /etc/ssl/certs/ $(keystore.secureFilePath)
  displayName: "Installing keystore file"

# build apk
- task: Gradle@2
  inputs:
    workingDirectory: ''
    gradleWrapperFile: 'gradlew'
    gradleOptions: '-Xmx3072m'
    publishJUnitResults: false
    testResultsFiles: '**/TEST-*.xml'
    tasks: 'assembleGoogleDebug'
  displayName: "Building default APK"

# Build test apk
- task: Gradle@2
  inputs:
    workingDirectory: ''
    gradleWrapperFile: 'gradlew'
    gradleOptions: '-Xmx3072m'
    publishJUnitResults: false
    testResultsFiles: '**/TEST-*.xml'
    tasks: 'assembleGoogleDebugAndroidTest'
  displayName: "Building test APK"

# sign the apks with the same signature
- task: AndroidSigning@3
  inputs:
    apkFiles: '**/*.apk'
    apksignerKeystoreFile: 'appName.jks'
    apksignerKeystorePassword: '***'
    apksignerKeystoreAlias: 'appName'
    apksignerKeyPassword: '***'
    zipalign: false
  displayName: "Signing APKs"

- task: BrowserStackConfig@0
  inputs:
    BrowserStackServiceEndPoint: 'BrowserStack connection'
  displayName: "Establishing BrowserStack connection..."

# Upload apk to BrowserStack
- task: BrowserStackAppUploader@0
  inputs:
    appPath: '/home/vsts/work/1/s/presentation/build/outputs/apk/google/debug/presentation-google-debug.apk'
    appCustomId: 'sport24'
  displayName: 'Uploading default APK to BrowserStack'

- script: |
   curl -u "name:key" \
   -X POST "https://api-cloud.browserstack.com/app-automate/espresso/v2/test-suite" \
   -F "file=@ /home/vsts/work/1/s/presentation/build/outputs/apk/androidTest/google/debug/presentation-google-debug-androidTest.apk"
   -F "custom_id=appNameTest"
   echo "triggering test" 
   
   curl -u "name:key" \
   -X POST "https://api-cloud.browserstack.com/app-automate/espresso/v2/build" \
   -d '{"app": "url", "testSuite": "bs://url", "devices": ["Samsung Galaxy S9 Plus-9.0"]}' \
   -H "Content-Type: application/json" 
  displayName: 'uploading test suite and triggering test'

# post test results
- task: BrowserStackResults@0

I'm signing both APKs with the same Keystore & the test class is located under com.example.appName/androidTest/ so I can't figure out what's causing the mismatch. Any ideas?

Stelios Papamichail
  • 955
  • 2
  • 19
  • 57

2 Answers2

2

I faced the exact same issue when I uploaded my apks to BrowserStack from Jenkins. My pipeline had separate steps for building the debug.apk and also the androidTest.apk. I only signed my release.apk however, which is in a separate step.

I managed to solve this issue by building the debug.apk and androidTest.apk in a single step.

So in my Jenkinsfile I combined these 2 steps

stage("Build Debug APK") {
    withDockerContainer (image: image) {
    sh script: 'bash ./gradlew assembleDebug'
    archiveArtifacts allowEmptyArchive: true, artifacts: '**/*.apk', onlyIfSuccessful: false
    }
}
stage("UI Tests") {
    withDockerContainer (image: image) {
    sh script: 'bash ./gradlew :app:assembleAndroidTest'
    archiveArtifacts allowEmptyArchive: true, artifacts: '**/*.apk', onlyIfSuccessful: false

    // Upload apks to BrowserStack
}

into the following:

stage("UI Tests") {
   withDockerContainer (image: image) {
   sh script: 'bash ./gradlew assembleDebug :app:assembleAndroidTest'
   archiveArtifacts allowEmptyArchive: true, artifacts: '**/*.apk', onlyIfSuccessful: false

   // Upload apks to BrowserStack
}

So to solve your issue you could also try combining your assembleGoogleDebug and assembleGoogleDebugAndroidTest into a single step.

Blitzkid
  • 21
  • 1
  • 4
2

This error seems to occur for a variety of reasons.

Looking at your .yml file I can suggest the following 2 corrections:

  1. You are missing a backslash \ right before the -F parameter where you specify the custom id of your test suite, this is required otherwise the custom id for the test suite will not be set, So replace this line:
-F "file=@ /home/vsts/work/1/s/presentation/build/outputs/apk/androidTest/google/debug/presentation-google-debug-androidTest.apk"

with this line (just add the \ at the end):

-F "file=@ /home/vsts/work/1/s/presentation/build/outputs/apk/androidTest/google/debug/presentation-google-debug-androidTest.apk" \
  1. In your final curl command where you trigger the execution of the tests, you should replace the URLs with the custom ids you set for the app and the test suit.

So edit this line:

-d '{"app": "url", "testSuite": "bs://url", "devices": ["Samsung Galaxy S9 Plus-9.0"]}' \

To this:

-d '{"app": "INSERT_CUSTOM_APP_ID_HERE", "testSuite": "INSERT_CUSTOM_TEST_ID_HERE", "devices": ["Samsung Galaxy S9 Plus-9.0"]}' \

What the above change basically does is that you now specify the app and test suite for test execution using the ids you previously set in contrast to using the URL's that differ with each execution probably causing the error in the first place.

Thomas Papapaschos
  • 86
  • 4