CircleCI 2.0 Android Build always Failing

Question

I have the following in my CircleCI 2.0 config in the Android project code-base.

version: 2

jobs:
    build:

        environment:
            TERM: dumb
            JVM_OPTS: -Xmx4g -Xms2g -XX:MaxPermSize=2g

#     ###### Use these for other Docker Images, if need be ##########
#      ANDROID_SDK_ROOT: /usr/local/android-sdk-linux
#      SDK_MANAGER: $ANDROID_SDK_ROOT/tools/bin/sdkmanager

            GIT_COMMIT_DESC: git log --format=oneline -n 1 $CIRCLE_SHA1

            ANDROID_COMPILE_VERSION: 27
            ANDROID_TARGET_VERSION: 27
            ANDROID_MIN_VERSION: 21
            ANDROID_BUILD_VERSION: 28.0.2
            ANDROID_SUPPORT_VERSION: 27.1.1
            KOTLIN_VERSION: 1.2.70

        working_directory: ~/workSpace

        branches:
          only:
            - develop
            - release/<*>
            - feature/circle_ci_cd
          ignore:
            - bugfix/<*>
            - refactor/<*>

        docker:
          - image: circleci/android:api-$ANDROID_TARGET_VERSION-alpha

#    java:
#      version: oraclejdk8

#     ####### Other Docker Images, if need be ############
#    dependencies:
#      pre:
#        - $SDK_MANAGER --install "tools" && yes | $SDK_MANAGER --licenses
#        - $SDK_MANAGER --install "platforms;android-$ANDROID_TARGET_VERSION" && yes | $SDK_MANAGER --licenses
#        - $SDK_MANAGER --install "build-tools;$ANDROID_BUILD_VERSION" && yes | $SDK_MANAGER --licenses
#        - $SDK_MANAGER --install "platform-tools" && yes | $SDK_MANAGER --licenses
#        - $SDK_MANAGER --install "extras;android;m2repository" && yes | $SDK_MANAGER --licenses
#        - $SDK_MANAGER --install "extras;google;m2repository" && yes | $SDK_MANAGER --licenses
#        - $SDK_MANAGER --install "extras;google;google_play_services" && yes | $SDK_MANAGER --licenses

        steps:
          - checkout

          - run:
              name: "Pull Submodules"
              command: |
                git submodule init
                git submodule sync
                git submodule update --remote

          - run:
              name: "Android SDK Properties"
              command: |
                sed -i "s/compileSdkVersion=*.*/compileSdkVersion=$ANDROID_COMPILE_VERSION/" gradle.properties
                sed -i "s/targetSdkVersion=*.*/targetSdkVersion=$ANDROID_TARGET_VERSION/"  gradle.properties
                sed -i "s/minSdkVersion=*.*/minSdkVersion=$ANDROID_MIN_VERSION/" gradle.properties
                sed -i "s/buildToolsVersion=*.*/buildToolsVersion=$ANDROID_BUILD_VERSION/" gradle.properties
                sed -i "s/supportVersion=*.*/supportVersion=$ANDROID_SUPPORT_VERSION/" gradle.properties
                sed -i "s/kotlinVersion=*.*/kotlinVersion=$KOTLIN_VERSION/" gradle.properties
                sed -i "s/versionCode=*.*/versionCode=${CIRCLE_BUILD_NUM:-1}/" gradle.properties

#      ######## Other Docker Images, if need be ###########
#      - run:
#          name: "Update Android"
#          command: $SDK_MANAGER --update && yes | $SDK_MANAGER --licenses

          - run:
              name: "Clean local.properties"
              command: rm -rf local.properties || true

          - run:
              name: Chmod permissions #if permission for Gradlew Dependencies fail, use this.
              command: chmod +x ./gradlew

          - restore_cache:
              key: jars-{{ checksum "build.gradle" }}-{{ checksum  "app/build.gradle" }}

          - run:
              name: Assemble
              command: ./gradlew clean assemble

          - save_cache:
              paths:
                - ${CIRCLE_WORKING_DIRECTORY}/.gradle
                - ${ANDROID_SDK_ROOT}
              key: jars-{{ checksum "build.gradle" }}-{{ checksum  "app/build.gradle" }}
#      - store_artifacts:
#          path: app/build/reports
#          destination: reports
#      - store_test_results:
#          path: app/build/test-results
# See https://circleci.com/docs/2.0/deployment-integrations/ for deploy examples

Few other properties in the gradle.properties file

org.gradle.jvmargs=-Xms2g -Xmx4g -XX:MaxPermSize=2g -XX:+HeapDumpOnOutOfMemoryError -XX:ReservedCodeCacheSize=2g -Dfile.encoding=UTF-8
org.gradle.configureondemand=false
org.gradle.caching=true
org.gradle.daemon=false
org.gradle.parallel=false
kotlin.incremental=false
kotlin.compiler.execution.strategy=in-process
android.enableBuildCache=true
android.enableR8=true

Gradle wrapper version is as follows.

https\://services.gradle.org/distributions/gradle-4.10.1-all.zip

Gradle Android Plugin.

'com.android.tools.build:gradle:3.3.0-alpha10'

Following is build-variants configuration.

flavorDimensions "product", "store"

productFlavors {
    free { dimension "product" }
    paid { dimension "product" }
    google { dimension "store" }
    amazon { dimension "store" }
}

Development machine is MacOS High Sierra: 10.13.6, 3.5 GHz Intel Core i5 CPU, 32 GB 2400 MHz DDR4 RAM.

Here's what's happenning.

The order of build-variants is chronological, as per the given flavor-dimensions, therefore following eight are created by gradle in the exact order.

freeAmazonDebug
freeAmazonRelease
freeGoogleDebug
freeGoogleRelease
paidAmazonDebug
paidAmazonRelease
paidGoogleDebug
paidGoogleRelease

Each of those build-variants will have a task-dependency list. Based on the task-dependency, this is what's happenning.

kaptFreeAmazonDebugKotlin executes without error.
kaptFreeAmazonReleaseKotlin fails with the following stack-trace but does not abruptly kill the build.

Compilation with Kotlin compile daemon was not successful java.rmi.UnmarshalException: Error unmarshaling return header; nested exception is: java.io.EOFException at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:236) at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161) at java.rmi.server.RemoteObjectInvocationHandler.invokeRemoteMethod(RemoteObjectInvocationHandler.java:227) at java.rmi.server.RemoteObjectInvocationHandler.invoke(RemoteObjectInvocationHandler.java:179) at com.sun.proxy.$Proxy106.compile(Unknown Source) at org.jetbrains.kotlin.compilerRunner.GradleCompilerRunner.nonIncrementalCompilationWithDaemon(GradleKotlinCompilerRunner.kt:256) at org.jetbrains.kotlin.compilerRunner.GradleCompilerRunner.compileWithDaemon(GradleKotlinCompilerRunner.kt:219) at org.jetbrains.kotlin.compilerRunner.GradleCompilerRunner.compileWithDaemonOrFallback(GradleKotlinCompilerRunner.kt:166) at org.jetbrains.kotlin.compilerRunner.GradleCompilerRunner.compileWithDaemonOrFallback(GradleKotlinCompilerRunner.kt:63) at org.jetbrains.kotlin.compilerRunner.KotlinCompilerRunner.runCompiler(KotlinCompilerRunner.kt:133) at org.jetbrains.kotlin.compilerRunner.GradleCompilerRunner.runJvmCompiler(GradleKotlinCompilerRunner.kt:119) at org.jetbrains.kotlin.gradle.internal.KaptWithKotlincTask.compile(KaptWithKotlincTask.kt:71) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:73) at org.gradle.api.internal.project.taskfactory.StandardTaskAction.doExecute(StandardTaskAction.java:46) at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:39) at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:26) at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:801) at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:768) at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$1.run(ExecuteActionsTaskExecuter.java:131) at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:300) at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:292) at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:174) at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:90) at org.gradle.internal.operations.DelegatingBuildOperationExecutor.run(DelegatingBuildOperationExecutor.java:31) at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeAction(ExecuteActionsTaskExecuter.java:120) at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:99) at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:77) at org.gradle.api.internal.tasks.execution.OutputDirectoryCreatingTaskExecuter.execute(OutputDirectoryCreatingTaskExecuter.java:51) at org.gradle.api.internal.tasks.execution.SkipCachedTaskExecuter.execute(SkipCachedTaskExecuter.java:105) at org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter.execute(SkipUpToDateTaskExecuter.java:59) at org.gradle.api.internal.tasks.execution.ResolveTaskOutputCachingStateExecuter.execute(ResolveTaskOutputCachingStateExecuter.java:54) at org.gradle.api.internal.tasks.execution.ResolveBuildCacheKeyExecuter.execute(ResolveBuildCacheKeyExecuter.java:79) at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:59) at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:101) at org.gradle.api.internal.tasks.execution.FinalizeInputFilePropertiesTaskExecuter.execute(FinalizeInputFilePropertiesTaskExecuter.java:44) at org.gradle.api.internal.tasks.execution.CleanupStaleOutputsExecuter.execute(CleanupStaleOutputsExecuter.java:91) at org.gradle.api.internal.tasks.execution.ResolveTaskArtifactStateTaskExecuter.execute(ResolveTaskArtifactStateTaskExecuter.java:62) at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:59) at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:54) at org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter.execute(ExecuteAtMostOnceTaskExecuter.java:43) at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:34) at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.run(EventFiringTaskExecuter.java:51) at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:300) at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:292) at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:174) at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:90) at org.gradle.internal.operations.DelegatingBuildOperationExecutor.run(DelegatingBuildOperationExecutor.java:31) at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.execute(EventFiringTaskExecuter.java:46) at org.gradle.execution.taskgraph.LocalTaskInfoExecutor.execute(LocalTaskInfoExecutor.java:42) at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareWorkItemExecutor.execute(DefaultTaskExecutionGraph.java:277) at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareWorkItemExecutor.execute(DefaultTaskExecutionGraph.java:262) at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$ExecutorWorker$1.execute(DefaultTaskPlanExecutor.java:135) at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$ExecutorWorker$1.execute(DefaultTaskPlanExecutor.java:130) at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$ExecutorWorker.execute(DefaultTaskPlanExecutor.java:200) at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$ExecutorWorker.executeWithWork(DefaultTaskPlanExecutor.java:191) at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$ExecutorWorker.run(DefaultTaskPlanExecutor.java:130) at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:63) at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:46) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:55) at java.lang.Thread.run(Thread.java:748) Caused by: java.io.EOFException at java.io.DataInputStream.readByte(DataInputStream.java:267) at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:222) ... 65 more Unable to clear jar cache after compilation, maybe daemon is already down: java.rmi.ConnectException: Connection refused to host: 127.0.0.1; nested exception is: java.net.ConnectException: Connection refused (Connection refused) Could not connect to kotlin daemon. Using fallback strategy.

kaptPaidAmazonDebugKotlin fails with the above exact same stacktrace, and also abruptly kills the gradle build.

I have tried every possible combination of the gradle options and java options and docker options that I could think of, and I could try and find on the Internet, but nothing to the rescue.

These failures happen only in CircleCI 2.0 on the cloud or with CircleCI CLI, with the given Docker-image. The gradle builds locally are stable.

Any pointers to a solution will be appreciated.

That's the question, why would kapt fail to connect to it's daemon on host 127.0.0.1 in a docker-image on the circleci cloud? — AndroidRocks, Sep 18 '18 at 13:44
any moment on this? We are seeing this now on gitlab but we use a circle ci image — satyajit, Oct 18 '19 at 23:00

Martin Zeitler · Answer 1 · 2018-09-19T19:51:32.560

1

when reviewing, there some to be several possible causes:

you are using the wrong buildToolsVersion, which can even be omitted in the build.gradle - in order to use the latest version matching API level 27.
using com.android.tools.build:gradle:3.3.0-alpha10 is questionable. better use stable version 3,1,4; even version 3.2.0 is currently still in beta.

Unable to clear jar cache after compilation

^ this hints pretty much for a file-system conflict (because the next one task already accesses it).

check build directory - and redefine the productFlavors accordingly, to work around the problem.

edited Sep 19 '18 at 19:51

answered Sep 19 '18 at 19:16

Martin Zeitler

1
19
155
216

1) Removed buildToolsVersion completely. 2) Moved the Android-Gradle plugin version to the stable 3.1.4 as suggested. 3) Increased JVM -Xmx space to 32g for both Docker and Gradle. 4) Turned off all daemons, all parallel processing, as much streamlined as possible. And yet, CircleCI 2.0 with Docker for Android Build process continues to fail with the same reasons as above, while simple command-line './gradlew clean assemble' always generates the required Builds successfully. It most definitely is not the Gradle build setup and configurations, but Docker itself. – AndroidRocks Sep 20 '18 at 13:37
@AndroidRocks ever tried `kotlin.incremental=false`? this is what the error message might hint for - and also your description of the behavior. see https://youtrack.jetbrains.com/issue/KT-15562 – Martin Zeitler Sep 20 '18 at 13:47
Please see contents of gradle.properties listed above. – AndroidRocks Sep 20 '18 at 14:00
@AndroidRocks well, there it reads `org.gradle.caching=true` and the crash reads `unable to clear jar cache after compilation` ?? – Martin Zeitler Sep 20 '18 at 14:06
Yes. I don't know what's going on in that Docker Image. I do know, './gradlew clean assemble' executes perfectly in the Terminal. – AndroidRocks Sep 20 '18 at 14:47
@AndroidRocks did you try `org.gradle.caching=false` yet? because with caching disabled, there barely can be a conflict within the cache's file-system... which could also hint for a permission issue (this would have to be the same group as the user, who runs `gradlew` there). – Martin Zeitler Sep 20 '18 at 14:48
Yes, I have tried that as well. I have tried every combination of values in that list of gradle.properties that I had presented above. Nothing appears to work as desired in the Docker Image, but './gradlew clean assemble' just works fine in the Terminal. – AndroidRocks Sep 20 '18 at 18:44

score 0 · Accepted Answer · answered Nov 30 '18 at 15:16

0

android {
    kapt {
        useBuildCache true
    }

    dexOptions {
        preDexLibraries true
        jumboMode false
    }
}

answered Nov 30 '18 at 15:16

AndroidRocks

292
4
16

2

Please add some kind of explanation as to what these options do. As a general rule of thumb it is never good to just post code without any kind of human readable explanation. Thanks – Joshua King May 11 '19 at 00:16

CircleCI 2.0 Android Build always Failing

2 Answers2