Android Gradle 3.0: Multiple dex files define Lorg/jetbrains/annotations/TestOnly

Question

Seems like a run of a mill unable to merge dex problem, however, I can't find where the duplicated dependency comes from. The app:dependencies printout is pasted below.

As you can see, org.jetbrains:annotations:13.0 exists only once in here. Am I reading it wrong? What other commands I can use to hunt the duplicate dependency?

- com.google.firebase:firebase-core:11.8.0
|    \--- com.google.firebase:firebase-analytics:11.8.0
|         +--- com.google.android.gms:play-services-basement:11.8.0
|         |    +--- com.android.support:support-v4:25.2.0 -> 27.0.2
|         |    |    +--- com.android.support:support-compat:27.0.2
|         |    |    |    +--- com.android.support:support-annotations:27.0.2
|         |    |    |    \--- android.arch.lifecycle:runtime:1.0.3
|         |    |    |         +--- android.arch.lifecycle:common:1.0.3
|         |    |    |         \--- android.arch.core:common:1.0.0
|         |    |    +--- com.android.support:support-media-compat:27.0.2
|         |    |    |    +--- com.android.support:support-annotations:27.0.2
|         |    |    |    \--- com.android.support:support-compat:27.0.2 (*)
|         |    |    +--- com.android.support:support-core-utils:27.0.2
|         |    |    |    +--- com.android.support:support-annotations:27.0.2
|         |    |    |    \--- com.android.support:support-compat:27.0.2 (*)
|         |    |    +--- com.android.support:support-core-ui:27.0.2
|         |    |    |    +--- com.android.support:support-annotations:27.0.2
|         |    |    |    \--- com.android.support:support-compat:27.0.2 (*)
|         |    |    \--- com.android.support:support-fragment:27.0.2
|         |    |         +--- com.android.support:support-compat:27.0.2 (*)
|         |    |         +--- com.android.support:support-core-ui:27.0.2 (*)
|         |    |         +--- com.android.support:support-core-utils:27.0.2 (*)
|         |    |         \--- com.android.support:support-annotations:27.0.2
|         |    \--- com.google.android.gms:play-services-basement-license:11.8.0
|         +--- com.google.firebase:firebase-common:11.8.0
|         |    +--- com.google.android.gms:play-services-basement:11.8.0 (*)
|         |    +--- com.google.android.gms:play-services-tasks:11.8.0
|         |    |    +--- com.google.android.gms:play-services-basement:11.8.0 (*)
|         |    |    \--- com.google.android.gms:play-services-tasks-license:11.8.0
|         |    \--- com.google.firebase:firebase-common-license:11.8.0
|         +--- com.google.firebase:firebase-analytics-impl:11.8.0
|         |    +--- com.google.android.gms:play-services-basement:11.8.0 (*)
|         |    +--- com.google.firebase:firebase-iid:11.8.0
|         |    |    +--- com.google.android.gms:play-services-basement:11.8.0 (*)
|         |    |    +--- com.google.firebase:firebase-common:11.8.0 (*)
|         |    |    +--- com.google.android.gms:play-services-tasks:11.8.0 (*)
|         |    |    \--- com.google.firebase:firebase-iid-license:11.8.0
|         |    +--- com.google.firebase:firebase-common:11.8.0 (*)
|         |    +--- com.google.android.gms:play-services-tasks:11.8.0 (*)
|         |    \--- com.google.firebase:firebase-analytics-impl-license:11.8.0
|         \--- com.google.firebase:firebase-analytics-license:11.8.0
+--- com.android.support:appcompat-v7:27.0.2
|    +--- com.android.support:support-annotations:27.0.2
|    +--- com.android.support:support-core-utils:27.0.2 (*)
|    +--- com.android.support:support-fragment:27.0.2 (*)
|    +--- com.android.support:support-vector-drawable:27.0.2
|    |    +--- com.android.support:support-annotations:27.0.2
|    |    \--- com.android.support:support-compat:27.0.2 (*)
|    \--- com.android.support:animated-vector-drawable:27.0.2
|         +--- com.android.support:support-vector-drawable:27.0.2 (*)
|         \--- com.android.support:support-core-ui:27.0.2 (*)
+--- com.android.support:support-v4:27.0.2 (*)
+--- com.android.support:design:27.0.2
|    +--- com.android.support:support-v4:27.0.2 (*)
|    +--- com.android.support:appcompat-v7:27.0.2 (*)
|    +--- com.android.support:recyclerview-v7:27.0.2
|    |    +--- com.android.support:support-annotations:27.0.2
|    |    +--- com.android.support:support-compat:27.0.2 (*)
|    |    \--- com.android.support:support-core-ui:27.0.2 (*)
|    \--- com.android.support:transition:27.0.2
|         +--- com.android.support:support-annotations:27.0.2
|         \--- com.android.support:support-compat:27.0.2 (*)
+--- org.jetbrains.kotlin:kotlin-stdlib:1.2.21
|    \--- org.jetbrains:annotations:13.0
+--- com.github.salomonbrys.kodein:kodein:4.1.0
|    +--- org.jetbrains.kotlin:kotlin-stdlib:1.1.3-2 -> 1.2.21 (*)
|    \--- com.github.salomonbrys.kodein:kodein-core:4.1.0
|         \--- org.jetbrains.kotlin:kotlin-stdlib:1.1.3-2 -> 1.2.21 (*)
+--- nl.komponents.kovenant:kovenant-core:3.3.0
+--- nl.komponents.kovenant:kovenant-android:3.3.0
|    +--- nl.komponents.kovenant:kovenant-core:3.3.0
|    \--- nl.komponents.kovenant:kovenant-ui:3.3.0
|         \--- nl.komponents.kovenant:kovenant-core:3.3.0
+--- com.github.ozodrukh:CircularReveal:2.0.1
+--- com.google.firebase:firebase-crash:11.8.0
|    +--- com.google.android.gms:play-services-basement:11.8.0 (*)
|    +--- com.google.firebase:firebase-iid:11.8.0 (*)
|    +--- com.google.firebase:firebase-common:11.8.0 (*)
|    +--- com.google.firebase:firebase-analytics:11.8.0 (*)
|    \--- com.google.firebase:firebase-crash-license:11.8.0
+--- com.squareup.retrofit2:retrofit:2.3.0
|    \--- com.squareup.okhttp3:okhttp:3.8.0
|         \--- com.squareup.okio:okio:1.13.0
+--- com.squareup.retrofit2:converter-gson:2.3.0
|    +--- com.squareup.retrofit2:retrofit:2.3.0 (*)
|    \--- com.google.code.gson:gson:2.7 -> 2.8.0
+--- com.github.salomonbrys.kotson:kotson:2.5.0
|    +--- com.google.code.gson:gson:2.8.0
|    \--- org.jetbrains.kotlin:kotlin-stdlib:1.0.6 -> 1.2.21 (*)
+--- org.funktionale:funktionale-all:1.1
|    \--- org.jetbrains.kotlin:kotlin-stdlib:1.1.2-2 -> 1.2.21 (*)
+--- io.reactivex.rxjava2:rxkotlin:2.1.0
|    +--- io.reactivex.rxjava2:rxjava:2.1.0
|    |    \--- org.reactivestreams:reactive-streams:1.0.0
|    \--- org.jetbrains.kotlin:kotlin-stdlib:1.1.3 -> 1.2.21 (*)
+--- com.github.kittinunf.fuel:fuel-android:1.10.0
|    +--- org.jetbrains.kotlin:kotlin-stdlib:1.1.4-3 -> 1.2.21 (*)
|    \--- com.github.kittinunf.fuel:fuel:1.10.0
|         +--- org.jetbrains.kotlin:kotlin-stdlib:1.1.4-3 -> 1.2.21 (*)
|         \--- com.github.kittinunf.result:result:1.2.0
|              \--- org.jetbrains.kotlin:kotlin-stdlib:1.1.4-3 -> 1.2.21 (*)
+--- com.beust:klaxon:0.30
+--- com.android.support:cardview-v7:27.0.2
|    \--- com.android.support:support-annotations:27.0.2
+--- com.android.support:recyclerview-v7:27.0.2 (*)
+--- com.nshmura:snappysmoothscroller:1.0.0
|    \--- com.android.support:recyclerview-v7:24.0.0 -> 27.0.2 (*)

Dependencies defined in build.gradle:

implementation 'com.android.support:appcompat-v7:27.0.2'
implementation 'com.android.support:support-v4:27.0.2'
implementation 'com.android.support:design:27.0.2'

implementation ("org.jetbrains.kotlin:kotlin-stdlib:$kotlin_version") {
    exclude group: 'org.jetbrains', module: 'annotations'
}
implementation 'com.github.salomonbrys.kodein:kodein:4.1.0'
implementation 'nl.komponents.kovenant:kovenant-core:3.3.0'
implementation 'nl.komponents.kovenant:kovenant-android:3.3.0'

implementation ('com.github.ozodrukh:CircularReveal:2.0.1@aar')

implementation 'com.google.firebase:firebase-core:11.8.0'
implementation 'com.google.firebase:firebase-crash:11.8.0'

implementation "com.squareup.retrofit2:retrofit:$retrofit_version"
implementation "com.squareup.retrofit2:converter-gson:$retrofit_version"
implementation'com.github.salomonbrys.kotson:kotson:2.5.0'
implementation "org.funktionale:funktionale-all:$functionale_version"
implementation "io.reactivex.rxjava2:rxkotlin:$rxkotlin_version"

implementation 'com.github.kittinunf.fuel:fuel-android:1.10.0'
implementation 'com.beust:klaxon:0.30'

implementation ('com.android.support:cardview-v7:27.0.2')
implementation ('com.android.support:recyclerview-v7:27.0.2', {
    exclude module: 'support-compat'
    exclude module: 'support-core-ui'
    exclude module: 'support-annotations'
})
implementation 'com.nshmura:snappysmoothscroller:1.0.0'

EDIT: Build command output

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':app:transformDexArchiveWithExternalLibsDexMergerForDebug'.
> com.android.builder.dexing.DexArchiveMergerException: Unable to merge dex

* Try:
Run with --info or --debug option to get more log output.

* Exception is:
org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':app:transformDexArchiveWithExternalLibsDexMergerForDebug'.
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:100)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:70)
        at org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter.execute(SkipUpToDateTaskExecuter.java:63)
        at org.gradle.api.internal.tasks.execution.ResolveTaskOutputCachingStateExecuter.execute(ResolveTaskOutputCachingStateExecuter.java:54)
        at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:58)
        at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:88)
        at org.gradle.api.internal.tasks.execution.ResolveTaskArtifactStateTaskExecuter.execute(ResolveTaskArtifactStateTaskExecuter.java:52)
        at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:52)
        at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:54)
        at org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter.execute(ExecuteAtMostOnceTaskExecuter.java:43)
        at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:34)
        at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter$EventFiringTaskWorker$1.run(DefaultTaskGraphExecuter.java:248)
        at org.gradle.internal.progress.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:336)
        at org.gradle.internal.progress.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:328)
        at org.gradle.internal.progress.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:197)
        at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:107)
        at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter$EventFiringTaskWorker.execute(DefaultTaskGraphExecuter.java:241)
        at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter$EventFiringTaskWorker.execute(DefaultTaskGraphExecuter.java:230)
        at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$TaskExecutorWorker.processTask(DefaultTaskPlanExecutor.java:124)
        at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$TaskExecutorWorker.access$200(DefaultTaskPlanExecutor.java:80)
        at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$TaskExecutorWorker$1.execute(DefaultTaskPlanExecutor.java:105)
        at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$TaskExecutorWorker$1.execute(DefaultTaskPlanExecutor.java:99)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionPlan.execute(DefaultTaskExecutionPlan.java:625)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionPlan.executeWithTask(DefaultTaskExecutionPlan.java:580)
        at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$TaskExecutorWorker.run(DefaultTaskPlanExecutor.java:99)
        at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:63)
        at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:46)
        at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:55)
Caused by: java.lang.RuntimeException: com.android.builder.dexing.DexArchiveMergerException: Unable to merge dex
        at com.android.builder.dexing.DxDexArchiveMerger.mergeMultidex(DxDexArchiveMerger.java:266)
        at com.android.builder.dexing.DxDexArchiveMerger.mergeDexArchives(DxDexArchiveMerger.java:133)
        at com.android.build.gradle.internal.transforms.DexMergerTransformCallable.call(DexMergerTransformCallable.java:97)
        at com.android.build.gradle.internal.transforms.ExternalLibsMergerTransform.transform(ExternalLibsMergerTransform.kt:121)
        at com.android.build.gradle.internal.pipeline.TransformTask$2.call(TransformTask.java:222)
        at com.android.build.gradle.internal.pipeline.TransformTask$2.call(TransformTask.java:218)
        at com.android.builder.profile.ThreadRecorder.record(ThreadRecorder.java:102)
        at com.android.build.gradle.internal.pipeline.TransformTask.transform(TransformTask.java:213)
        at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:73)
        at org.gradle.api.internal.project.taskfactory.DefaultTaskClassInfoStore$IncrementalTaskAction.doExecute(DefaultTaskClassInfoStore.java:173)
        at org.gradle.api.internal.project.taskfactory.DefaultTaskClassInfoStore$StandardTaskAction.execute(DefaultTaskClassInfoStore.java:134)
        at org.gradle.api.internal.project.taskfactory.DefaultTaskClassInfoStore$StandardTaskAction.execute(DefaultTaskClassInfoStore.java:121)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$1.run(ExecuteActionsTaskExecuter.java:122)
        at org.gradle.internal.progress.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:336)
        at org.gradle.internal.progress.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:328)
        at org.gradle.internal.progress.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:197)
        at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:107)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeAction(ExecuteActionsTaskExecuter.java:111)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:92)
        ... 27 more
Caused by: com.android.builder.dexing.DexArchiveMergerException: Unable to merge dex
        at com.android.builder.dexing.DexArchiveMergerCallable.call(DexArchiveMergerCallable.java:72)
        at com.android.builder.dexing.DexArchiveMergerCallable.call(DexArchiveMergerCallable.java:36)
        ... 46 more
Caused by: com.android.dex.DexException: Multiple dex files define Lorg/jetbrains/annotations/TestOnly;
        at com.android.dx.merge.DexMerger.readSortableTypes(DexMerger.java:661)
        at com.android.dx.merge.DexMerger.getSortedTypes(DexMerger.java:616)
        at com.android.dx.merge.DexMerger.mergeClassDefs(DexMerger.java:598)
        at com.android.dx.merge.DexMerger.mergeDexes(DexMerger.java:171)
        at com.android.dx.merge.DexMerger.merge(DexMerger.java:198)
        at com.android.builder.dexing.DexArchiveMergerCallable.call(DexArchiveMergerCallable.java:61)
        ... 47 more

I tried all the basic stuff I could find like:

clean build
remove .gradle dir

do you have anything in your logcat ? or just "unabe to merge dex" ?, also try clean project - rebuild project and compile , it had to bring a new error with a path or something — Gastón Saillén, Jan 29 '18 at 15:47
@IDroid yes I did, I just updated my question to add the log output. — kar, Jan 29 '18 at 16:02

zsmb13 · Accepted Answer · 2018-01-30T14:40:18.750

Turns out it's Funktionale being a bit of a bad citizen that's causing the problem. Using class lookup (Ctrl + N on Windows) for the given class, I got two results. One of them inside the expected JetBrains annotation package, the other inside Funktionale's.

Looking at the contents of this package, you can see that it actually contains a whole bunch of externally defined annotations inside it:

I can't say why they did this instead of having these libraries declared as dependencies though.

Now, for a list of possible solutions.

Open up the .jar file provided by the dependency, remove the offending packages from it, repack it, and include it in your project as a local .jar file.
Look into hacky solutions that try to remove .class files from a Gradle dependency. I found this while looking into this, but it didn't seem to work for me.
Raise an issue with the library authors on GitHub and wait for a response and a new release.
Fork and re-publish the library with the offending packages removed.
Use Arrow instead. This is a new functional Kotlin library created by the maintainers of Funktionale and Kategory together. They talk about it on the a Talking Kotlin podcast episode here. Unfortunately, I don't have personal experience with either of these libraries, but it might suit your needs.

Funny, I just managed to come to the same conclusion the moment you posted your answer :). Reported the issue [here](https://github.com/MarioAriasC/funKTionale/issues/42). Your answer is very informative, I'm impressed. Thank you! — kar, Jan 29 '18 at 19:13
I had some time to spare, this one took a bit of research for sure ;) Upvoted the GitHub issue. — zsmb13, Jan 29 '18 at 19:16

Android Gradle 3.0: Multiple dex files define Lorg/jetbrains/annotations/TestOnly

1 Answers1