Documentation says Fabric Crashlytics SDK uses a ContentProvider to initialize itself. So it is too late call setCrashlyticsCollectionEnabled function in your Application Code.
Fabric's API key is no longer used by the new SDK. Instead, Crashlytics now uses your app’s google-services.json file to associate your app with your Firebase project and retain your historic crash data. If you have an io.fabric.ApiKey declared in your AndroidManifest.xml file, remove it.
If you want to disable automatic crash reporting and enable it only
for select users, use the Android meta-data tag in your
AndroidManifest.xml file. Then, you can enable crash reporting using
the new setCrashlyticsCollectionEnabled instance method.
https://firebase.google.com/docs/crashlytics/upgrade-sdk?platform=android#firebaseno_longer_works_with_the_fabric_sdk
So you need to disable data collection in your manifest. In order to do that you need to add firebase_crashlytics_collection_enabled meta-tag to your manifest.
<meta-data
android:name="firebase_crashlytics_collection_enabled"
android:value="false" />
https://github.com/firebase/firebase-android-sdk/blob/5440af41f0d15ff1358038dc31fcd1d4eac0a89c/firebase-crashlytics/src/main/java/com/google/firebase/crashlytics/internal/common/DataCollectionArbiter.java#L74
You might want to inject build variables to manifest file since you only want to disable Crashlytics in your test builds. See the link down below.
https://developer.android.com/studio/build/manifest-build-variables.html