Android Studio - Activity gets destroyed when minimizing / opening new intent, but only in variants

Question

I have a project setup with three flavors/variants. It originally was based on just one flavor, and later the latter two were added extra. The weird thing is that in the flavor (based on the original app) I can minimize the app, and reopen it without any issues. But when opening any intent or minimizing the app with the other flavours, the last activity is always destroyed from the stack.

At the moment I minimize, the onDestroy() of the activity gets called straight away.

This is how my manifest is built up:

<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
    package="xxxxxxxxxxx">

    <uses-permission android:name="android.permission.INTERNET" />
    <uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
    <uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
    <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />

    <uses-feature android:name="android.hardware.camera" />

    <application
        android:allowBackup="true"
        android:icon="@mipmap/ic_launcher"
        android:label="@string/app_label"
        android:roundIcon="@mipmap/ic_launcher_round"
        android:supportsRtl="true"
        android:theme="@style/Theme.AppCompat.NoActionBar"
        android:usesCleartextTraffic="true">
        <activity
            android:name=".SubMenuActivity"
            android:exported="false"
            android:label="Submenu"
            android:parentActivityName=".WebActivity"
            android:theme="@style/Theme.AppCompat.DayNight.DarkActionBar"
            />
        <activity
            android:name=".login.LoginActivity"
            android:alwaysRetainTaskState="true"
            android:configChanges="orientation"
            android:label="@string/title_activity_login"
            android:launchMode="singleTop"
            android:noHistory="false"
            android:screenOrientation="portrait"
            android:exported="true">
            <intent-filter android:autoVerify="true">
                <action android:name="android.intent.action.VIEW" />

                <category android:name="android.intent.category.DEFAULT" />
                <category android:name="android.intent.category.BROWSABLE" />

                <data
                    android:host="xxxxx"
                    android:pathPattern="xxxx"
                    android:scheme="https" />
            </intent-filter>
        </activity>
        <activity
            android:name=".MainActivity"
            android:alwaysRetainTaskState="true"
            android:configChanges="orientation"
            android:launchMode="singleTop"
            android:noHistory="false"
            android:screenOrientation="portrait"
            android:exported="true">
            <intent-filter>
                <action android:name="android.intent.action.MAIN" />
                <category android:name="android.intent.category.LAUNCHER" />
            </intent-filter>
        </activity>
        <activity
            android:name=".WebActivity"
            android:alwaysRetainTaskState="true"
            android:configChanges="orientation"
            android:launchMode="singleTop"
            android:noHistory="false"
            android:screenOrientation="portrait" />

        <service
            android:name=".MyFirebaseMessagingService"
            android:stopWithTask="false"
            android:exported="false">
            <intent-filter>
                <action android:name="com.google.firebase.MESSAGING_EVENT" />
            </intent-filter>
        </service>
        <provider
            android:name="androidx.core.content.FileProvider"
            android:authorities="@string/fileprovider"
            android:exported="false"
            android:grantUriPermissions="true">
            <meta-data
                android:name="android.support.FILE_PROVIDER_PATHS"
                android:resource="@xml/fileprovider" />
        </provider> 
        <meta-data
            android:name="com.google.firebase.messaging.default_notification_icon"
            android:resource="@drawable/ic_action_name" />
        <meta-data
            android:name="com.google.firebase.messaging.default_notification_color"
            android:resource="@color/colorAccent" />
    </application>

</manifest>

And this is how (most of) my gradle.app looks like:

android {
    ..

    ..
    flavorDimensions "version"

    compileSdkVersion 31
    defaultConfig {
        applicationId "xxxxxx"
        minSdkVersion 26
        targetSdkVersion 31
        versionCode 203
        versionName "2.0.3"
        testInstrumentationRunner 'androidx.test.runner.AndroidJUnitRunner'
    }

    productFlavors {
        janbarn {
            dimension "version"
            applicationIdSuffix ".flavor1"
        }
        shapeshifterzz {
            dimension "version"
            applicationIdSuffix ".flavor2"
        }
        dwleven {
            dimension "version"
            applicationIdSuffix ".flavor3"
        }
    }

    buildTypes {
        release {
            proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
            productFlavors.flavor1.signingConfig signingConfigs.flavor1
            productFlavors.flavor2.signingConfig signingConfigs.flavor2
            productFlavors.flavor3.signingConfig signingConfigs.flavor3
        }
    }

    sourceSets {
        janbarn {
            res {
                srcDirs 'src/flavor1/res'
            }
            java {
                srcDirs 'src/flavor1/java'
            }
        }
        shapeshifterzz {
            res {
                srcDirs 'src/flavor2/res'
            }
            java {
                srcDirs 'src/flavor2/java'
            }
        }
        dwleven {
            res {
                srcDirs 'src/flavor3/res'
            }
            java {
                srcDirs 'src/flavor3/java'
            }
        }
    }

    compileOptions{
        sourceCompatibility JavaVersion.VERSION_11
        targetCompatibility JavaVersion.VERSION_11
    }
    lint {
        checkReleaseBuilds false
    }
    buildFeatures {
        viewBinding true
    }
}

I have anonymized some of the names since im running this project for a friend of mine.

I hope someone can help me with this one, might safe me some more hours of digging!

I have tried to use different launchmodes, tried the persistent state and alwaysRetainTaskState but nothing seems to help. And again in flavor1 there is no problem.