Decrypting using AES256 with BountyCastle .NET fails with "mac check in GCM failed"

Question

I'm using the BountyCastle.NetCore NuGet library to decrypt an already prepared encrypted message from our servers. I was able to utilize both the Java cryptography libraries (native) and Python (pycrptodome) implementation with success. In general, our servers are preparing the message as this:

The decrypt strategy is:

• Prepare the AES-256 secret key by base64 decoding the cryptographyKey.

• Prepare the cipher text bytes by base64 decoding the message body.

• Remove Tag from the cipher text.
• Prepare AAD by copying the first 16 bytes from cipher text.
• Prepare NONCE (IV) by copying bytes 4 – 16 from AAD.
• Then decrypt with AES – GCM, tag length = 16 bytes; with the input
  from 1, 3, 4, 5.

Applying the general algorithm based on this example, my implementation looks like this:

Note: I didn't apply the Tag within the code below because it is more of a verification step and I couldn't get past decrypting the message.

public string Decrypt(string cryptoKey, string cypherText)
{
    string sR = string.Empty;
    try
    {
        // Prepare the AES-256 secret key by decoding the cryptographyKey
        var keyBytes = Convert.FromBase64String(cryptoKey);

        // Prepare the Cypher bytes by decoding the cypher text
        var cypherBytes = Convert.FromBase64String(cypherText);

        // <------AAD----->
        // [---][--NONCE--][-----Encrypted Message-----][---TAG---]
        // <-------------------CypherText------------------------->

        // Prepare the AAD - first AAD_LENGTH from cypher
        var aad = new Byte[AAD_LENGTH];
        Array.Copy(cypherBytes, 0, aad, 0, AAD_LENGTH);

        // Prepare NONCE - bytes 4-16 from AAD
        var nonce = new Byte[NONCE_LENGTH];
        Array.Copy(aad, AAD_LENGTH - NONCE_LENGTH, nonce, 0, NONCE_LENGTH);

        // Extract the encrypted message within the body - between AAD and TAG section
        var encryptedMsgSize = cypherBytes.Length - TAG_LENGTH - AAD_LENGTH;
        var encryptedMsg = new Byte[encryptedMsgSize];
        Array.Copy(cypherBytes, AAD_LENGTH, encryptedMsg, 0, encryptedMsgSize);

        GcmBlockCipher cipher = new GcmBlockCipher(new AesEngine());
        AeadParameters parameters = new AeadParameters(new KeyParameter(keyBytes), 128, nonce);

        cipher.Init(false, parameters);
        cipher.ProcessAadBytes(cypherBytes, 0, AAD_LENGTH);

        byte[] plainBytes = new byte[cipher.GetOutputSize(encryptedMsg.Length)];
        Int32 retLen = cipher.ProcessBytes(encryptedMsg, 0, encryptedMsg.Length, plainBytes, 0);
        cipher.DoFinal(plainBytes, retLen);

        sR = Encoding.UTF8.GetString(plainBytes).TrimEnd("\r\n\0".ToCharArray());
    }
    catch (Exception ex)
    {
        Console.WriteLine(ex.Message);
        Console.WriteLine(ex.StackTrace);
    }

    return sR;
}

Despite trying many different combinations, setting different parameters etc, it always throws an exception on the line:

cipher.DoFinal(plainBytes, retLen);

with "mac check in GCM failed".

Any other suggestions would be welcomed :-)