0

I'm in the middle of porting an older application to .Net 6, and have hit a stumbling block of the encryption / decryption method is now failing. It still works perfectly fine under .Net 4.x.x.

The error being thrown is,

"Padding is invalid and cannot be removed."

Code: - Updated to actual original code. This code worked fine when targeting .Net 4.7.2, however after moving the code to .Net 6.0 RC2, it started to lose anything greater than 32 chars of the decrypted string, which lead to errors elsewhere as the strings weren't complete.

For context. This was running on a webhost & a desktop client, to encrypt messages in transit. The webhost has been updated and validated to be sending the correct encrypted value (decrypting the message using the .Net 4 client is successful). However, the .Net 6 desktop client isn't decrypting it correctly and is losing characters in the decrypted string.

#region Encrypt method(s)

    private const int Keysize = 256;
    private const int Blocksize = 128;
    private const int DerivationIterations = 1000;

    public async Task<string> EncryptStringWithValidatedPadding(string plainText, string passPhrase)
    {
        string encrypted = null;
        
        bool valid = false;

        while (!valid)
        {
            encrypted = await Encrypt(plainText, passPhrase);
            if (!string.IsNullOrEmpty(await Decrypt(encrypted, passPhrase)))
            {
                valid = true;
            }
        }
        return encrypted;
    }

    private async Task<string> Encrypt(string plainText, string passPhrase)
    {
        var saltStringBytes = GenerateRandomEntropy(32); // 256 bits
        var ivStringBytes = GenerateRandomEntropy(16); // 128 bits
        byte[] plainTextBytes = Convert.FromBase64String(plainText);
        using (var password = new Rfc2898DeriveBytes(Convert.FromBase64String(passPhrase), saltStringBytes, DerivationIterations))
        {
            var keyBytes = password.GetBytes(Keysize / 8);
            using (var symmetricKey = new AesManaged())
            {
                symmetricKey.KeySize = Keysize;
                symmetricKey.BlockSize = Blocksize;
                symmetricKey.Mode = CipherMode.CBC;
                symmetricKey.Padding = PaddingMode.PKCS7;
                using (var encryptor = symmetricKey.CreateEncryptor(keyBytes, ivStringBytes))
                {
                    using (var memoryStream = new MemoryStream())
                    {
                        using (var cryptoStream = new CryptoStream(memoryStream, encryptor, CryptoStreamMode.Write))
                        {
                            cryptoStream.Write(plainTextBytes, 0, plainTextBytes.Length);
                            cryptoStream.FlushFinalBlock();
                            var cipherTextBytes = saltStringBytes;
                            cipherTextBytes = cipherTextBytes.Concat(ivStringBytes).ToArray();
                            cipherTextBytes = cipherTextBytes.Concat(memoryStream.ToArray()).ToArray();
                            memoryStream.Close();
                            cryptoStream.Close();
                            var encrypted64String = Convert.ToBase64String(cipherTextBytes);
                            return encrypted64String;
                        }
                    }
                }
            }
        }
    }

    private static byte[] GenerateRandomEntropy(int byteSize)
    {
        var randomBytes = new byte[byteSize];
        using (var rngCsp = new RNGCryptoServiceProvider())
        {
            rngCsp.GetBytes(randomBytes);
        }
        return randomBytes;
    }

    #endregion

    #region Decrypt method


    public static async Task<string> Decrypt(string cipherText, string passPhrase)
    {
        try
        {
            var cipherTextBytesWithSaltAndIv = Convert.FromBase64String(cipherText);
            var saltStringBytes = cipherTextBytesWithSaltAndIv.Take(Keysize / 8).ToArray();
            var ivStringBytes = cipherTextBytesWithSaltAndIv.Skip(Keysize / 8).Take(Blocksize / 8).ToArray();
            var cipherTextBytes = cipherTextBytesWithSaltAndIv.Skip((Keysize / 8) + Blocksize / 8).Take(cipherTextBytesWithSaltAndIv.Length - ((Keysize / 8) + Blocksize / 8)).ToArray();
            using (var password = new Rfc2898DeriveBytes(Convert.FromBase64String(passPhrase), saltStringBytes, DerivationIterations))
            {
                var keyBytes = password.GetBytes(Keysize / 8);
                using (var symmetricKey = new AesManaged())
                {
                    symmetricKey.KeySize = 256;
                    symmetricKey.BlockSize = Blocksize;
                    symmetricKey.Mode = CipherMode.CBC;
                    symmetricKey.Padding = PaddingMode.PKCS7;
                    using (var decryptor = symmetricKey.CreateDecryptor(keyBytes, ivStringBytes))
                    {
                        using (var memoryStream = new MemoryStream(cipherTextBytes))
                        {
                            using (var cryptoStream = new CryptoStream(memoryStream, decryptor, CryptoStreamMode.Read))
                            {
                                var plainTextBytes = new byte[cipherTextBytes.Length];
                                var decryptedByteCount = cryptoStream.Read(plainTextBytes, 0, plainTextBytes.Length);
                                memoryStream.Close();
                                cryptoStream.Close();
                                return Encoding.UTF8.GetString(plainTextBytes, 0, decryptedByteCount);
                            }
                        }
                    }
                }
            }
        }
        catch (Exception ex)
        {
            Console.WriteLine(ex.Message);
        }
        return null;
    }

    #endregion

This is called with,

encryptedString = await new EncryptDecrypt().EncryptStringWithValidatedPadding(b64String, Convert.ToBase64String(Encoding.UTF8.GetBytes(passPhrase)));

I am assuming that saving the IV should solve this, but I'm wondering if there are any obvious flaws here that I'm just not seeing.

Can anyone explain it?

Update: As suggested I've refactored the code to the below. I've also stripped it right back for the minute to ensure the underlying algo's work.

Ref: https://learn.microsoft.com/en-us/dotnet/api/system.security.cryptography.aes?view=net-6.0

namespace Encryption_Helper
{
    public class EncryptDecrypt
    {
        #region Encrypt method(s)

        private static byte[] bytes = new byte[] { 0x49, 0x76, 0x61, 0x6e, 0x20, 0x4d, 0x65, 0x64, 0x76, 0x65, 0x64, 0x65, 0x76 };

        private const int Keysize = 256;
        private const int Blocksize = 128;
        private const int DerivationIterations = 1000;

        public static async Task<string> EncryptStringWithValidatedPadding(string plainText, string passPhrase)
        {
            string encrypted = null;

            bool valid = false;

            while (!valid)
            {
                encrypted = await Encrypt(plainText, passPhrase);
                if (!string.IsNullOrEmpty(await Decrypt(encrypted, passPhrase)))
                {
                    valid = true;
                }
            }
            return encrypted;
        }

        private static async Task<string> Encrypt(string plainText, string passPhrase)
        {
            using (var password = new Rfc2898DeriveBytes(Convert.FromBase64String(passPhrase), bytes, DerivationIterations))
            {
                var keyBytes = password.GetBytes(Keysize / 8);
                var ivBytes = password.GetBytes(Blocksize / 8);
                using (var aes = Aes.Create())
                {
                    aes.Key = keyBytes;
                    aes.IV = ivBytes;

                    ICryptoTransform encryptor = aes.CreateEncryptor(aes.Key, aes.IV);

                    using (MemoryStream msEncrypt = new MemoryStream())
                    {
                        using (CryptoStream csEncrypt = new CryptoStream(msEncrypt, encryptor, CryptoStreamMode.Write))
                        {
                            using (StreamWriter swEncrypt = new StreamWriter(csEncrypt))
                            {
                                swEncrypt.Write(plainText);
                            }
                            plainText = Convert.ToBase64String(msEncrypt.ToArray());
                        }
                    }
                }
                return plainText;
            }
        }

        #endregion

        #region Decrypt method


        public static async Task<string> Decrypt(string cipherText, string passPhrase)
        {
            try
            {
                using (var password = new Rfc2898DeriveBytes(Convert.FromBase64String(passPhrase), bytes, DerivationIterations))
                {
                    var keyBytes = password.GetBytes(Keysize / 8);
                    var ivBytes = password.GetBytes(Blocksize / 8);
                    using (var aes = Aes.Create())
                    {
                        aes.Key = keyBytes;
                        aes.IV = ivBytes;
                     
                        ICryptoTransform decryptor = aes.CreateDecryptor(aes.Key, aes.IV);

                        using (var memoryStream = new MemoryStream(Convert.FromBase64String(cipherText)))
                        {
                            using (var cryptoStream = new CryptoStream(memoryStream, decryptor, CryptoStreamMode.Read))
                            {
                                using (StreamReader srDecrypt = new StreamReader(cryptoStream))
                                { 
                                    cipherText = srDecrypt.ReadToEnd();
                                }
                            }
                        }
                        return cipherText;
                    }
                }
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex.Message);
                return null;
            }
        }

        #endregion
    }
}

It is still throwing a padding error!

user2295457
  • 101
  • 11
  • 2
    I assume the error is thrown when trying to decrypt? A few other questions: why do the back-and-forth base64 conversion with string instead of passing in byte[] from the start? And what exactly is the `while (!valid)` loop for? I.e. what reason would there be for the for the decryption to not be valid on the first try but then work in a subsequent iteration? Is it possible that there was a problem in the previous version and this loop was there to catch it? Maybe it had to "hope" for the correct padding length to occur? – Marcus Ilgner Oct 22 '21 at 15:40
  • 1
    I saw in the API documentation that they're not calling write on the stream directly but instead use ``` using (StreamWriter swEncrypt = new StreamWriter(csEncrypt)){ //Write all data to the stream. swEncrypt.Write(plainText); } ``` Not sure whether this is equivalent but it might be worth a shot... – Marcus Ilgner Oct 22 '21 at 15:43
  • Yes the error is produced while attempting to decrypt. The while (!valid) loop is purely something I've added to allow for quick assessment of the encryption / decryption process. It's also intended to test for bad encryption via iteration. – user2295457 Oct 22 '21 at 15:48
  • 3
    Salt, IV and ciphertext are separated in `Decrypt()`. In `Encrypt()`, however, the corresponding concatenation seems to be missing. Actually, this shouldn't work under .Net 4.x.x either. – Topaco Oct 22 '21 at 16:21
  • 1
    It does seem weird that there's so much more happening than in the official docs. https://learn.microsoft.com/en-us/dotnet/api/system.security.cryptography.cryptostream?view=net-6.0 - they don't seem to have to do salting and IV generation manually and everything looks much more concise. If possible (with regards to compatibility with encrypted data from old versions), I'd recommend refactoring it in a way that makes it more akin to the docs there. With crypto, the less one does manually, the better. – Marcus Ilgner Oct 22 '21 at 16:36
  • 1
    To make clear what @Topaco is saying, you must change `Encrypt` to prepend the salt and IV bytes to the ciphertext prior to base64 encoding. I would imagine you could simply `Write()` them into the MemoryStream before wrapping the CryptoStream around it. This could not possibly have worked in any previous version of .Net. – President James K. Polk Oct 22 '21 at 17:16
  • Apologies! I messed up and pasted the wrong snippet in the op.. Updated to reflect the actual (previously working in .Net 4 code) & refactored code intended for .Net 6. – user2295457 Oct 22 '21 at 17:52
  • I can't reproduce the problem. The only message regarding padding is thrown if the password is not a BASE64 string. Once that's fixed `EncryptStringWithValidatedPadding ` just works. Which is a strange requirement -why not just use the password as-is ? Replacing `Convert.FromBase64String(passPhrase)` with just `passPhrase` works – Panagiotis Kanavos Oct 26 '21 at 13:14

1 Answers1

3

Solved it!

public class EncryptDecrypt
    {
        #region Encrypt method(s)

        private const int Keysize = 256;
        private const int Blocksize = 128;
        private const int DerivationIterations = 1000;

        public static async Task<string> EncryptStringWithValidatedPadding(string plainText, string passPhrase)
        {
            string encrypted = null;

            bool valid = false;

            while (!valid)
            {
                encrypted = await Encrypt(plainText, passPhrase);
                if (!string.IsNullOrEmpty(await Decrypt(encrypted, passPhrase)))
                {
                    valid = true;
                }
            }
            return encrypted;
        }

        private static async Task<string> Encrypt(string plainText, string passPhrase)
        {
            var saltStringBytes = GenerateRandomEntropy(Keysize / 8); // 256 bits
            var ivStringBytes = GenerateRandomEntropy(Blocksize / 8); // 128 bits
            byte[] plainTextBytes = Convert.FromBase64String(plainText);
            using (var password = new Rfc2898DeriveBytes(Convert.FromBase64String(passPhrase), saltStringBytes, DerivationIterations))
            {
                var keyBytes = password.GetBytes(Keysize / 8);
                using (var aes = Aes.Create())
                {
                    aes.KeySize = Keysize;
                    aes.BlockSize = Blocksize;
                    aes.Mode = CipherMode.CBC;
                    aes.Padding = PaddingMode.PKCS7;

                    using (var memoryStream = new MemoryStream())
                    {
                        using (var cryptoStream = new CryptoStream(memoryStream, aes.CreateEncryptor(keyBytes, ivStringBytes), CryptoStreamMode.Write))
                        {
                            cryptoStream.Write(plainTextBytes, 0, plainTextBytes.Length);
                            cryptoStream.FlushFinalBlock();
                            var cipherTextBytes = saltStringBytes;
                            cipherTextBytes = cipherTextBytes.Concat(ivStringBytes).ToArray();
                            cipherTextBytes = cipherTextBytes.Concat(memoryStream.ToArray()).ToArray();
                            memoryStream.Close();
                            cryptoStream.Close();
                            var encrypted64String = Convert.ToBase64String(cipherTextBytes);
                            return encrypted64String;
                        }
                    }
                }
            }
        }

        private static byte[] GenerateRandomEntropy(int byteSize)
        {
            var randomBytes = new byte[byteSize];
            using (var rngCsp = new RNGCryptoServiceProvider())
            {
                rngCsp.GetBytes(randomBytes);
            }
            return randomBytes;
        }

        #endregion

        #region Decrypt method


        public static async Task<string> Decrypt(string cipherText, string passPhrase)
        {
            try
            {
                var cipherTextBytesWithSaltAndIv = Convert.FromBase64String(cipherText);
                var saltStringBytes = cipherTextBytesWithSaltAndIv.Take(Keysize / 8).ToArray();
                var ivStringBytes = cipherTextBytesWithSaltAndIv.Skip(Keysize / 8).Take(Blocksize / 8).ToArray();
                var cipherTextBytes = cipherTextBytesWithSaltAndIv.Skip((Keysize / 8) + Blocksize / 8).Take(cipherTextBytesWithSaltAndIv.Length - ((Keysize / 8) + Blocksize / 8)).ToArray();
                using (var password = new Rfc2898DeriveBytes(Convert.FromBase64String(passPhrase), saltStringBytes, DerivationIterations))
                {
                    var keyBytes = password.GetBytes(Keysize / 8);
                    using (var aes = Aes.Create())
                    {
                        aes.KeySize = Keysize;
                        aes.BlockSize = Blocksize;
                        aes.Mode = CipherMode.CBC;
                        aes.Padding = PaddingMode.PKCS7;

                        using (var ms = new MemoryStream(cipherTextBytes))
                        {
                            using (var cs = new CryptoStream(ms, aes.CreateDecryptor(keyBytes, ivStringBytes), CryptoStreamMode.Read))
                            {
                                using (StreamReader srDecrypt = new StreamReader(cs))
                                {

                                    // Read the decrypted bytes from the decrypting stream
                                    // and place them in a string.
                                    cipherText = srDecrypt.ReadToEnd();
                                }
                            }
                        }
                        return cipherText;
                    }
                }
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex.Message);
            }
            return null;
        }

        #endregion

After updating the decryption method, all's good in the world again.

It appears to me that .Net 6 broke the nested using loops, closing the stream before the return value was completely set.

user2295457
  • 101
  • 11
  • No it didn't. Your code had a bug. .NET 6 is .NET *Core* 6. .NET Core 1 was released 5 years ago. It's not new. If anything as fundamental as `using` blocks (not loops) was broken, thousands of developers would have noticed 5 years ago. Nothing would work if `using` was broken – Panagiotis Kanavos Oct 25 '21 at 15:11
  • @PanagiotisKanavos, ok I can appreciate that, care to elaborate on where you think the bug is? I posted this question to learn from it (my code issue is resolved with the above). – user2295457 Oct 26 '21 at 12:44
  • For starters, I don't get any error. The only error is thrown because the password needs to be a BASE64-encoded string. Once that's fixed by either removing `Convert.FromBase64String` or encoding the string before calling the method, everything works – Panagiotis Kanavos Oct 26 '21 at 13:17
  • Weird... The code I have calling this method generates a 256 bit password then converts said password to base64 via "Convert.ToBase64String(Encoding.UTF8.GetBytes(passPhrase))" for use in this encryption routine (as shown)... That password generation code hasn't been touched in several years so wasn't included here, the only changes I made were to the encryption / decryption methods as shown in this question. – user2295457 Oct 26 '21 at 15:06
  • Why convert to BASE64 if you need just the bytes? Why not pass the 32-byte password directly? – Panagiotis Kanavos Oct 26 '21 at 15:15
  • From what I can see in the commit history, I believe the base64 requirement was added due to string encoding issues that led to invalid padding errors. – user2295457 Oct 27 '21 at 10:45
  • What string encoding issues? This *caused* string encoding issues. Assuming something used by millions of developers for 20 years is broken only results in *more* problems. In this case assuming there is some kind of encoding issue caused the addition of code that does nothing at best (it produces the original bytes). If someone forgets about this though, it causes errors, like the one you encountered – Panagiotis Kanavos Oct 27 '21 at 11:46
  • Ok, so... It's a "string" value being used as a password. Calling Encoding.UTF8.GetBytes(password) and passing the returned "byte[]" to the encryption method led to invalid padding errors when decrypting. Converting the string to BASE64 first appeared to resolve this issue. Also, I'm not assuming the encryptions broken. Like I say, I can cut the code from the working perfectly fine .Net 4 app & paste the code straight into a new .Net 6 app and it will fail to work, something I did several times before posing this question. I must be missing something... – user2295457 Oct 27 '21 at 13:46
  • Also for some more context, the usage of this encryption method is for server / client secret sharing. The client encrypts the secret, passes the secret to the server, then via a separate channel sends the base64 string to decrypt the secret. There are multiple clients using this same technique, this is for isolation of secrets to "sender / server can decrypt". – user2295457 Oct 27 '21 at 13:51
  • None of this matters. The code you posted (the second part) works if you provide a real BASE64 string. It throws a padding error if you don't. This has nothing to do with any `using` bug. You haven't posted code that reproduces the problem nor the full exception text, only parts of the message – Panagiotis Kanavos Oct 27 '21 at 13:59