Encrypt and Decrypt C# Rijndael Managed out of memory

Question

I am currently programing a little software like WinRAR. When I encrypt or decrypt a array of bytes corresponding to my file, I have a large memory leak (or a high RAM consumption) and sometimes when I want to encrypt a file of 500MB it throws an OutOfMemoryException.

There is my code:

internal static class ResourceSecurity
{
    #region STATIC FIELDS

    private static readonly String PASSWORD_HASH = "ceýº–hÞ6Ü+rdÚ‡AÃÎ^oð#Èh";

    private static readonly String SALT_KEY = "žÀÚÔceýº–h•e:÷KÚ‡AÃÎ^oð#ÈhÀÍÄ";

    private static readonly String IV_KEY = "ª<±kéÞ6Ü+\f¥Ÿ„’²";

    #endregion

    #region STATIC METHODS

    private static RijndaelManaged CreateAlgo()
    {
        Byte[] _keyBytes = new Rfc2898DeriveBytes(PASSWORD_HASH, Encoding.ASCII.GetBytes(SALT_KEY)).GetBytes(256 / 8);
        RijndaelManaged _rijndael = new RijndaelManaged();

        _rijndael.Mode = CipherMode.CBC;
        _rijndael.Padding = PaddingMode.Zeros;
        _rijndael.Key = _keyBytes;
        _rijndael.IV = Encoding.ASCII.GetBytes(IV_KEY);

        return _rijndael;
    }


    private static Byte[] ConvertBuffer(Byte[] buffer, ICryptoTransform transformation)
    {
        Byte[] _newBuffer = null;
        MemoryStream _memoryStream = new MemoryStream();

        using (CryptoStream _crypto = new CryptoStream(_memoryStream, transformation, CryptoStreamMode.Write))
        {
            _crypto.Write(buffer, 0, buffer.Length);
            _newBuffer = _memoryStream.ToArray();
        }

        return _newBuffer;
    }

    internal static Byte[] Encrypt(Byte[] bufferToEncrypt)
    {
        return ConvertBuffer(bufferToEncrypt, CreateAlgo().CreateEncryptor());
    }

    internal static Byte[] Decrypt(Byte[] encryptedBuffer)
    {
        return ConvertBuffer(encryptedBuffer, CreateAlgo().CreateDecryptor());
    }

    #endregion
}

Thank you!

High memory consumption is not the same thing as memory leak. — Mihai Caracostea, Aug 09 '15 at 18:41
Your hash/salt/iv would look and feel a lot nicer in some more readable notation (hex or even byte array). — Mihai Caracostea, Aug 09 '15 at 18:53
Agreed with Mihai Caracostea, a byte array is the conventional way to go here. But if you really want a string for this, even Base64 is better than trying to use these wild characters. The risk you take here besides difficulty reading it is what happens if and when your bytes yield an unprintable character in whatever encoding you are using. Bytes bypass that issue entirely, and Base64 encoding handles it in an acceptable way. — WDS, Aug 09 '15 at 20:21
I understand, I will change it to a byte array then. Thanks for the advice — Eastrall, Aug 10 '15 at 10:16
Large object heap fragmentation can cause OOM in this case. See "Large Object Heap" in this article https://www.jetbrains.com/dotmemory/help/NET_Memory_Management_Concepts.html — Ed Pavlov, Aug 10 '15 at 20:03

score 1 · Answer 1 · answered Aug 09 '15 at 18:50

1

Most likely your process is running in x86 mode and hitting a low memory limit. See here for Windows memory limits for processes. Make sure you run your process as x64 as a workaround. As a definite solution, try not to keep in memory all that amount of data. Only for your example, the cleartext and cypher data adds to 1GB. Try to perform the encryption on an input stream targetting a non-memory stream (maybe a filestream or network stream?). That way your application won't put as much stress on the system memory and could scale a lot nicer.

answered Aug 09 '15 at 18:50

Mihai Caracostea

8,336
4
27
46

Actually, at its worst, it appears to be reaching 1.5 GB. 500 megabytes each for buffer, _newbuffer, and _memoryStream inside the CryptoStream "using" block. But you are right about not keeping it all in memory. If Orion were to pull the plaintext in from a FileStream, convert it in the CryptoStream, and push it directly back into an output FileStream, the memory consumption for this program would drop to almost nothing. – WDS Aug 09 '15 at 20:25
@WDS Indeed, I've been using this kind of strategy for some time now. Most libraries I create using encryption this way only use Stream references for both in and out. This way you can throw any kind of stream at it... FileStream, NetworkStream, MemoryStream, custom streams... And in any in-out combination. Memory footprint is barely noticeable. – Mihai Caracostea Aug 09 '15 at 22:20
The problem is that I need to get the data from the file and keep it in memory for a short time. When I close the streams it should release the crypto stream and memory stream data right ? Maybe I should pass a reference to the buffer array like this: `private static Byte[] ConvertBuffer(Byte[] buffer, ICryptoTransform transformation)` ? – Eastrall Aug 10 '15 at 10:20
Why exactly do you need to keep it in memory? Maybe there's an architectural flaw in your application right there. – Mihai Caracostea Aug 10 '15 at 10:36

Encrypt and Decrypt C# Rijndael Managed out of memory

1 Answers1