Read all contents of memory mapped file or Memory Mapped View Accessor without knowing the size of it

Question

I need something similar to ReadToEnd or ReadAllBytes to read all of the contents of the MemoryMappedFile using the MappedViewAccessor if I don't know the size of it, how can I do it?

I have searched for it, I have seen this question, but it is not the thing I am looking for:

How can I quickly read bytes from a memory mapped file in .NET?

Edit:

There is a problem, the (int)stream.Length is not giving me the correct length, it rather gives the size of the internal buffer used! I need to refresh this question because it is very pressing.

Answer 1

Rather use the Stream:

public static Byte[] ReadMMFAllBytes(string fileName)
{
    using (var mmf = MemoryMappedFile.OpenExisting(fileName))
    {
        using (var stream = mmf.CreateViewStream())
        {
            using (BinaryReader binReader = new BinaryReader(stream))
            {
                return binReader.ReadBytes((int)stream.Length);
            }
        }
    }
}

Answer 2

This is difficult to answer since there are still many details of your application that you haven't specified, but I think both Guffa's and Amer's answers are still partially correct:

• A MemoryMappedFile is more memory than file; it is a sequence of 4Kb pages in memory. So, stream.Length will in fact give you all of the bytes (there is no "internal buffer size"), but it might give you more bytes than you expect since the size will always be rounded up to a 4Kb boundary.
• The "file" semantic comes from associating the MemoryMappedFile to a real filesystem file. Assuming that the process which creates the file always adjusts the file size, then you can get the precise size of the file via the fileSystem.

If all of the above would fit your application, then the following should work:

    static byte[] ReadMemoryMappedFile(string fileName)
    {
        long length = new FileInfo(fileName).Length;
        using (var stream = File.Open(fileName, FileMode.OpenOrCreate, FileAccess.Read, FileShare.ReadWrite))
        {
            using (var mmf = MemoryMappedFile.CreateFromFile(stream, null, length, MemoryMappedFileAccess.Read, null, HandleInheritability.Inheritable, false))
            {
                using (var viewStream = mmf.CreateViewStream(0, length, MemoryMappedFileAccess.Read))
                {
                    using (BinaryReader binReader = new BinaryReader(viewStream))
                    {
                        var result = binReader.ReadBytes((int)length);
                        return result;
                    }
                }
            }
        }
    }

To write the data, you can use this:

    private static void WriteData(string fileName, byte[] data)
    {
        using (var stream = File.Open(fileName, FileMode.OpenOrCreate, FileAccess.ReadWrite, FileShare.ReadWrite))
        {
            using (var mmf = MemoryMappedFile.CreateFromFile(stream, null, data.Length, MemoryMappedFileAccess.ReadWrite, null, HandleInheritability.Inheritable, true))
            {
                using (var view = mmf.CreateViewAccessor())
                {
                    view.WriteArray(0, data, 0, data.Length);
                }
            }

            stream.SetLength(data.Length);  // Make sure the file is the correct length, in case the data got smaller.
        }
    }

But, by the time you do all of the above you might do just as well to use the file directly and avoid the memory mapping. If mapping it to the filesystem isn't acceptable, then Guffa's answer of encoding the length (or an end marker) in the data itself is probably best.

Answer 3

You can't do that.

A view accessor is created with a minimum size of a system page, which means that it may be larger than the actual file. A view stream is just a stream form of an accessor, so it will also give the same behaviour.

"views are provided in units of system pages, and the size of the view is rounded up to the next system page size"

http://msdn.microsoft.com/en-us/library/dd267577.aspx

The accessor will gladly read and write outside the actual file without throwing an exception. When reading, any bytes outside the file will just be zero. When writing, the bytes written outside the file are just ignored.

To read the file from a memory mapped file with the exact size of the original file, you have to already know that size.

Answer 4

Stream created by MemoryMappedFile has a length aligned to file system page size (usually 4096). You have to get the file size from somewhere else. If it is memory mapped file you could use that code:

byte[] ReadAllMemoryMappedFileBytes(string filePath)
{
    var fileInfo = new FileInfo(filePath);
    using (var file = MemoryMappedFile.CreateFromFile(filePath, FileMode.Open))
    using (var stream = file.CreateViewAccessor())
    {
        byte[] bytes = new byte[fileInfo.Length];
        stream.ReadArray(0, bytes, 0, bytes.Length);
        return bytes;
    }
}

Answer 5

Use FileInfo class to get length as shown below

using System.Data;
using System.IO;
using System.IO.Compression;
using System.IO.MemoryMappedFiles;

// ...

public void WriteToMemoryMap(DataSet ds, string key, string fileName)
{
    var bytes = CompressData(ds);
    using (MemoryMappedFile objMf = MemoryMappedFile.CreateFromFile(fileName, FileMode.OpenOrCreate, key, bytes.Length))
    {
        using (MemoryMappedViewAccessor accessor = objMf.CreateViewAccessor())
        {
            accessor.WriteArray(0, bytes, 0, bytes.Length);
        }
    }
}
public DataSet ReadFromMemoryMap(string fileName)
{
    var fi = new FileInfo(fileName);
    var length = (int)fi.Length;
    var newBytes = new byte[length];
    using (MemoryMappedFile objMf = MemoryMappedFile.CreateFromFile(fileName, FileMode.Open))
    {
        using (MemoryMappedViewAccessor accessor = objMf.CreateViewAccessor())
        {
            accessor.ReadArray(0, newBytes, 0, length);
        }
    }
    return DecompressData(newBytes);
}
public byte[] CompressData(DataSet ds)
{
    try
    {
        byte[] data = null;
        var memStream = new MemoryStream();
        var zipStream = new GZipStream(memStream, CompressionMode.Compress);
        ds.WriteXml(zipStream, XmlWriteMode.WriteSchema);
        zipStream.Close();
        data = memStream.ToArray();
        memStream.Close();
        return data;
    }
    catch (Exception)
    {
        return null;
    }
}
public DataSet DecompressData(byte[] data)
{
    try
    {
        var memStream = new MemoryStream(data);
        var unzipStream = new GZipStream(memStream, CompressionMode.Decompress);
        var objDataSet = new DataSet();
        objDataSet.ReadXml(unzipStream, XmlReadMode.ReadSchema);
        unzipStream.Close();
        memStream.Close();
        return objDataSet;
    }
    catch (Exception)
    {
        return null;
    }
}

Answer 6

Just the @Amer Sawan solution translated to Vb.NET:

' Usage Example:
' Dim ReadBytes As Byte() = ReadMemoryMappedFile(Name:="My MemoryMappedFile Name") ' Read the byte-sequence from memory.
' Dim Message As String = System.Text.Encoding.ASCII.GetString(ReadBytes.ToArray) ' Convert the bytes to String.
' Message = Message.Trim({ControlChars.NullChar}) ' Remove null chars (leading zero-bytes)
' MessageBox.Show(Message, "", MessageBoxButtons.OK) ' Show the message.    '
'
''' <summary>
''' Reads a byte-sequence from a <see cref="IO.MemoryMappedFiles.MemoryMappedFile"/> without knowing the exact size.
''' Note that the returned byte-length is rounded up to 4kb, 
''' this means if the mapped memory-file was written with 1 byte-length, this method will return 4096 byte-length. 
''' </summary>
''' <param name="Name">Indicates an existing <see cref="IO.MemoryMappedFiles.MemoryMappedFile"/> assigned name.</param>
''' <returns>System.Byte().</returns>
Private Function ReadMemoryMappedFile(ByVal Name As String) As Byte()

    Try
        Using MemoryFile As IO.MemoryMappedFiles.MemoryMappedFile =
            IO.MemoryMappedFiles.MemoryMappedFile.OpenExisting(Name, IO.MemoryMappedFiles.MemoryMappedFileRights.ReadWrite)

            Using Stream = MemoryFile.CreateViewStream()

                Using Reader As New BinaryReader(Stream)

                    Return Reader.ReadBytes(CInt(Stream.Length))

                End Using ' Reader

            End Using ' Stream

        End Using ' MemoryFile

    Catch exNoFile As IO.FileNotFoundException
        Throw
        Return Nothing

    Catch ex As Exception
        Throw

    End Try

End Function

Answer 7

I would like to have something from MemoryStream .ToArray() method to return all bytes, and the code below work for me:

using (MemoryMappedFile mmf = MemoryMappedFile.OpenExisting(MemoryMappedName))
{
    using (MemoryMappedViewStream stream = mmf.CreateViewStream())
    {
        using (MemoryStream memStream = new MemoryStream())
        {
            stream.CopyTo(memStream);
            return memStream.ToArray();
        }
    }
}

Cheers!