tl;dr: Yes - a C# MemoryMappedFile will expand - but only to the next largest multiple of Environment.SystemPageSize (typically 4,096 bytes). Once its size exceeds that multiple an exception will be thrown.
I came across this question because I was looking into ways to use MemoryMappedFile to get around the fact that MemoryStream (whose backing field is a byte array) is limited to int.MaxValue values.
I saw the same behaviour that you did where data could be written/read beyond the specified capacity, and noted that the documentation for MemoryMappedFile.CreateOrOpen(string mapName, long capacity) clearly says (emphasis mine):
capacity
Int64
The maximum size, in bytes, to allocate to the memory-mapped file.
which clearly doesn't tally with what we saw.
As Hans Passant points out, the size of the view stream is effectively a multiple of Environment.SystemPageSize which, according to the Wikipedia page for Page Size traditionally had a fixed value of 4,096 bytes.
You can confirm this by inspecting some of the properties on MemoryMappedViewStream, e.g.:
var pageSize = Environment.SystemPageSize;
Console.WriteLine($"System page size: {pageSize} bytes");
Console.WriteLine();
var capacity = 1;
Console.WriteLine($"Creating MemoryMappedFile with capacity of {capacity} byte(s)");
using ( var memoryMappedFile = MemoryMappedFile.CreateOrOpen("Demo", capacity) )
using ( var viewStream = memoryMappedFile.CreateViewStream() )
{
Console.WriteLine($"View stream capacity: {viewStream.Capacity} bytes");
Console.WriteLine($"View stream length: {viewStream.Length} bytes");
Console.WriteLine();
var randomBytes = new byte[pageSize];
new Random().NextBytes(randomBytes);
viewStream.Write(randomBytes, offset: 0, count: randomBytes.Length);
Console.WriteLine($"{randomBytes.Length} bytes written to view stream");
viewStream.Position = 0;
for ( int index = 0; index < randomBytes.Length; index++ )
{
var writtenByte = randomBytes[index];
var readByte = viewStream.ReadByte();
if ( readByte != writtenByte )
throw new Exception($"Read byte at index {index} ({readByte}) was not the same as written ({writtenByte})");
}
Console.WriteLine($"{randomBytes.Length} bytes successfully read and verified from view stream");
Console.WriteLine();
// Attempt to write another byte (this should throw a NotSupportedException )
try
{
viewStream.WriteByte(0);
throw new InvalidOperationException("An extra byte was written to the view stream when it should not have");
}
catch ( NotSupportedException ex )
{
Console.WriteLine($"Unable to write additional bytes to view stream:{Environment.NewLine} {ex.Message}");
}
}
which outputs:
System page size: 4096 bytes
Creating MemoryMappedFile with capacity of 1 byte(s)
View stream capacity: 4096 bytes
View stream length: 4096 bytes
4096 bytes written to view stream
4096 bytes successfully read and verified from view stream
Unable to write additional bytes to view stream:
Unable to expand length of this stream beyond its capacity.
Note: The types in the C# System.IO.MemoryMappedFiles namespace are effectively C# wrappers for Windows API calls, e.g. CreateFileMappingW and OpenFileMappingW, for which there is a decent amount of documentation in the Microsoft documentation.
For instance, on Creating a File Mapping Object it says (emphasis mine):
File Mapping Size
The size of the file mapping object is independent of the size of the file being mapped. However, if the file mapping object is larger than the file, the system expands the file before CreateFileMapping returns. If the file mapping object is smaller than the file, the system maps only the specified number of bytes from the file.
The dwMaximumSizeHigh and dwMaximumSizeLow parameters of CreateFileMapping allow you to specify the number of bytes to be mapped from the file:
- When you do not want the size of the file to change (for example, when mapping read-only files), call
CreateFileMapping and specify zero for both dwMaximumSizeHigh and dwMaximumSizeLow. Doing this creates a file mapping object that is exactly the same size as the file. Otherwise, you must calculate or estimate the size of the finished file because file mapping objects are static in size; once created, their size cannot be increased or decreased.
On Managing Memory-Mapped Files it says:
What Do Memory-Mapped Files Have to Offer?
One advantage to using MMF I/O is that the system performs all data transfers for it in 4K pages of data.
This agrees with the documentation for MemoryMappedFile.CreateViewStream which says (emphasis mine):
To create a complete view of the memory-mapped file, specify 0 (zero) for the size parameter. If you do this, the size of the view might be larger than the size of the source file on disk. This is because views are provided in units of system pages, and the size of the view is rounded up to the next system page size.
In other words, as an implementation detail of MemoryMappedFile (i.e. it being a wrapper for Windows API calls), the capacity passed to the static constructors is effectively rounded up to the next largest multiple of the system page size because the virtual address space is split into pages (of typically 4,096 bytes).
Related questions:
