0

I'm trying to transfer a large file in "chunks" that then have their hashes validated. I am looking into some performance issues, particularly in UNC paths, and I wrote an IO test that exhibits strange behavior.

Here is the code:

string path = "\\\\unc\\path\\test.txt";
long fileSize = 1000000000;
int chunkSize = 1000000;

if (File.Exists(path))
{
    File.Delete(path);
}

using (FileStream fs = File.Create(path))
{
    fs.SetLength(fileSize);
}

byte[] data = new byte[chunkSize];
for (long i = 0; i < fileSize; i+= chunkSize)
{
    for (int j = 0; j < chunkSize; j++)
    {
        data[j] = (byte)i; // this is just to write different data each time
    }
    int thisChunkSize = (int)Math.Min(fileSize - i, chunkSize);

    using (FileStream fs = new FileStream(path, FileMode.Open, FileAccess.ReadWrite, FileShare.ReadWrite))
    {
        fs.Lock(i, thisChunkSize);

        fs.Seek(i, SeekOrigin.Begin);
        fs.Write(data, 0, thisChunkSize);

        //fs.Seek(i, SeekOrigin.Begin);
        //fs.Read(data, 0, thisChunkSize);
    }

    using (SHA1 sha1 = SHA1.Create())
    {
        sha1.ComputeHash(data);
    }
}

Running the code as-is, it completes in about 2.5 minutes. When I uncomment the fs.Seek and fs.Read, it completes in about 30 seconds. Running on a local path, it takes about 6.5 seconds either way.

My main theory is there there is some OS bottleneck that is slowing me down when repeatedly opening and closing FileStreams back-to-back. Is there any explanation for why a more expensive operation would result in better performance?

AMunns
  • 3
  • 2
  • Does `path` change? Why are you opening and closing it in the loop? – rhughes Jan 10 '22 at 22:42
  • Path stays the same in practice. Normally it would be a set of separate web requests, and I do cache the FileStream so that I'm not opening and closing it each time. I was just wondering why this behavior was showing so I can maybe get some insight into FileStream performance on UNC paths. – AMunns Jan 10 '22 at 23:26

0 Answers0