The basic idea is to open the MID file for reading and writing. The simple single-threaded way to do it is:
private static void FunkyCopy(string srcFname, string midFname, string dstFname)
{
using (FileStream srcFile = new FileStream(srcFname, FileMode.Open, FileAccess.Read, FileShare.None),
midFile = new FileStream(midFname, FileMode.Create, FileAccess.ReadWrite,
FileShare.ReadWrite),
dstFile = new FileStream(dstFname, FileMode.Create, FileAccess.Write, FileShare.None))
{
long totalBytes = 0;
var buffer = new byte[65536];
while (totalBytes < srcFile.Length)
{
var srcBytesRead = srcFile.Read(buffer, 0, buffer.Length);
if (srcBytesRead > 0)
{
// write to the mid file
midFile.Write(buffer, 0, srcBytesRead);
// now read from mid and write to dst
midFile.Position = totalBytes;
var midBytesRead = midFile.Read(buffer, 0, srcBytesRead);
if (midBytesRead != srcBytesRead)
{
throw new ApplicationException("Error reading Mid file!");
}
dstFile.Write(buffer, 0, srcBytesRead);
}
totalBytes += srcBytesRead;
}
}
}
As you noted, that's going to be pretty slow. You can speed it somewhat by making two threads: one for doing the SRC -> MID copy, and another for doing the MID -> DST copy. It's a little more involved, but not terribly so.
static void FunkyCopy2(string srcFname, string midFname, string dstFname)
{
var cancel = new CancellationTokenSource();
const int bufferSize = 65536;
var finfo = new FileInfo(srcFname);
Console.WriteLine("File length = {0:N0} bytes", finfo.Length);
long bytesCopiedToMid = 0;
AutoResetEvent bytesAvailable = new AutoResetEvent(false);
// First thread copies from src to mid
var midThread = new Thread(() =>
{
Console.WriteLine("midThread started");
using (
FileStream srcFile = new FileStream(srcFname, FileMode.Open, FileAccess.Read, FileShare.None),
midFile = new FileStream(midFname, FileMode.Create, FileAccess.Read,
FileShare.ReadWrite))
{
var buffer = new byte[bufferSize];
while (bytesCopiedToMid < finfo.Length)
{
var srcBytesRead = srcFile.Read(buffer, 0, buffer.Length);
if (srcBytesRead > 0)
{
midFile.Write(buffer, 0, srcBytesRead);
Interlocked.Add(ref bytesCopiedToMid, srcBytesRead);
bytesAvailable.Set();
}
}
}
Console.WriteLine("midThread exit");
});
// Second thread copies from mid to dst
var dstThread = new Thread(() =>
{
Console.WriteLine("dstThread started");
using (
FileStream midFile = new FileStream(midFname, FileMode.Open, FileAccess.Read,
FileShare.ReadWrite),
dstFile = new FileStream(dstFname, FileMode.Create, FileAccess.Write, FileShare.Write)
)
{
long bytesCopiedToDst = 0;
var buffer = new byte[bufferSize];
while (bytesCopiedToDst != finfo.Length)
{
// if we've already copied everything from mid, then wait for more.
if (Interlocked.CompareExchange(ref bytesCopiedToMid, bytesCopiedToDst, bytesCopiedToDst) ==
bytesCopiedToDst)
{
bytesAvailable.WaitOne();
}
var midBytesRead = midFile.Read(buffer, 0, buffer.Length);
if (midBytesRead > 0)
{
dstFile.Write(buffer, 0, midBytesRead);
bytesCopiedToDst += midBytesRead;
Console.WriteLine("{0:N0} bytes copied to destination", bytesCopiedToDst);
}
}
}
Console.WriteLine("dstThread exit");
});
midThread.Start();
dstThread.Start();
midThread.Join();
dstThread.Join();
Console.WriteLine("Done!");
}
That'll speed things up quite a bit because the read and write in the second thread can largely overlap the read and write in the first thread. Most likely, your limiting factor will be the speed of the disk that MID is stored on.
You can get some speed increase by doing asynchronous writes. That is, have the thread read a buffer and then fire off an asynchronous write. While that write is executing, the next buffer is being read. Just remember to wait for the asynchronous write to finish before starting another asynchronous write in that thread. So each thread looks like:
while (bytes left to copy)
Read buffer
wait for previous write to finish
write buffer
end while
I don't know how much of a performance boost that will give you, because you're gated on the concurrent access to the MID file. But it's probably worth the effort to try.
I know that the synchronization code there will prevent the second thread from trying to read when it shouldn't. I think it will prevent a situation in which the second thread locks up because it's waiting for a signal after the first thread has exited. If there is any doubt, you can either have a ManualResetEvent that is used to say that the first thread is done, and use WaitHandle.WaitAny to wait on it and the AutoResetEvent, or you can use a timeout on the WaitOne, like this:
bytesAvailable.WaitOne(1000); // waits a second before trying again
