I am creating a file using a cygwin bash script on remote:
(echo $$ > tail.pid; exec tail -f -n+0 --retry FIX.log) | gzip -c >> /faraway/log.gz
At the same time, on my local machine, I am reading /faraway/log.gz in c# using:
txtFileReader = new StreamReader(new GZipStream(new FileStream(@"log.gz"), FileMode.Open, FileAccess.Read, FileShare.ReadWrite), CompressionMode.Decompress, true));
bool isDone = false; // Set to true when "done.txt" file appears to signal log.gz is completed
while (true) // Read until finished
{
if (!isDone)
{
if (txtFileReader.Peek() == -1) // Wait if file is still being transferred
{
while (!File.Exists(@"done.txt"))
{
Thread.Sleep(10000);
}
isDone = true;
// Goes wrong here when gzip closes on remote
}
}
if ((line = txtFileReader.ReadLine()) == null)
{
return "";
}
}
I have indicated the point in the code where I think it goes wrong.
When (Peek() == -1) and gzip is still running (i.e. it is just slow to transfer data) the next ReadLine works correctly, BUT when gzip is closed on remote, my next ReadLine does not return a correct line - it returns a random part of a line.
Later, when I zcat the file, I see the correct lines were infact sent. So it appears that I am losing the "position" in the gzipstream when gzip shuts down.
What happens when Gzip closes? Does it affect anything previously written?
