Capturing standard out from tail -f "follow"

Question

I am trying to capture the output from tail in follow mode, where it outputs the text as it detects changes in the file length - particularly useful for following log files as lines are added. For some reason, my call to StandardOutput.Read() is blocking until tail.exe exits completely.

Relevant code sample:

var p = new Process() {
  StartInfo = new ProcessStartInfo("tail.exe") {
    UseShellExecute = false,
    RedirectStandardOutput = true,
    Arguments = "-f c:\\test.log"
  }
};
p.Start();

// the following thread blocks until the process exits
Task.Factory.StartNew(() => p.StandardOutput.Read());
// main thread wait until child process exits
p.WaitForExit();

I have also tried using the support for the OutputDataReceived event handler which exhibits the same blocking behavior:

p.OutputDataReceived += (proc, data) => {
  if (data != null && data.Data != null) {
    Console.WriteLine(data.Data);
  }
};
p.BeginOutputReadLine();

I do have a little bit more code around the call to StandardOutput.Read(), but this simplifies the example and still exhibits the undesirable blocking behavior. Is there something else I can do to allow my code to react to the availability of data in the StandardOutput stream prior to the child application exiting?

Is this just perhaps a quirk of how tail.exe runs? I am using version 2.0 compiled as part of the UnxUtils package.

Update: this does appear to be at least partially related to quirks in tail.exe. I grabbed the binary from the GnuWin32 project as part of the CoreUtils package and the version bumped up to 5.3.0. If I use the -f option to follow without retries, I get the dreaded "bad file descriptor" issue on STDERR (easy to ignore) and the process terminates immediately. If I use the -F option to include retries it seems to work properly after the bad file descriptor message has come by and it attempts to open the file a second time.

Is there perhaps a more recent win32 build from the coreutils git repository I could try?

"tail -f" simply checks the file every x time units and, if it's grown, reads the new stuff. I'm don't know c#, but i imagine it would be easy to implement there. You may be able to get inspiration from the source code for an open source version of tail (like GNU). It may be easier to implement the functionality you need in c# than to solve this problem. — theglauber, Jan 20 '12 at 18:40
Have you considered just using a FileSystemWatcher or Timer and storing a stream position for quick seeks using a StreamReader instead of launching another executable? — JamieSee, Jan 27 '12 at 23:37

kmp · Accepted Answer · 2012-02-09T08:51:38.330

I know it is not exatly what you are asking but as James says in the comments, you could do the equivalent functionality directly in c# to save you having to launch another process.

One way you can do it is like this:

using System;
using System.IO;
using System.Text;
using System.Threading;

public class FollowingTail : IDisposable
{
    private readonly Stream _fileStream;
    private readonly Timer _timer;

    public FollowingTail(FileInfo file,
                         Encoding encoding,
                         Action<string> fileChanged)
    {

        _fileStream = new FileStream(file.FullName,
                                     FileMode.Open,
                                     FileAccess.Read,
                                     FileShare.ReadWrite);

        _timer = new Timer(o => CheckForUpdate(encoding, fileChanged),
                           null,
                           0,
                           500);
    }

    private void CheckForUpdate(Encoding encoding,
                                Action<string> fileChanged)
    {
        // Read the tail of the file off
        var tail = new StringBuilder();
        int read;
        var b = new byte[1024];
        while ((read = _fileStream.Read(b, 0, b.Length)) > 0)
        {
            tail.Append(encoding.GetString(b, 0, read));
        }

        // If we have anything notify the fileChanged callback
        // If we do not, make sure we are at the end
        if (tail.Length > 0)
        {
            fileChanged(tail.ToString());
        }
        else
        {
            _fileStream.Seek(0, SeekOrigin.End);
        }
    }

    // Not the best implementation if IDisposable but you get the idea
    // See http://msdn.microsoft.com/en-us/library/ms244737(v=vs.80).aspx
    // for how to do it properly
    public void Dispose()
    {
        _timer.Dispose();
        _fileStream.Dispose();
    }
}

Then to call for example:

new FollowingTail(new FileInfo(@"C:\test.log"),
                  Encoding.ASCII,
                  s =>
                  {
                      // Do something with the new stuff here, e.g. print it
                      Console.Write(s);
                  });

Thanks for the answer - your code is quite good. I actually tried something similar but this doesn't work for files that are currently in use. tail.exe will still read locked files while FileStream won't - at least not with options I have been able to determine. — Goyuix, Feb 08 '12 at 18:11
I think I just worked out the locking problem - you need to specify FileShare.ReadWrite which means allow other processes readwrite access (http://msdn.microsoft.com/en-us/library/system.io.fileshare.aspx). I have updated my answer with this change plus I noticed some oddities using the FileSystemWatcher so just chucked in a timer so it will check the file every 500ms and report anything new. — kmp, Feb 09 '12 at 08:51

Capturing standard out from tail -f "follow"

1 Answers1