C# StreamReader.ReadLine returning null before end of stream

Question

I am using the SSH.NET library to implement a file system watcher on a remote linux server using the inotifywait command. Essentially it is a wrapper around:

ssh myhost "inotifywait -m -e close_write --format '%:e %f' /dropzone"

That command will print out (to STDOUT):

CLOSE_WRITE:CLOSE foo
CLOSE_WRITE:CLOSE bar
CLOSE_WRITE:CLOSE baz

Which is simple enough to parse and turn into events. Anyway, my c# code is essentially:

        var privateKeyFile = new PrivateKeyFile(identity);
        var client = new SshClient(hostname, username, privateKeyFile);

        SshCommand command = null;
        IAsyncResult result = null;
        try
        {
            client.Connect();
            command = client.CreateCommand("inotifywait -m -e close_write --format '%:e %f' " + dropZone);
            result = command.BeginExecute();

            Console.WriteLine("Watching for events");
            var reader = new StreamReader(command.OutputStream);
            string line = null;
            while ((line = reader.ReadLine()) != null)
            {
                Console.WriteLine(line);
            }
            Console.WriteLine("Reached end of stream");
        }
        finally
        {
            if (client != null)
            {
                Console.WriteLine("Close the connection");
                client.Dispose();
            }
        }

        Console.WriteLine("Press enter to quit");
        Console.ReadLine();

And running it results in this output after a single file is written:

Watching for events
CLOSE_WRITE:CLOSE baz
Reached end of stream
Close the connection
Press enter to quit

The Watching for events shows up immediately and will wait for the first file to be written (blocking wait as I would expect from StreamReader). However, the the next ReadLine, rather than another blocking wait, returns null (which indicates end of stream), even though the command is still happily running. I know that I can change the loop thusly:

            while (!result.IsCompleted)
            {
                line = reader.ReadLine();
                if (line != null)
                {
                    Console.WriteLine(line);
                }
            }

Which results in:

Watching for events
CLOSE_WRITE:CLOSE baz
CLOSE_WRITE:CLOSE bar
CLOSE_WRITE:CLOSE foo
...

As desired, but it gets rid of the blocking wait for new input which means the loop is constantly spinning (obviously undesired...)

Can you explain this behavior? Any suggestions for another approach?

---- UPDATE ----

It looks like the library is in the process of being migrated to github, and updated. I have submitted this issue in an attempt to resolve this problem.

Answer 1

The reason of observed behavior is PipeStream class. It works like bytes queue. When you read bytes from PipeStream, you are actually dequeue them, so stream length decreases. When you read all bytes, stream length becomes 0. That means after you read first "line" (that can be multiple lines really, just first portion of data) - stream has length 0 and so is effectively ended. Next reads will just return without blocking until next portion of data arrives (if any).

Unfortunately it seems those streams were not designed to work in your case - they are designed to execute command, receive one result and done. If you want to read continuous stream of data (like your case or for example "tail -f" results - your only option it seems is to fall back to Thread.Sleep between reads, at least after quick search I didn't found any alternative.

Update: still with some reflection you can achieve the result you want. Undelying channel has DataReceived event which you can use to get notified whenever new data is available. The code below should do the trick (note this is a sketch so take care):

    static void Main(string[] args) {
        var privateKeyFile = new PrivateKeyFile(@"somefile");
        using (var client = new SshClient("somehost", "someuser", privateKeyFile)) {                
            client.Connect();
            var command = client.CreateCommand("tail -f /tmp/test.txt");

            var result = command.BeginExecute();
            var channelField = command.GetType().GetField("_channel", BindingFlags.Instance | BindingFlags.NonPublic);
            var channel = channelField.GetValue(command);
            var receivedEvent = channel.GetType().GetEvent("DataReceived", BindingFlags.Instance | BindingFlags.Public);
            Console.WriteLine("Watching for events");
            using (var handler = new ReceivedHandler()) {
                // add event handler here
                receivedEvent.AddEventHandler(channel, Delegate.CreateDelegate(receivedEvent.EventHandlerType, handler, handler.GetType().GetMethod("OnReceive")));
                while (true) {
                    // wait on both command completion and our custom wait handle. This is blocking call
                    WaitHandle.WaitAny(new[] {result.AsyncWaitHandle, handler.Signal});
                    // if done - break
                    if (result.IsCompleted)
                        break;
                    var line = handler.ReadLine();
                    Console.WriteLine(line);
                }
            }                                
            Console.WriteLine("Reached end of stream");                
            Console.ReadKey();
        }

    }

    public class ReceivedHandler : IDisposable {
        private readonly AutoResetEvent _signal;
        private readonly StringBuilder _buffer = new StringBuilder();
        public ReceivedHandler() {
            _signal = new AutoResetEvent(false);
        }

        public void OnReceive(object sender, EventArgs e) {
            var dataProp = e.GetType().GetProperty("Data", BindingFlags.Instance | BindingFlags.Public);
            var rawData = (byte[])dataProp.GetValue(e);
            var data = Encoding.ASCII.GetString(rawData);
            lock (_buffer) {
                // append to buffer for reader to consume
                _buffer.Append(data);
            }
            // notify reader
            Signal.Set();
        }

        public AutoResetEvent Signal => _signal;

        public string ReadLine() {
            lock (_buffer) {
                // cleanup buffer
                var result = _buffer.ToString();
                _buffer.Clear();
                return result;
            }
        }

        public void Dispose() {
            _signal.Dispose();
        }
    }

Of course it's always better if you contact developer of this library and explain the issue, maybe they will be able to add missing behavior.

Answer 2

@Evk's answer is right that PipeStream is the main culprit. Another problem with PipeStream is that if you attempt to read more than the available bytes, it will block. For performance reasons, blocking should be the job of the consumer of the PipeStream. I use SSH.NET to execute SshCommand and asynchronously read the standard output/error. My workaround for the problems is to write to an intermediary MemoryStream and then use standard mechanisms like StreamReader. This is a more generalized answer to read from the PipeStream:

public class SshCommandStreamReader : IDisposable
{
    private readonly Stream stream;
    private readonly MemoryStream intermediateStream;
    private readonly StreamReader reader;

    public SshCommandOutputReader(Stream stream)
    {
        this.stream = stream;
        this.intermediateStream = new MemoryStream();
        this.reader = new StreamReader(intermediateStream, Encoding.UTF8);
    }

    private int FlushToIntermediateStream()
    {
        var length = stream.Length;

        if (length == 0)
        {
            return 0;
        }

        // IMPORTANT: Do NOT read with a count higher than the stream length (which is typical of reading
        // from streams). The streams for SshCommand are implemented by PipeStream (an internal class to
        // SSH.NET). Reading more than the current length causes it to *block* until data is available.
        // If the stream is flushed when reading, it does not block. It is not reliable to flush and then
        // read because there is a possible race condition where a write might occur between flushing and
        // reading (writing resets the flag that it was flushed). The only reliable solution to prevent
        // blocking when reading is to always read the current length rather than an arbitrary buffer size.
        var intermediateOutputBuffer = new byte[length];
        var bytesRead = stream.Read(intermediateOutputBuffer, 0, intermediateOutputBuffer.Length);
        intermediateStream.Write(intermediateOutputBuffer, 0, bytesRead);
        return bytesRead;
    }

    public string Read()
    {
        var bytesFlushed = FlushToIntermediateStream();

        // Allow reading the newly flushed bytes.
        intermediateStream.Position -= bytesFlushed;

        // Minor optimization since this may be called in a tight loop.
        if (intermediateStream.Position == intermediateStream.Length)
        {
            return null;
        }
        else
        {
            var result = reader.ReadToEnd();
            return result;
        }
    }

    public void Dispose()
    {
        reader.Dispose();
        intermediateStream.Dispose();
    }
}

And then use it:

using (var command = client.CreateCommand("your command text"))
{
    var cmdAsyncResult = command.BeginExecute();

    using (var standardOutputReader = new SshCommandStreamReader(command.OutputStream))
    {
        while (!cmdAsyncResult.IsCompleted)
        {
            var result = standardOutputReader.Read();
            if (!String.IsNullOrEmpty(result))
            {
                Console.Write(result);
            }

            // Or what ever mechanism you'd like to use to prevent CPU thrashing.
            Thread.Sleep(1);
        }

        // This must be done *after* the loop and *before* EndExecute() so that any extra output
        // is captured (e.g. the loop never ran because the command was so fast).
        var resultFinal = standardOutputReader.Read();
        if (!String.IsNullOrEmpty(resultFinal))
        {
            Console.Write(resultFinal);
        }
    }

    command.EndExecute(cmdAsyncResult);
}

You should be able to modify this sample to read from standard error (via the ExtendedOutputStream) and also change it to read line-by-line which is specific to your application.