Both get STDOUT in realtime and provide STDIN

Question

I have a command that requires I give it some STDIN data, as in my-command <<< my-data. I don’t have control over the command; the info is meant to be given interactively and <<< works.

I want to automate this command as part of a larger script, but since its action takes a while — but outputs progress to STDOUT — I want to print STDOUT in real time. I also want to be able to capture the exit status of the command, to determine if it failed or not.

If I use system, I get the STDOUT as it happens, but can’t provide STDIN data.

system('my-command')

If I use Open3, I can provide STDIN data but STDOUT is only printed at the end (if I capture it at all).

Open3.capture2('my-command', stdin_data: 'my-data')[1].success?

Any way I can get the best of both worlds, preferably with Open3?

Answer 1

Here's a snippet for realtime stdout with optional stdin data. You need to use IO.select to peek into the streams to see if they're readable.

require 'open3'

class Runner
  class Result
    attr_accessor :status
    attr_reader :stdout, :stderr

    def initialize
      @stdout = +''
      @stderr = +''
      @status = -127
    end

    def success?
      status.zero?
    end
  end

  attr_reader :result

  def initialize(cmd, stdin: nil, print_to: nil)
    @stdin = stdin
    @cmd = cmd
    @result = Result.new
    @print_to = print_to
  end

  def run
    Open3.popen3(@cmd) do |stdin, stdout, stderr, wait_thr|
      # Dump the stdin-data into the command's stdin:
      unless stdin.closed?
        stdin.write(@stdin) if @stdin
        stdin.close
      end

      until [stdout, stderr].all?(&:eof?)
        readable = IO.select([stdout, stderr])
        next unless readable&.first

        readable.first.each do |stream|
          data = +''
          begin
            stream.read_nonblock(1024, data)
          rescue EOFError
            # ignore, it's expected for read_nonblock to raise EOFError
            # when all is read
          end

          next if data.empty?

          if stream == stdout
            result.stdout << data
            @print_to << data if @print_to
          else
            result.stderr << data
            @print_to << data if @print_to
          end
        end
      end

      result.status = wait_thr.value.exitstatus
    end
    result
  end
end


result = Runner.new('ls -al').run
puts "Exit status: %d" % result.status
puts "Stdout:"
puts result.stdout
puts "Stderr:"
puts result.stderr

# Print as it goes:
result = Runner.new('ls -al', print_to: $stdout).run

If you need to simulate real time stdin (keypresses), then you need to create some sort of matcher for the stdout data stream and write the responses to the command's stdin when the expected prompt comes through the stream. In that case you are probably better off with PTY.spawn or using a 3rd party gem.

You could also write the stdin data to a tempfile and use the shell's own redirection:

require 'tempfile'
tempfile = Tempfile.new
tempfile.write(stdin_data)
tempfile.close
system(%(do_stuff < "#{tempfile.path}"))
tempfile.unlink