Ruby - Open3 not finishing subprocess

Question

I'm using:

- Ruby 1.9.3-p448
- Windows Server 2008

I have a file that contains commands that is used by a program, I'm using it in this way

C:\> PATH_TO_FOLDER/program.exe file.txt

File.txt have some commands so program.exe will do the following:

- Execute commands 
- Reads from a DB using an ODBC method used by program
- Outputs result in a txt file

Using powershell this command works fine and as expected.

Now I have this in a file (app.rb):

require 'sinatra'
require 'open3'

get '/process' do
  program_path = "path to program.exe"
  file_name = "file.txt"
  Open3.popen3(program_path, file_name) do |i, o, e, w|
    # I have some commands here to execute but just as an example I'm using o.read
    puts o.read
  end
end

Now when using this by accessing http://localhost/process, Open3 works by doing this (I'm not 100% sure but after trying several times I think is the only option)

• Reads commands and executes them (this is ok)

• Tries to read from DB by using ODBC method (Here is my problem. I need to receive some output from Open3 so I can show it in a browser, but I guess when it tries to read it starts another process that Open3 is not aware of, so Open3 goes on and finish without waiting for it)

• Exits

Exits

I've found about following:

• Use Thread.join (in this case, w.join) in order to wait for process to finish, but it doesn't work
• Open4 seems to handle child process but doesn't work on Windows
• Process.wait(pid), in this case pid = w.pid, but also doesn't work
• Timeout.timeout(n), the problem here is that I'm not sure how long will it take.

Is there any way of handling this? (waiting for Open3 subprocess so I get proper output).

Answer 1

We had a similar problem getting the exit status and this is what we did

Open3.popen3(*cmd) do |stdin, stdout, stderr, wait_thr|

  # print stdout and stderr as it comes in
  threads = [stdout, stderr].collect do |output|
    Thread.new do
      while ((line = output.gets rescue '') != nil) do
        unless line.blank?
          puts line
        end
      end
    end
  end

  # get exit code as a Process::Status object
  process_status = wait_thr.value #.exitstatus

  # wait for logging threads to finish before continuing
  # so we don't lose any logging output
  threads.each(&:join)

  # wait up to 5 minutes to make sure the process has really exited
  Timeout::timeout(300) do
    while !process_status.exited?
      sleep(1)
    end
  end rescue nil

  process_status.exitstatus.to_i
end

Answer 2

Using Open3.popen3 is easy only for trivial cases. I do not know the real code for handling the input, output and error channels of your subprocess. Neither do I know the exact behaviour of your subprocesses: Does it write on stdout? Does it write on stderr? Does it try to read from stdin?

This is why I assume that there are problems in the code that you replaced by puts o.read. A good summary about the problems you can run into is on http://coldattic.info/shvedsky/pro/blogs/a-foo-walks-into-a-bar/posts/63.

Though I disagree with the author of the article, Pavel Shved, when it comes to finding a solution. He recommends his own solution. I just use one of the wrapper functions for popen3 in my projects: Open3.capture*. They do all the difficult things like waiting for stdout and stderr at the same time.