Node.js stream data from a terminal command to the client

Question

There were already a few questions here about node.js executing commands and outputting the data, but I still can't get this working. What I want is that using node.js I want to execute a python script that runs for a long time and produces some intermediate outputs. I want to stream these outputs to the client as soon as they are produced. I have tried the following, but what I get is that I get the whole output only once the command has finished. How can I make it pass on the data into the socket in real time? Thanks.

function run_cmd(cmd, args) {
  var spawn = require('child_process').spawn,
  child = spawn(cmd, args);
  return child;
}

io.sockets.on('connection', function (socket) {
  var foo = new run_cmd('python', ['test.py']);
  foo.stdout.setEncoding('utf-8');
  foo.stdout.on('data', function(data) {
     console.log('sending data');
     io.sockets.emit('terminal', {output: data});;
  });
);

Is the python script something you have control over? If so, try doing sys.stdout.flush() whenever you have some output to send. See here: http://stackoverflow.com/questions/18849112/stream-child-process-output-in-flowing-mode — Lukas S., Feb 02 '14 at 00:46
I see, that works. Is it possible to launch the process in a way where the flushing is not necessary? I know adding stdio:inherit makes the child process use the stdout of the app, but I couldn't make the app to listen on this. Is that doable? — Laky, Feb 02 '14 at 01:45
Ok, I found a way, running the python with `-u` flag makes it flush after each print. Thanks a lot. — Laky, Feb 02 '14 at 02:12

score 9 · Accepted Answer · edited Feb 02 '14 at 02:19

all your node.js code is okay.your code sends data only once because your code gets data only once.

The point is puts or print commands are not enough to trigger foo.stdout.on try adding '$stdout.flush' at the point you want to send chunk in ruby code. You need to order explicitly to output data.

here is my test code.

js

var spawn = require('child_process').spawn;
var cmd  = spawn('ruby', ['testRuby.rb']);
var counter = 0;
cmd.stdout.on('data', function(data) {
    counter ++;
  console.log('stdout: ' + data);
});

cmd.stderr.on('data', function(data) {
  console.log('stderr: ' + data);
});

cmd.on('exit', function(code) {
  console.log('exit code: ' + code);
  console.log(counter);
});

testRuby.rb

def execute_each_sec(sleep_sec)
  yield
  sleep sleep_sec
end

5.times do
  execute_each_sec(1) do ||
    puts "CurrentTime:#{Time.now}"
    $stdout.flush
  end
end

as you can see I'm calling $stdout.flush to output data explicitly in testRuby.rb. if I remove that,node.js won't get anything until execution of testRuby.rb's finished.

edited lol my bad. I used ruby instead of python. in the case of python, use sys.stdout.flush() like svkk says

Edit: In python you can also use -u flag to force it to flush after each print.

Thanks, I was trying really hard to understand what's wrong with my code, now I get it. I also found out about the possibility of running Python in a way that makes it flush after each write, so now I can achieve exactly what I wanted. — Laky, Feb 02 '14 at 02:17

Node.js stream data from a terminal command to the client

1 Answers1