12

I spawn a child process like this:

var child = require('child_process');
var proc = child.spawn('python', ['my_script.py', '-p', 'example']);

I also set some data handling:

proc.stdin.setEncoding('utf8');
proc.stdout.setEncoding('utf8');
proc.stderr.setEncoding('utf8');

proc.stdout.on('data', function (data) {
  console.log('out: ' + data);
});

proc.stderr.on('data', function (data) {
  console.log('err: ' + data);
});

proc.on('close', function (code) {
  console.log('subprocess exited with status ' + code);
  proc.stdin.end();
});

My Python script reads lines from stdin and for each line does some operations and prints to stdout. It works fine in the shell (I write a line and I get the output immediately) but when I do this in Node:

for (var i = 0; i < 10; i++) {
  proc.stdin.write('THIS IS A TEST\n');
}

I get nothing.

I got to fix it calling proc.stdin.end() but that also terminates the child process (which I want to stay in background, streaming data).

I also triggered a flush filling the buffer with lots of writes, but that's not really an option.

Is there any way to manually flush the stream?

kaoD
  • 1,534
  • 14
  • 25
  • If `write()` returns true, then it has been flushed. If it returns false, you can listen for the `'drain'` event to verify when the write buffer has been completely flushed. You're saying you can write data, and it never gets flushed at all? I would suspect the child process is chunking the data and looking for something other than `\n` to breakup the input. When you overflow its input buffer, or end the stream, the child is forced to process the data at that time. – Bret Copeland Jun 17 '13 at 05:14
  • @BretCopeland yup, `write()` returns `true`. I thought maybe it wasn't the write being queued... but perhaps the read? It's weird, but I'm sure the children is not chunking the data since it works in the shell, and I can verify it's looking for '\n'. Running the script and just writing lines and pressing enter works as expected (data is written to stdout as enter is pressed and the process doesn't exit). I'm really stuck here. – kaoD Jun 17 '13 at 05:30
  • 1
    Same problem here buddy. spawned grep never outputs. – kilianc Sep 16 '13 at 17:02
  • @kilianc sorry to hear that. I ended up using Python against my wishes. It did the job anyways. Perhaps it's time to file a Node bug report. – kaoD Sep 16 '13 at 20:37
  • @kaoD the problem in my case was `grep`. The guy wants a `--line-buffered` option otherwise it will buffer ~3k lines before outputting something. – kilianc Sep 16 '13 at 23:12

1 Answers1

4

You are not flushing the output from Python after print statement. I had similar problem and @Alfe answered my question. Take a look at this:

Stream child process output in flowing mode

Community
  • 1
  • 1
iFadey
  • 320
  • 3
  • 11
  • Unfortunately I discarded the code so I can't check if it solves the problem. Thanks for the answer! – kaoD Oct 04 '13 at 00:02