How can I communicate between running python code and nodejs

Question

I'd like to have some python code running and communicating with a nodejs express server. So far, I can get my nodejs server to call python functions via one of two mechanisms, either to spawn a python task or to have it talk to a zerorpc python server.

For the first, a la http://www.sohamkamani.com/blog/2015/08/21/python-nodejs-comm/, this works:

var express = require( "express" );
var http = require( "http" );
var app = express();
var server = http.createServer( app ).listen( 3000 );
var io = require( "socket.io" )( server );

app.use( express.static( "./public" ) );

io.on( "connection", function( socket ) {

    // Repeat interval is in milliseconds
    setInterval( function() {

        var spawn = require( 'child_process' ).spawn,
        py    = spawn( 'python', [ 'mytime.py' ] ),
        message = '';

        py.stdout.on( 'data', function( data ) {
            message += data.toString();
        });

        py.stdout.on( 'end', function() {
            socket.emit( "message", message );
        });

    }, 50 );
});

Where mytime.py is

from datetime import datetime
import sys

def main():
    now = datetime.now()
    sys.stdout.write( now.strftime( "%-d %b %Y %H:%M:%S.%f" ) )

And with zerorpc http://www.zerorpc.io/, if this python code is running:

from datetime import datetime
import sys
import zerorpc

class MyTime( object ):
    def gettime( self ):
        now = datetime.now()
        return now.strftime( "%-d %b %Y %H:%M:%S.%f" )

s = zerorpc.Server( MyTime() )
s.bind( "tcp://0.0.0.0:4242" )
s.run()

This nodejs code works:

var express = require( "express" );
var http = require( "http" );
var app = express();
var server = http.createServer( app ).listen( 3000 );
var io = require( "socket.io" )( server );
var zerorpc = require( "zerorpc" );
var client = new zerorpc.Client();
client.connect( "tcp://127.0.0.1:4242" );

app.use( express.static( "./public" ) );

io.on( "connection", function( socket ) {

    // Repeat interval is in milliseconds
    setInterval( function() {

        client.invoke( "gettime", function( error, res, more ) {
            socket.emit( "message", res.toString( 'utf8' ) );
        } );

    }, 50 );
});

But what I'd like to be able to do is instead of just having python functions called, I'd like a separate python process running and sending messages to the nodejs server which listens for them and then handles them. I've experimented with middleware socketio-wildcard, but if I try to set up a python server with zerorpc on the same port as the nodejs express server, it gives a zmq.error.ZMQError: Address already in use error.

I know that I'm not thinking about this right--I know that I'm missing some logic around interprocess communication due to my naïveté here--so if there is a better way to do message sending from a python process with a nodejs server listening, I'm all ears.

Any ideas?

Many thanks in advance!

Why not build something similar in nature to how you built your node app? You ultimately have set up your node app to have some "endpoint" a client can hit. So, do the same in Python. Check out something like [Flask](http://flask.pocoo.org/). It's very well documented, and guides you well to bring up the application as a listening service. — idjaw, Jul 21 '17 at 21:52
Which, also probably means you don't need a messaging queue like ZeroMQ. Maybe you *might* need it for the type of data you are sending around. But for simple communication between these two services, I think setting up a simple app with Flask, will do just fine. — idjaw, Jul 21 '17 at 21:53
I'm not *too* sure what the usage of `var io = require( "socket.io" )( server );` is here either. If you are simply looking to run your node app, and provide the ability for other clients to hit a particular endpoint on your service running node, I don't *think* you need to do that? I think there is a more idiomatic way to get a node app running. But, don't hold me to that one. I just don't remember doing *that* with some of the node stuff I've dealt with. — idjaw, Jul 21 '17 at 21:56
The one thing about Flask that makes me a little concerned is that it doesn't look like it's asynchronous. — Dribbler, Jul 21 '17 at 22:01
Ah. So this is where it now starts getting in to the domain of determining what the app actually has to handle. If you absolutely do need asynchronous behaviour on the Python side, there are several options out there. However, not so much "out-of-the-box" where there is the *exact* equivalent of Node. There is piecing together on the Python side to get async behaviour. — idjaw, Jul 21 '17 at 22:08
So, the language itself supports async as you can see [here](https://docs.python.org/3/library/asyncio-task.html). Furthermore, in order to integrate async behaviour with web services, there have been implementations made to facilitate all this. This answer [here](https://stackoverflow.com/questions/31866796/making-an-asynchronous-task-in-flask) provides a simple example of one of the tools used. Furthermore, take a look [here](http://python-guide-pt-br.readthedocs.io/en/latest/scenarios/web/) for a nice breakdown of web frameworks in Python land. There is a mention of Tornado for async. — idjaw, Jul 21 '17 at 22:10
I hope this information helps and isn't further complicating matters. — idjaw, Jul 21 '17 at 22:13

score 17 · Accepted Answer · answered Jul 23 '17 at 20:54

For those trying to figure this out, here's a solution thanks to Zeke Alexandre Nierenberg

For the node.js server code:

var express = require( "express" );
var app = express();
var http = require( "http" );
app.use( express.static( "./public" ) ); // where the web page code goes
var http_server = http.createServer( app ).listen( 3000 );
var http_io = require( "socket.io" )( http_server );

http_io.on( "connection", function( httpsocket ) {
    httpsocket.on( 'python-message', function( fromPython ) {
        httpsocket.broadcast.emit( 'message', fromPython );
    });
});

and the python code that sends it messages:

from datetime import datetime
from socketIO_client import SocketIO, LoggingNamespace
import sys

while True:
    with SocketIO( 'localhost', 3000, LoggingNamespace ) as socketIO:
        now = datetime.now()
        socketIO.emit( 'python-message', now.strftime( "%-d %b %Y %H:%M:%S.%f" ) )
        socketIO.wait( seconds=1 )

Voilà!

Worth noting socket.io 1.7.4 is probably required for node. – Eddie Sep 28 '18 at 04:37 — Eddie, Sep 28 '18 at 04:37

score 6 · Answer 2 · answered May 18 '18 at 09:42

I had some problems with socketIO version...

so, this is my Solution:

NodeJS:

   var app = require("express")();
   var http = require('http').Server(app);
   var bodyParser = require('body-parser');

    app.use(bodyParser.json())
    app.post('/',function(req,res){
            var msg=req.body.msg;
            console.log("python: " + msg);
    });

     http.listen(3000, function(){
     console.log('listening...');
     });

on Python:

  import requests
  import json

  url = "http://localhost:3000"
  data = {'msg': 'Hi!!!'}
  headers = {'Content-type': 'application/json', 'Accept': 'text/plain'}
  r = requests.post(url, data=json.dumps(data), headers=headers)

How can I communicate between running python code and nodejs

2 Answers2