ZeroMQ: how to keep a connection of PAIR/PAIR between python and node?

Question

I have a python server and a node.js client. The problem is that at times when I restart my node.js script it does not reconnect.

Also, what I don't understand is why is there an infinite loop on python. Can't I use events to listen for connections.

Below is the server and client.

---

context = zmq.Context()
socket = context.socket(zmq.PAIR)
print("Binding socket to port: " + str(port))
socket.bind("tcp://*:%s" % port)
print("Connection has been established")
while True: // cant we listen for events here i.e on connection etc
    msg = socket.recv()
    print(msg)
    socket.send("Server response")

---

'use strict';
var globevt  = require('../lib/emitter');
var sprintf  = require("sprintf-js").sprintf;
var logger   = require('../lib/logat');
var zmq      = require('zeromq')
    , socket = zmq.socket('pair');
const fs     = require('fs');

socket.connect('tcp://127.0.0.1:3000');
socket.on('message', function (msg) {
    console.log(msg);
});

Also this only works if the server starts first then the client. How can I make the client to re-try until it connects?

Answer 1

From the zmq guide, pair sockets are intended for inter thread communication in the same process. The intended use is to have the parent thread bind, then pass the socket context to the child thread, and have that connect to the parent thread socket as a pair/pair combo

Exclusive pair, which connects two sockets exclusively. This is a pattern for connecting two threads in a process, not to be confused with “normal” pairs of sockets.

For inter process communication, you should use req/rep, pub/sub or some other pattern. If you need simple reliable interprocess (there are very many levels of reliability), where the client can handle the server going away, have a look at the lazy pirate pattern

Answer 2

Q : "... at times when I restart my node.js script it does not reconnect."

Well, there are a few tricks to somehow cure this, yet it is fair to note, that both the ZeroMQ version & the actual 3rd-party language wrapper/binding version (and a level of strictness of adhering to the then valid API documented properties) matters...

socket( zmq.PAIR )-instances, as documented in v4.3.2 API, have two main differences in how these behave in production:

a) A PAIR-socket archetype, after going .bind()-into-RTO-state and having there accepted an incoming peer, who did succesfully .connect(...), will never accept "another" .connect() coming

b) A PAIR-socket archetype do not operate, on other archetypes so common, low level transport-class connection-management services, so do not expect any silently automated "auto-reconnection" or smart receiving new, being _...almost... "free" again, .connect()-connection during internally not completed handling of the "old" peer-interconnection socket remaining still in a closing state of the tcp-transport-class.

Still, we can create our own, smart meta-layer for handling any of these kinds of behaviour, atop the built-in archetype primitives. If in doubts, do remember the Zen-of-Zero principles, so well crafted into the ZeroMQ. Further inspiration might be found here and for getting a wider, if not full context, one may try this.

Let me present a few touches that may improve an immunity in this direction:

1. always keep explicitly setting the .setsockopt( zmq.LINGER, 0 ) that avoids "witing till ever" for a dead-peer on an already disconnected interconnect...

2. prefer to set .setsockopt( zmq.IMMEDIATE ) which prevents from buffering any messages for a "later" delivery on not transmission-ready interconnections...

3. always design all the code to keep using non-blocking forms of all the .recv()-methods, indeed always - your code will un-salvageable block forever on the line msg = socket.recv(), where it will remain for the rest of the time in cases the remote peer did not send anything yet or will never send any next message for whatever the reason is. Always use .recv( zmq.NOBLOCK )
   and
   either do a --- pre-poll the POSACK'd presence of anything ready
   to next perform the .recv( zmq.NOBLOCK )
   using .poll( someContextSpecificFeasibleWaitingTime )
   or do a --- post-.recv( zmq.NOBLOCK ) handling for all cases there was indeed nothing in to actually expect from .recv( zmq.NOBLOCK ) and so you indeed do get a blank empty nothing back from calling the .recv( zmq.NOBLOCK )-method

---

Q : "Also, what I don't understand is why is there an infinite loop on python. Can't I use events to listen for connections."

Well, the loop is simpler and principally avoids "clashes" between multiple event-loop frameworks ( so common in Tkinter's cases, where mainloop() typically evicts all other event-handlers and "co-operating" event-handling frameworks are almost an oxymoron ).

Loop is simple, pure-[SERIAL] code-execution structure ( also remember the GIL-lock chopping the actual flow of code-execution-time ) and is safe.