1

I'm trying to communicate between a c#(5.0) and a python (3.9) application via ZeroMQ. For .Net I'm using NetMQ and for python PyZMQ.

I have no trouble letting two applications communicate, as long as they are in the same language

  • c# app to c# app;
  • python -> python;
  • java -> java,

but trouble starts when I try to connect between different languages.

  • java -> c# and reverse works fine as well [edited]

I do not get any errors, but it does not work either.

I first tried the PUB-SUB Archetype pattern, but as that didn't work, I tried REQ-REP, so some remainders of the "PUB-SUB"-version can still be found in the code.

My Python code looks like this :

def run(monitor: bool):
loop_counter: int = 0

context = zmq.Context()
# socket = context.socket(zmq.PUB)
# socket.bind("tcp://*:5557")
socket = context.socket(zmq.REP)
socket.connect("tcp://localhost:5557")

if monitor:
    print("Connecting")

# 0 = Longest version, 1 = shorter version, 2 = shortest version
length_version: int = 0

print("Ready and waiting for incoming requests ...")

while True:
    message = socket.recv()

    if monitor:
        print("Received message:", message)

    if message == "long":
        length_version = 0
    elif message == "middle":
        length_version = 1
    else:
        length_version = 2

    sys_info = get_system_info(length_version)

    """if not length_version == 2:
        length_version = 2

    loop_counter += 1

    if loop_counter == 15:
        length_version = 1

    if loop_counter > 30:
        loop_counter = 0
        length_version = 0"""

    if monitor:
        print(sys_info)

    json_string = json.dumps(sys_info)
    print(json_string)
    socket.send_string(json_string)

My C# code :

static void Main(string[] args)
    {
        //using (var requestSocket = new RequestSocket(">tcp://localhost:5557"))
        using (var requestSocket = new RequestSocket("tcp://localhost:5557"))    
        {
            while (true) {
                Console.WriteLine($"Running the server ...");
                string msg = "short";
                requestSocket.SendFrame(msg);
                var message = requestSocket.ReceiveFrameString();
                Console.WriteLine($"requestSocket : Received '{message}'");
                //Console.ReadLine();
                Thread.Sleep(1_000);
            }
        }
    }
Oli
  • 9,766
  • 5
  • 25
  • 46
Geertie
  • 237
  • 4
  • 15
  • I have set breakpoints in both programs (with the same port numbers set :)) and it does appear to send data (c#), but the breakpoint in the python code (just under message = socket.recv()) never gets activated. I tried to use requestSocket.SendFrameEmpty(); in C#, just to trigger anything in Python, but without success. – Geertie Dec 14 '20 at 14:31
  • As stated below, this is only a side project and unfortunately my time to try solve this was limited. I have solved it by dumping ZMQ in the python project (not in Java and C# - where everything works just fine) and replacing it with good old TCP and sockets and now it works just fine. I do not know what to do with this question since it is technically speaking not solved, but I will of course no longer put effort into it. I cannot publish my TCP code either as that is not the answer to this specific question. Your input is welcome. – Geertie Dec 15 '20 at 11:01
  • Thank you. Does the request socket receive data as well? – Avv Oct 04 '22 at 23:46
  • 1
    If you want this you can use a 'pulsing system' requesting every x millisecs if data is available from the receiver in case of TCP. But if you want full 'duplex', better techniques like websockets exist. – Geertie Oct 07 '22 at 06:04
  • Thank you. So, this 'pulsing system' exists in PyZMQ in case I have `REQ-REP` pattern? Also what about 'duplex' mode, can I have it in PyZMQ ? – Avv Oct 07 '22 at 13:38
  • Sorry for the late reply. The 'pulsing system' is just something you make yourself like for instance a timer periodically sending a 'request'-message and the other side is 'responding' but the response is more like a request (inversion). This has nothing to do with PyZMQ at all, so I would be surprised if it is in there somewhere. You can create duplex mode by making one channel REQ-REP going from A to B and than the inverse another channel (REQ-REP) going form B to A and creating your own protocol for linking both, but again, you are basically talking about websockets (doing exactly that). – Geertie Oct 21 '22 at 07:49
  • Thank you. So I should create REQ-REP (client side) and REP-REQ (server side) please for duplex system? Also, last thing please, what do you mean by "but the response is more like a request (inversion)"? – Avv Oct 21 '22 at 15:43
  • Point A : Yes. Point B : In normal situation you just send some message (containing data or a command) and the listener just executes or handles that. In the 'pulsing system' (no duplex) you 'ask' the listener if there is some message for the sender and this behavior as such is inverse and the 'response' of the listener becomes the 'request' for the sender. – Geertie Oct 22 '22 at 05:37

3 Answers3

1

Q : "How to set up a ZeroMQ request-reply between a c# and python application"

The problem starts with the missed understanding of how REQ/REP archetype works.

Your code uses a blocking-form of the .recv()-method, so you remain yourselves hanging Out-of-the-Game, forever & unsalvageable, whenever a REQ/REP two-step gets into troubles (as no due care was taken to prevent this infinite live-lock).

Rather start using .poll()-method to start testing a presence / absence of a message in the local AccessNode-side of the queue and this leaves you in a capability to state-fully decide what to do next, if a message is already or is not yet present, so as to keep the mandatory sequence of an API-defined need to "zip" successful chainings of
REQ-side .send()-.recv()-.send()-.recv()-... with
REP-side .recv()-.send()-.recv()-.send()-... calls, are the REQ/REP archetype works as a distributed-Finite-State-Automaton (dFSA), that may easily deadlock itself, due to "remote"-side not being compliant with the local-side expectations.

Having a code, that works in a non-blocking, .poll()-based mode avoids falling into these traps, as you may handle each of these unwanted circumstances while being still in a control of the code-execution paths (which a call to a blocking-mode method in a blind belief it will return at some future point in time, if ever, simply is not capable of).

Q.E.D.


If in doubts, one may use a PUSH/PULL archetype, as the PUB/SUB-archetype may run into problems with non-matching subscriptions ( topic-list management being another, version dependent detail ).

There ought be no other problem for any of the language-bindings, if they passed all the documented ZeroMQ API features without creating any "shortcuts" - some cases were seen, where language-specific binding took "another" direction for PUB/SUB, when sending a pure message, transformed into a multi-part message, putting a topic into a first frame and the message into the other. That is an example of a binding not compatible with the ZeroMQ API, where a cross-language / non-matching binding-version system problems are clear to come.

user3666197
  • 1
  • 6
  • 50
  • 92
  • 1
    Thanks for the clear and understandable explanation ! To be honest I based my python code on an example from pyZMQ. It is at least a little misleading ... I will modify my code tomorrow and I will use the async method in whatever language I use 0MQ from now on ! I will modify the code and update it in this topic if this solves the issue (as I expect - since it was hanging on the .recv method). – Geertie Dec 14 '20 at 21:37
  • I just modified the python code to poll, but without results. I come from a TCP background and I am used to starting 'the server' first, in this case the python code, and that was something that bothered me in the explanation you gave. I should at least get one trigger from the initial startup from the later started c# requester, blocking or non-blocking. Since this is a side project, I can no longer spent time on it and will try to solve it with either tcp or a shared file, whatever works. The most important for me is that for both Java and C# ZMQ does work. Thanks anyways. – Geertie Dec 15 '20 at 08:39
  • Thank you. Does the request socket receive data as well please? – Avv Oct 04 '22 at 23:46
1

Seeing the period of your problems maybe it's because of versions. I run fine a program for long time with communications from Windows/C# with NTMQ 4.0.0.207 239,829 7/1/2019 on one side and Ubuntu/Python with zeromq=4.3.1 and pyzmq=18.1.0. I just tried updating to use same NETMQ version but with new versions zeromq=4.3.3 and pyzmq=20.0.0 but there is a problem/bug somewhere and it doesn't run well anymore.

So your code doesn't look bad may be it's software versions issues not doing well try with NTMQ 4.0.0.207 on c# side and zeromq=4.3.1 with pyzmq=18.1.0 on python side

totoleroi
  • 26
  • 1
  • Thanks totoleroi ! As I said before, I no longer use zeromq in the communication and to be honest I deleted that project so I don't know the version. However, I will mark your answer as the answer to my question as you have two working programs using the settings you provide and that is way more than I had. – Geertie Jan 16 '21 at 08:42
0

I might be late, but this same thing happened to me. I have a python Subscriber using pyzmq and a C# Publisher using NetMQ.

After a few hours, it occurred to me that I needed to let the Publisher some time to connect. So a simple System.Threading.Thread.Sleep(500); after the Connect/Bind did the trick.

Oggie
  • 1