3

I'm attempting to use libzmq (4.2.2) via czmqz (4.0.2) to establish an IPC communication channel between a Python program and my C++ application.

I'm using the ZMQ publisher/subscriber pattern, where the Python program binds to a port and publishes messages using pyzmq (16.0.2), and my C++ program uses the zsock class from CZMQ to connect as a subscriber.

The problem is, whenever my subscriber attempts to connect, I get back error code 11, Resource temporarily unavailable. Strangely, this system seems to work just fine on my development machine, but not on the target system I'm deploying to.

The problem occurs on the very first line of initializing the socket in the following abridged method:

bool ZmqSocketWrapper::connectSubscriber(string address)
{
    m_address = address;
    m_pSocket = zsock_new_sub(address.c_str(), "");

    int errorCode = zmq_errno();
    if (errorCode != 0)
    {
        printf(zmq_strerror(errorCode));
        return false;
    }

    return true;
}

This is called as follows:

m_subscriberSocket->connectSubscriber("tcp://127.0.0.1:5555");

And I've also tried other variations, with the same result:

m_subscriberSocket->connectSubscriber("tcp://localhost:23232");
m_subscriberSocket->connectSubscriber("ipc:///tmp/test");

When searching online, it appears that most other people have this problem when attempting to send/receive, so it seems odd that I'm having this when just attempting to open the socket.

A few other details:

  • My ZMQ publisher is written in Python using pyzmq and is working fine on the same target system, which suggests that the problem is in czmq.
  • The machine where I'm seeing the problem is a Raspberry Pi, in case that's relevant, although bear in mind the point above.
  • No, nothing else is using that port, and I've confirmed using netstat that the server port is listening.
  • Yes, I've tried running my client as root.

Any help much appreciated!

Benp44
  • 1,548
  • 1
  • 10
  • 14
  • 1
    I suggest you run your app in strace and see what exactly is failing there. It might give you some hints. – SergeyA Aug 07 '17 at 18:15
  • Thanks @SergeyA. I tried using `strace -e trace=network` on both the Python program and the C++ program, and made them both do the same thing (just bind a socket to tcp://*:5555). The only difference in the strace output is the file descriptor ID that is allocated. In C++ I _always_ get `socket(PF_INET, SOCK_STREAM|SOCK_CLOEXEC, IPPROTO_TCP) = 27` whereas in Python I _always_ get `socket(PF_INET, SOCK_STREAM|SOCK_CLOEXEC, IPPROTO_TCP) = 10`. – Benp44 Aug 08 '17 at 15:34
  • In both cases, the following `strace` lines are `setsockopt(10, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0; bind(10, {sa_family=AF_INET, sin_port=htons(6666), sin_addr=inet_addr("0.0.0.0")}, 16) = 0; listen(10, 100) = 0; getsockname(10, {sa_family=AF_INET, sin_port=htons(6666), sin_addr=inet_addr("0.0.0.0")}, [16]) = 0;` – Benp44 Aug 08 '17 at 15:35

1 Answers1

2

Good news is given a pure pyzmq scenario works fine,

the hardware or O/S related issues seem to have been excluded from the trouble list.

Damned bindings ( that blind user-programs by a "high-level" skewed-abstration )

The first thing is,
that some high-level industrialists promoting to escape from native-API remove user-side options.

CZMQ did this for example in that it assumes, that every SUB-side wants to just .connect(), not leaving the reversed .bind()/.connect() choice available. Old good ZeroMQ API always let this option on user and that is fair. Similarly, there are many performance tuning variables that need to get set before a socket-connection is setup, which makes the "high-level" binding more troublesome for distributed-system implementations.

Next is the strange thing with a code running fine on one box and not on another.

My worries would be on user-side responsibility to set LINGER to zero + .close() any such created SUB socket ( which need not happen, if due exception handling is not in place ) before a zmq.Context() instance is safely .term()-inated. This may hang orphans that later cause a real-resource being ( still ) occupied, resulting in errno code 11, Resource temporarily unavailable.

Last but not least - resources are not consumables/disposables

The proposed code cannot better exhibit such resources inefficient behaviour and accumulates all the setup / configuration / tear-down overhead costs per each call. This is a very poor design practice ( while users may meet it in many "schoolbook"-examples ). If interested in more details on efficient distributed-computing arguments, also related to this, may also like this post.

bool ZmqSocketWrapper::connectSubscriber(string address)
{   b_RetFLAG = true;
    m_address = address;
    m_pSocket = zsock_new_sub(address.c_str(), "");

    int errorCode = zmq_errno();
    if (errorCode != 0)
    {
        printf(zmq_strerror(errorCode));
        b_RetFLAG = false;
    }
 // -------------------------------------------------------------
    zsock_destroy( &m_pSocket ); // USER's MANDATORY DUTY PRE-RET
 // -------------------------------------------------------------
    return b_RetFLAG;
}

If indeed in a trouble,

use the native API, where your code is fully controlling the stage.

user3666197
  • 1
  • 6
  • 50
  • 92
  • Many thanks for your response. Rest assured in other parts of my code I do disconnect (if connected) and destroy (if created) my sockets! I'm now creating the socket using `m_pSocket = zsock_new(ZMQ_SUB);`, and then connecting it using `zsock_connect(m_pSocket, "%s", address.c_str());`, but this results in the same error as before. I also tried `zsock_set_linger(m_pSocket, 500);` before the connect. This doesn't seem to do anything, and when I check the error code after this call, I get error code 22: _Invalid argument_. – Benp44 Aug 08 '17 at 12:58
  • Is that very same happening with ZeroMQ API 3.x? – user3666197 Aug 08 '17 at 13:04
  • I've not been able to try that, it seemed to me that using the latest would be sensible, particularly as I'm also using pyzmq. Do you think it would be worth me trying to downgrade to an older zmq and czmq? If so, why? – Benp44 Aug 08 '17 at 13:30
  • Well, the point was not to make you re-install, but to decide to run a B-part of an A/B-test to { revoke | confirm }-the issue, once not using the CZMQ, but doing the same scenario by native-API calls - starting & retaining the full control from first `zmq_context()` up to last `zmq_term()`. Remaining dependent on CZMQ-wrapped flow, many details stay outside of reach of your controls / diagnostics. Why mentioning 3.x >>> http://api.zeromq.org/3-2:_start ? Recent 4.x+ has started to touch untouchable principles ( sharing sockets & this *may* cause new issues in updating language bindings & al ) – user3666197 Aug 08 '17 at 16:23
  • OK, thanks. I think I'll try with the native-API calls. As you say, that should hopefully help diagnose the problem. – Benp44 Aug 08 '17 at 17:28
  • 1
    Yep - I'm now using the native ZMQ calls, and no longer seeing the problem. My best guess is that I was getting false positives when error checking the C binding function calls. – Benp44 Aug 09 '17 at 11:55