Python amqp broken pipe error

Question

Quick question:

Do I need to pray for not having two independent applications use same channel at the same time while communicating RabbitMQ in order to prevent "Broken Pipe Error"? (Or is threading different from having two or more independent applications?)

History:

I have written some applications and one of them (io-server) is acting like a server from the point of view of my other applications and is acting like a client from the point of view of RabbitMQ server.

Now, everything works as expected for about 10 minutes. Then, my io-server crashes. Here is the last part of traceback:

    File "/usr/local/lib/python2.7/dist-packages/amqp-1.4.2-py2.7.egg/amqp/transport.py", line 163, in write_frame
    frame_type, channel, size, payload, 0xce,
  File "/usr/lib/python2.7/socket.py", line 224, in meth
    return getattr(self._sock,name)(*args)
error: [Errno 32] Broken pipe

Here is the relevant part of RabbitMQ log:

=ERROR REPORT==== 31-Mar-2014::12:29:53 ===
AMQP connection <0.22183.0> (running), channel 1 - error:
{amqp_error,unexpected_frame,
            "expected content header for class 60, got non content header frame instead",
            'basic.publish'}

=INFO REPORT==== 31-Mar-2014::12:30:23 ===
closing AMQP connection <0.22183.0> (127.0.0.1:43367 -> 127.0.0.1:5672)

As far as I could search the net, the most promising answer is here:

...
So yes, RabbitMQ closes the connection due to a connection-level error
with frame interleaving. 
...
 * Avoid publishing on *the same* channel from multiple threads
 * Synchronize publishing in your own code  

I may make my apps publish synchronously. But how can I guarantee that all of my applications run in sync? Do I really have to?

Answer 1

Do I need to pray for not having two independent applications use same channel

No. By "channel" here, the links that you share mean a shared Channel object. Different applications (processes) will always have a different channel for each if they do not share memory.

Is threading different from having two or more independent applications?

Yes. The amqp libraries that you use are not thread-safe. This is a problem if you share a Channel() object across different threads in your "IO server". This would however not be a problem if your server is single threaded and you run many parallel instances.

I suggest you keep it simple and do not use threading in your server application. Assuming your IO server accepts HTTP connections from the other apps; Rely on nginx+uwsgi to accept parallel requests.

Answer 2

I've experienced this exact issue and (as @istepaniuk) points out this is related to threading. You cannot reuse the same connection in multiple threads, you will have to open 1 connection per thread.