1

There are two techniques for implementing Comet. One uses HTTP streaming, which uses a single persisted TCP connection to send and receive multiple HTTP requests/responses between client/server.The second is HTTP long polling, which keeps a connection open by the server, and, as soon as an event occurs, the response is committed and the connection is closed. Then, a new long-polling connection is reopened immediately by the client waiting for new events to arrive.

I am using the Faye ruby gem and I noticed it uses Comet/Bayeux out of the box. But I cannot find out which type of Comet technique it uses. I just gather that Bayeux is publish-subscribe protocol. I'm curious to know if it suffers the same shortcomings of HTTP streaming and long polling. Does it allow full-duplex communication (communication in both directions, and, unlike half-duplex, allows this to happen simultaneously.)?

Donato
  • 2,727
  • 6
  • 29
  • 59

1 Answers1

3

Your definition of HTTP streaming and long-polling are not correct.

In HTTP streaming, the client sends a request to the server, and the server replies with an "infinite" response that contains small chunks of data (messages), typically using the chunked transfer encoding. This mechanism has been standardized as EventSource (a.k.a Server-Sent Events). It is a server-to-client only push of events. For the client to send another message to the server, it has to open a new connection.

In HTTP long-polling, the client sends a request that is held by the server until an event (or a timeout) occurs, then the response is committed but the connection is not closed. The connection is kept open and other requests may be sent on that connection, both normal or long-polling requests (one at a time, of course).

The Bayeux protocol is an application protocol on top of a transport protocol such as HTTP or WebSocket.

HTTP is a full duplex protocol in the context of a single request/response exchange. Multiple HTTP exchanges are serialized (that is, executed one after the other). The HTTP request/response exchange is the unit of serialization.

WebSocket is a full duplex protocol in the context of WebSocket messages. WebSocket messages may be sent and received simultaneously. The WebSocket message is the unit of serialization.

Bayeux inherits the characteristics of the transport protocol is it carried on. The Bayeux protocol itself does not itself have any "duplexness" characteristics, you can think of it just as a way to format messages in a particular textual form.

Both CometD and Faye use Bayeux over both WebSocket and HTTP long-polling.

sbordet
  • 16,856
  • 1
  • 50
  • 45
  • How does Comet not cause a http server like apache2 to crash with so many open tcp connections, consuming memory. I mean if you have 64,000 open tcp connections, each one 1mb, that's 64gb of memory alone, surely causing most servers to crash. – Donato Jul 10 '15 at 17:52
  • 1
    A tcp connection alone does not consume 1 MiB of memory. It's the thread associated with that connection that consumes that MiB. That is why apache2 is not the right server to use for long-polling. Other servers such as [Jetty](http://eclipse.org/jetty) (which powers [CometD](http://cometd.org)), nginx, etc. are able to sustain tens of thousands of connections without consuming much memory because they use asynchronous I/O and thread in a more efficient way for this case. I was able to connect 400,000 CometD clients to a single CometD/Jetty server before hitting memory limits. – sbordet Jul 10 '15 at 21:57
  • Thanks for this wonderful feedback. I was using apache2, but after reading your comments, nginx it is. – Donato Jul 10 '15 at 22:33