0

I really like this: http://services.digg.com/2.0/stream

Data is continuously received using a single connection, content-type is application/json. Also, it's consumable by a browser. How is this implemented? Looking around a bit it does not seem to be long-polling, HTML5 server-sent events or even websockets.

How would I go about building something like that? Are there any open-source solutions I can use as a starting point?

Thanks,

/David

OG Dude
  • 936
  • 2
  • 12
  • 22

1 Answers1

2

It's just a stream of bytes transmitted with chunked transfer encoding. Basically it's just HTTP chunked streaming.

Since you mentioned server-sent events, you can think of this as SSE without the special formatting. Messages are JSON objects delimited by newline. On the server you can implement this the same way you would implement SSE minus formatting.

A client for such an API needs to be able to parse such a stream of data and split messages on newline. Some browsers support a readyState of 4 on the XMLHttpRequest, which is called for each chunk, thus allowing you to process the streamed data.

igorw
  • 27,759
  • 5
  • 78
  • 90
  • Hi Igor, thank you for your response. Are there any reasons against chunked transfer encoding and for SSE in the context of building a streaming API? – OG Dude May 27 '12 at 22:49
  • Sure, if your API is not targeting browsers, you can simply use chunks. In those use cases they are way easier to parse than the EventSource protocol. Another thing worth considering is cross-domain communication. Many of the EventSource implementations have no support for CORS (Cross-origin resource sharing) or only just got it. Whereas with AJAX it is supported everywhere. This is another case where it might be an option. – igorw May 27 '12 at 23:07
  • Thanks again, so the Digg API works in most browsers as a "top level" document even though it uses chunks. Are you saying that maybe it cannot be consumed by way of XMLHttpRequest? – OG Dude May 29 '12 at 22:15