How do I receive push notifications with Indy?

Question

I have a device that uses restful Web services and I have used its request/response functionality whereby I send it a command via HTTP GET and it responds with the appropriate XML.

I now need to use the device's pus notifications. I have tried the same approach as above whereby I supply the TIdHTTP.Get procedure with the relevant HTTP URL and stream in which to place the response, but this doesn't seem to work. The call to Get doesn't come back. This makes sense to me in that with push notifications you are opening an HTTP stream of communication between the device and the program and this connection will remain open for streaming until closed.

My problem though is I don't know how to get the XML from the stream if the Get method doesn't return. It is as if the program has hung. I have tried to put the communication via GET with the device and the reading of the stream into a thread so that this can continue on its own and then my main application can just check the resultant XML but this too does not work.

I am wondering if I am over complicating this and if there is a simple solution. When I simply send a request and get a response, it works fine; it's just the push streaming that I can't get to work.

If I use the URL in Internet Explorer I can see the XML returned as well as the "busy" logo constantly running indicating that the stream is open.

I have run the command through my browser Firefox Mozilla 20.0.1 and seen what wireshark captures for the push request and response. The HTTP part is shown below:

GET /elite/notifications/stream?resume=2013-05-06T00:00:00Z HTTP/1.1 
Host: 192.168.10.10 
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:20.0) Gecko/20100101 Firefox/20.0 
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8 
Accept-Language: en-US,en;q=0.5 
Accept-Encoding: gzip, deflate 
Connection: keep-alive

The request I am trying to do in Delphi is as follows:

if fEventStream = nil then
  fEventStream := TStringStream.Create('');
try
  TapItem.fHTTP.Get(TapItem.I2IReaderIP+'/elite/notifications/stream?resume=2013-05-01T00:00:00Z',fEventStream);
  TapItem.fEventXML := fEventStream.ReadString(fEventStream.Size);
except
  on e:exception do begin
    ShowMessage(e.message)
  end;
end;

I have also tried with TidTCPConnection

  fTCPConn.IOHandler.Writeln(TapItem.I2IReaderIP+I2I_PUSH_EVENT+'2013-05-01T00:00:00Z');
  While not Terminated do begin
    XMLString := XMLString + fTCPConn.IOHandler.Readln;
  end;

Any assistance in this matter would be appreciated.

SOLUTION

In my case simply adding the GET keyword to the beginning of the call worked. I assumed the TCP Connection component requires knowledge of what action (ie GET PUT etc) it is you wish to do, it cant read my mind, makes sense

fTCPConn.IOHandler.Writeln('GET ' + TapItem.I2IReaderIP+I2I_PUSH_EVENT+'2013-05-01T00:00:00Z');
While not Terminated do begin
  XMLString := XMLString + fTCPConn.IOHandler.Readln;
end;

Hint: on Stackoverflow, you can add your solution as an answer (and accept it) — mjn, Mar 06 '14 at 13:11

score 3 · Answer 1 · answered May 09 '13 at 20:19

3

TIdHTTP does not support server-side pushes at this time (and even then, there are multiple ways that server-side pushes can be implemented - which one is your REST service using?). You will have to switch to TIdTCPClient, format and send the HTTP request manually, and then use a timer or thread to read and parse the pushed responses as needed as they arrive.

answered May 09 '13 at 20:19

Remy Lebeau

555,201
31
458
770

I don't suppose you can provide me with a simple example of how to do this? I have updated my code to used the TidTCPClient instead now and after some fiddling got some form of response back but it is HTML. Looks like document header information. There is no XML being returned. Cant find good examples on how to do this. thnx – MarkZA May 10 '13 at 07:10
1

I suggest you use a packet sniffer, such as Wireshark or Fiddler, to see how IE is requesting the URL, then duplicate those request values in Indy. Especially the `User-Agent` in particular, as some servers send different content for different types of clients, and when it comes to server pushes, a webserver is likely to use different types of pushes for different types of webbrowsers since there is no standardize format for pushes that everyone agrees on yet. If you update your question with an example of a push captured by a sniffer, I can show you how to replicate it with `TIdTCPClient`. – Remy Lebeau May 10 '13 at 07:32
Hi. Not entirely sure what you are looking for. I have run the command through my browser and seen what wireshark captures for the push request. The HTTP part is shown below: GET /elite/notifications/stream?resume=2013-05-06T00:00:00Z HTTP/1.1 Host: 192.168.10.10 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:20.0) Gecko/20100101 Firefox/20.0 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: en-US,en;q=0.5 Accept-Encoding: gzip, deflate Connection: keep-alive – MarkZA May 10 '13 at 08:58
I can also see in Wireshark the relevant XML being returned. My problem is how do I implement this in Delphi. my code in Delphi is try fTCPConn.IOHandler.Writeln(TapItem.I2IReaderIP+I2I_PUSH_EVENT+'2013-05-01T00:00:00Z'); While not Terminated do begin XMLString := XMLString + fTCPConn.IOHandler.Readln; end; finally ShowMessage(XMLString); end; – MarkZA May 10 '13 at 09:22
The reply I get back in Delphi is : EPFCA01423-FC00110000000105 [v1.02.005] ' – MarkZA May 10 '13 at 09:23
Could it be something to do with the Content-Type either being sent or received. Also in my example above where I write using the IDTCPClient do I need to format the URL request into an HTTP packet of some sort. Just thinking perhaps the idHTTP.GET method might have done something like this? – MarkZA May 10 '13 at 09:24
Yes, it is related to the Content-Type that the server is sending. What does it send to IE exactly? Yes, you have to send a fully formatted HTTP request manually (I told you that). Again, please put the actual request+response data in your question, not in comments. Or post the Wireshark log somewhere for download. – Remy Lebeau May 10 '13 at 15:55
HI, Have updated the original question with the HTTP request part captured through wireshark. In the meatime I think I might have figured out my problem, just need to test a little further. COrrect me if I am wrong but I think the TidHTTP.GET Procedure was formatting the request in the sense that it added the GET keyword to the beginning of the request. The TIdTCPConnection did not do this because of course it cant smell what it is that I am trying to do. When I added the GET keyword to the beginning of my request string I got XML back – MarkZA May 13 '13 at 05:24
It is not just a matter of preceding the first line with `GET`. The entire request must conform to HTTP standards, and yours does not (no HTTP version number, missing required headers, no double line break after the headers). I also asked you to provide the actual server push data that is being sent, but you have not done that yet. As I said, [there are several different types of server pushes](http://en.wikipedia.org/wiki/Push_technology) used online, so I would need to see what kind of pushing this particular server is actually using in order to explain how to receive it properly with Indy. – Remy Lebeau May 13 '13 at 05:52
Perhaps the device I am working with is not true push, all I do know is that once I added GET to the front of the URL which I then sent through using the IdTCPClient component every worked. I received the XML I expected, the application didn't block and all new triggered events also get sent. Thank you to all for their input, much appreciated. I have updated the question with the few lines of code that I am now using to achieve the push in my application – MarkZA May 15 '13 at 05:12

mjn · Answer 2 · 2014-03-06T13:08:12.827

The HTTP client (TIdHTTP) in the trunk of Indy 10.6 now also supports "Server Push", if the server sends a multipart/... response:

New TIdHTTP hoNoReadMultipartMIME flag

A new hoNoReadMultipartMIME flag has been added to the TIdHTTP.HTTPOptions property. The purpose of this flag is to specify whether TIdHTTP should read the body content of "multipart/..." responses, such as "multipart/x-mixed-replace" or "multipart/byteranges", into the target TStream or to exit immediately and let the caller read the content manually instead.

As there are many different ways to implement Server Push, this answer can be more or less helpful for a given server.

How do I receive push notifications with Indy?

2 Answers2