3

I have a Delphi 6 application that uses an Indy TIdTCPClient instance to communicate with a web server. The reason I am not using an HTTP client directly is because the the server is an image streaming server that uses the same socket connection for receiving the command to start streaming as it does to start "pushing" images back to you. In other words, after you send it a typical HTTP POST request, it replies with an HTTP response, and immediately after that it starts sending out a stream of JPEG images.

I already know how to craft a proper POST request and send it using the TIdTCPClient WriteBuffer() method and then use the ReadBuffer() method to receive reply data. What I'd like to do instead is to send a POST request and then ask Indy to wait for a typical HTTP response including retrieving all the bytes in the response body if there is a Content-Length header variable. I of course want it to leave the JPEG frames intact that may have piled in after the HTTP response in the receive queue until I start requesting them (that is, I don't want it including any of the JPEG frames in the HTTP response to my streaming request command until I ask for them using a successive read call).

Is there a method that I can call on a TIdTCPClient that will retrieve completely a typical HTTP response with body content, and nothing else? I thought about using SendCmd() and checking the LastCmdResult property (type: TIdRFCReply) for the response, but I can't tell from the Indy documentation if it retrieves the response body content too if there is a Content-Length header variable as part of the response it returns, nor can I tell if it leaves the rest of the receive queue after the response intact.

What is the best way to accomplish this mixed mode interaction with an HTTP web server that pushes out a stream of JPEG frames right after you make the HTTP request to start streaming?

Also, if there is a clever way to have Indy split the frames using the JPEG frame WINBONDBOUDARY delimiting string, rather than accumulating blocks of data and parsing them out myself, please share that technique.

Robert Oschler
  • 14,153
  • 18
  • 94
  • 227
  • 1
    Why are you using `TIdTCPClient` directly and not `TIdHTTP` instead? `TIdHTTP` handles the response reading for you, and it is possible to read streaming data with `TIdHTTP` with a little extra work (it does not natively support streaming data). – Remy Lebeau Jun 05 '12 at 00:45
  • @RemyLebeau - I have to send a repeating HTTP command in a tight loop to a robot. If there is the slightest pause between commands the motors wind down and it is impossible to achieve smooth motion. I originally did use the TIdHTTP component, with all the keep-alive tips you gave me on another post, but it is not responsive enough. Sometimes it returns immediately, other times it takes 200-500 and even 3000 extra milliseconds to return a response. This is all on a background thread. Since I switched to TIdTCPClient and handle the HTTP handshaking myself all those problems went away. – Robert Oschler Jun 05 '12 at 03:15
  • 1
    If you are using an up-to-date version of Indy, the `TIdHTTP.HTTPOptions` property was updated not too long ago with a new `hoWaitForUnexpectedData` flag to deal with some response delays, just make sure that flag is disabled. – Remy Lebeau Jun 05 '12 at 17:25
  • @RemyLebeau - I'm using Indy 9 since I could not get newer versions to compile successfully with Delphi 6. I just did a full text search through the Indy 9 source code and could not find the hoWaitForUnexpectedData flag so I guess it's in the newer versions. Fortunately my code is working now using an IdTCPClient instance. – Robert Oschler Jun 05 '12 at 22:03
  • 1
    Indy 9 and 10 both support D6. If you are having problems installing Indy 10 on D6, feel free to post a new question about that, or ask in the Embarcadero or Indy forums. `hoWaitForUnexpectedData` is only in Indy 10, and it is a fairly recent addition. Earlier releases of Indy 10, and Indy 9, include some hard-coded delays in `TIdHTTP` to deal with some buggy HTTP servers that send data when they are not supposed to, so the delays are for checking if unexpected data actually show up on the socket. – Remy Lebeau Jun 06 '12 at 00:12

1 Answers1

2

The correct way to read an HTTP response is to first read the CRLF-delimited response headers line-by-line until a blank line is encountered, aka a CRLF+CRLF sequence, then you can use those headers to decide how to read the remaining response data. The headers will tell you not only what kind of stream is being sent (via the Content-Type header), but also how the data is being framed (Content-Length, Transfer-Encoding: chunked, something specific to the particular Content-Type, etc).

To receive the headers, you can use the connection's Capture() method, setting its ADelim parameter to a blank string.

How you read the remaining data afterwards depends on the actual formatting/framing of the stream. Without knowing exactly what kind of stream you are receiving, there is no way to advise you how best to read it, as there are several different types of streaming protocols used by HTTP servers, and most of them are not standardized. Provide that information, then I/we can show you how to implement it with Indy.

You cannot use SendCmd() as the HTTP protocol does not format its responses in a way that is compatible with that method.

Remy Lebeau
  • 555,201
  • 31
  • 458
  • 770
  • The response stream is either a simple header only response or a response with a body whose length is specified by the Content-Length header variable. I simply read that variable (if it exists) and then do a blocking read for that many bytes. Works fine as of now. – Robert Oschler Jun 05 '12 at 03:16
  • 1
    Read [RFC 2616 Section 4.4](http://tools.ietf.org/html/rfc2616#section-4.4) for all of the official rules for determining if a response body is present and how to read it correctly. – Remy Lebeau Jun 05 '12 at 17:26