1

I am using IdHTTP.Post in my application to upload data to a server in the cloud. A DLL in the cloud saves the data to a SQL server database, and returns a simple message back to the application.

The uploaded data is simply a bunch of long strings uploaded in a loop (won't go into why here). Each string in the loop is pretty much always the same length.

For example:

For I:=1 to 10 do
begin
  ...
  Reply:=IdHTTP.Post(URL,String);
  ...
end;

This works perfectly well 98% of the time. Sometimes, when the loop is more than 100 say, then 'Reply' will be blank. If I try again, then it works fine.

I tried various things, one of which is to add

Sleep(2000);

after each iteration of the loop, this also seems to do the trick!

So, in short, my upload seems to be tying itself into a knot every now and then when the upload is on the large size. Any recommendations as to how to handle this better than 'Sleep' would be appreciated.

I have not made any changes to the IdHTTP component from the default settings, apart from:

IdHTTP.ConnectTimeout:=5000;
IdHTTP.ReadTimeout   :=5000;

I am using Delphi 2010.

Note that the string lengths are the same for each iteration, whether I am looping 10 times, or 100 times. So the string length itself does not seem to be the issue.

Update

So ignore pretty much most of what I thought was happening above. @Remy Lebeau correctly identified that the problem was that the reply was not being decoded correctly, and that is because IdHTTPOptions.[hoForceEncodeParams] was set to false.

The reason for this is outlined in a previous question:

Delphi TIdHTTP POST does not encode plus sign

@Remy, my version of Indy, as you pointed out in the old question, and in response to this one, is certainly outdated. I was a bit hesitant to install a later version as the install instructions looked a bit scary, and so I opted to encode manually instead in the meantime. This looks to have backfired, so will look at it now for sure.

At least now I know what is causing my problem.

I have a few follow up questions that you might be able to help me with:

  1. I think I might be doing something wrong, but even if I set hoForceEncodeParams to True or False on the component, when looking at Fiddler, it doesn't always seem to be consistent in terms of the encoding going through to the server. I have only picked this up now because I'm looking at it in detail. The only way to be SURE of the encoding either way is to set it in the code right before posting, in other words ignore the setting on the component. Can this be correct, or what am I missing? This is why I understood the problem incorrectly in the first place, as the encoding was correct on the first iteration of the loop, but then inexplicably changed on the subsequent iterations.
  2. With a IdHTTP.Get, I DO get a reply even if hoForceEncodeParams is set to false. Does this make sense?
Community
  • 1
  • 1
Alex
  • 543
  • 1
  • 9
  • 21
  • Do you handle Indy exceptions? Is there any exception? What is the response from server (whole headers)? – smooty86 Jan 21 '16 at 18:22
  • Did you verify that the server's reply actually contains the expected string data when `TIdHTTP` returns a blank string? Unless an exception is being raised, the only ways that `Post()` could return a blank string are if either the server is not sending any data to begin with, or if there is an issue with charset-decoding the data from bytes to characters. – Remy Lebeau Jan 21 '16 at 18:41
  • Code that is fixed by adding a Sleep of a certain amount is actually usually NOT fixed, just fixed for your observed case. You have to debug this yourself, I don't think we can do that. I suspect race conditions or basic logic bugs. I suggest you add enough logging on both the Client and Server side of the HTTP Post that you can find out what happened to that missing response. Was it sent? Not sent? Until you find that out, you are speculating without profit. What do you mean "in a knot"? The server did not receive the data? Do you know how to use WireShark? – Warren P Jan 21 '16 at 20:46
  • Thanks for the pointers. I used Fiddler to try and debug this based on your comments. Not very experienced at all with Fiddler, but the reply that I am expecting seems to be returned correctly from the server. I just can't see it when I step over 'Reply:=IdHTTP.Post(URL,String)' in my application. – Alex Jan 22 '16 at 14:37
  • @Remy you might have nailed it on the decoding front, will follow this up later when I get a chance. But, in short, I have set HTTPOptions hoForceEncodeParams to false, because previously we have found that the IdHTTP component encoded the '+' incorrectly. We now encode manually to get around this. So this could be backfiring, will confirm. – Alex Jan 22 '16 at 14:47
  • You are likely using an outdated version of Indy. I know for a fact that `hoForceEncodeParams` encodes properly, because I rewrote that logic a few years ago to better match W3C specs. – Remy Lebeau Jan 22 '16 at 16:48
  • @Remy, I have added an update to my question based on this new information, along with a few new questions. Thanks again. – Alex Jan 23 '16 at 11:15
  • @Alex: if you are seeing inconsistencies in the encoding, you should post an example to [Indy's forum](http://forums2.atozed.com) or [Indy's mailing list](https://groups.yahoo.com/neo/groups/Indy-Dev-Public/info) so we can discuss the details. As for `Get()`, it does not post any encoded parameter data, so it is not affected by `hoForceEncodeParams`. Any data you send with `Get()` is in the URL itself, and you are responsible for passing in a properly encoded URL. – Remy Lebeau Jan 24 '16 at 08:17
  • @Remy, my problem is totally resolved if I set HTTPOptions within the loop, right before the Post command. Even if I set it to [] (ie no encoding), the reply comes through correctly. What seems to be happening is the component has the HTTPOptions set to [], but after the 1st iteration of the loop my manual encoding gets re-encoded, and then things go pear from there. I still don't understand why the initial iteration of the loop doesn't produce a reply, as I can see on Fiddler that the encoding goes correctly to the server. But anyway, just by setting it again within the loop to [] .... – Alex Jan 25 '16 at 13:30
  • ....resolves the problem. Thanks again for the help. – Alex Jan 25 '16 at 13:30

0 Answers0