TIdHTTP.Get timeouts while the same call done with Postman succeeds: possible reasons?

Question

I call a webapi with a Delphi app, in some pcs, the call timeouts, while in other it works fine.

The request done with Postman works fine.

It is a simple custom ping webservice (URL is in Edit1.Text in the code below), in fact the answer is a textual "Pong".

This is the Delphi code of the call:

 errormsg := '';
  {
  old way of setting custom headers
   IdHTTP1.Request.CustomHeaders.AddValue('X-HTTP-Method-Override', 'ForwardCommand');
  IdHTTP1.Request.CustomHeaders.AddValue('Connection', 'keep-alive');
  IdHTTP1.Request.CustomHeaders.AddValue('Accept', '*/*');
  IdHTTP1.Request.CustomHeaders.AddValue('User-Agent', 'QualibusSilent');
  IdHTTP1.Request.CustomHeaders.AddValue('Content-Type', 'text/plain');
   }
   //better way of setting custom headers
  IdHTTP1.Request.MethodOverride := 'ForwardCommand';
  IdHTTP1.Request.Connection := 'keep-alive';
  IdHTTP1.Request.UserAgent   := 'myCustomUserAgent';
  IdHTTP1.Request.ContentType   := 'text/plain';
  IdHTTP1.Request.Accept   := '*/*';
      IdSSLIOHandlerSocketOpenSSL1 :=  TIdSSLIOHandlerSocketOpenSSL.Create(IdHTTP1);
      IdSSLIOHandlerSocketOpenSSL1.SSLOptions.Mode       := sslmClient;
      IdSSLIOHandlerSocketOpenSSL1.SSLOptions.SSLVersions:= 
        [sslvTLSv1,sslvTLSv1_1,sslvTLSv1_2];
    
  startTime := GetTickCount;
  Try
  sHTML := IdHTTP1.Get(Edit1.Text);
  Except
    On E:Exception do
      errormsg := e.Message;
  End;
  EndTime := GetTickCount;
  ShowMessage('Time taken: ' +
    IntToStr(endTime-startTime)+#13#10+'Error:'+errormsg);

Basically it is a call where instead of GET I do a custom method (ForwardCommand) that I call with X-HTTP-Method-Override.

In the code above I tried to add many headers so that the call is really as the Postman one.

If the call is done directly to the IP address it works, but if I call the https URL it timeouts, there is no evidence of error in the proxy server.

Checking the logs at the webserver side it seems the call is not performed at all.

And this occurs only from some Windows 10 machines, while in the majority of them the call is performed correctly.

Could you please suggest which could be the cause of the error? What should I try to change in the Delphi code to avoid the timeout so that Delphi behaves like Postman?

Thanks.

Sure [Postman](https://www.postman.com) is doing exactly the same thing? Sure your Delphi program uses TLSv2 first and _Postman_ does the same? Have you verified this with i.e. [WireShark](https://www.wireshark.org/)? Have you tried HTTP instead of HTTPS to rule out TLS problems? — AmigoJack, Jul 18 '22 at 16:45
A few notes: 1) are you assigning `IdSSLIOHandlerSocketOpenSSL1` to `IdHTTP1.IOHandler`? 2) in general, consider using `Request.CustomHeaders.Values['...'] := '...';` instead of `Request.CustomHeaders.AddValue('...', '...')`; 3) However, don't use `Request.CustomHeaders` AT ALL in this example, you should be using the `Request.MethodOverride`, `Request.Connection`, `Request.Accept`, `Request.UserAgent`, and `Request.ContentType` properties instead; 4) don't enable `sslvSSLv2` or `sslvSSLv3` AT ALL, they are no longer secure and nobody uses them anymore, TLS 1.0 should be the absolute minimum — Remy Lebeau, Jul 18 '22 at 19:14
In any case, you mean the error only occurs when requesting a URL by hostname, not by IP. Are you sure DNS is working properly on the failing systems? Also, you mention a proxy server, what kind of proxy are you using, and are you configuring `TIdHTTP` to use that proxy? — Remy Lebeau, Jul 18 '22 at 19:20
hi @RemyLebeau, i rewrote the customheaders part (i also modified it in the question, for instance i wrote `IdHTTP1.Request.MethodOverride := 'ForwardCommand';`) and it works the same, so I assume this is mainly a most elegant way to use Indy. I also removed `sslvSSLv2`, `sslvSSLv23` and `sslvSSLv3`. It still does not work on some pcs. — UnDiUdin, Jul 19 '22 at 08:26
hi @RemyLebeau (CONTINUED) The reason why i mentioned "proxy" is that we use nginx as reverse proxy on the webserver so the API is accessable as https too, but that proxy does not even recieve the call. Moreover `IdSSLIOHandlerSocketOpenSSL1` is assigned at design time to `IdHTTP1.IOHandler`. So all seem ok, as far as I know. — UnDiUdin, Jul 19 '22 at 08:27
@AmigoJack I do not use WireShark but i tried with Fiddler and I setup the TIdHTTP.ProxyParams to use Fiddler. The interesting thing I noticed is that if I add `IdHTTP1.ProxyParams.ProxyServer:= '127.0.0.1'; IdHTTP1.ProxyParams.ProxyPort:= 8888;` the call works and the response is immediate (even if the response seems not encoded correctly) also in the "problematic PCs", while if i do not run that code timeout occurs. Any idea? — UnDiUdin, Jul 19 '22 at 08:27
@RemyLebeau an interesting thing I just observed is that even trying to get fom www.google.com (doing a get and not a method override) gives timeout on the problematic machines, so the API is not involved, but somehow the call made with Indy timeouts. I also noticed that if i use ProxyParams (passing the settings to use Fiddler described above) the get works fine. Do you have an idea on what could be the reason? Why when Tidhttp uses proxyParams timeout does not occur and the call succeeds? Thanks! — UnDiUdin, Jul 19 '22 at 09:28
I finally got the reason for the Postman vs Indy behavior: Proxy. By passing proxy IP and port to TIdHTTP it works, Postman manages to retrieve the system proxy automatically and therefore it works. @RemyLebeau could you please suggest a way to overcome this? Thanks! — UnDiUdin, Jul 19 '22 at 14:13
@UnDiUdin "*Moreover `IdSSLIOHandlerSocketOpenSSL1` is assigned at design time to `IdHTTP1.IOHandler`*" - then why are you re-creating it in code at all? You should not be doing that. All of its settings (and `TIdHTTP`'s too) that you are assigning are available at design-time. — Remy Lebeau, Jul 19 '22 at 18:34
@RemyLebeau you are right about design time vs code, but in fact, I do all in code in my real application, in the prototype I wrote to investigate this proxy issue I did not do that. Anyway good point, thanks. — UnDiUdin, Jul 20 '22 at 12:51

score 2 · Accepted Answer · answered Jul 19 '22 at 18:33

2

As stated in comments:

Why when Tidhttp uses proxyParams timeout does not occur and the call succeeds?
...
I finally got the reason for the Postman vs Indy behavior: Proxy. By passing proxy IP and port to TIdHTTP it works, Postman manages to retrieve the system proxy automatically and therefore it works.

There is no "system proxy" on Windows, however there is a proxy in the WinInet API, which is what Internet Explorer (and Edge?) relies on.

In any case, it sounds like the failing PCs don't have direct access to the Internet to begin with, only through a proxy. Indy has no concept of any "system proxy" on any platform, so you will have to assign the proxy settings to TIdHTTP manually, as you have discovered.

answered Jul 19 '22 at 18:33

Remy Lebeau

555,201
31
458
770

Thanks for the reply. This was the difference. I would like to successfully guess the proxy using win apis, i will experiment more and then likely ask a specific question. Thanks a lot! – UnDiUdin Jul 20 '22 at 07:20
Yes, Windows knows no "system proxy", but nowadays enough programs run on that OS that support the [`http_proxy` (and `https_proxy`) environment variable](https://unix.stackexchange.com/a/386435). I let my programs also do that, because the effort to do so is minimal, and likewise `TIdHTTP` could have an option. – AmigoJack Dec 13 '22 at 10:23
@AmigoJack `TIdHTTP` has proxy settings. Nothing stops a program from reading in the environment variable and then setting the corresponding `TIdHTTP` settings as needed. Indy will not read the environment directly. – Remy Lebeau Dec 13 '22 at 16:55

TIdHTTP.Get timeouts while the same call done with Postman succeeds: possible reasons?

1 Answers1