Indy 10 + SSL = works in Windows 7, does not work on XP

Question

I'm using the Indy 10 Http Client (latest SVN build) and a SSL Handler (Delphi 7) to get the content of the https://www.webtide.com/choose/jetty.jsp website.

It works fine on Windows 7 x64 (tested on two systems), but on WindowsXP x86 (tested on 3 systems) the test app simply hangs on TIdHTTP.Get() without the possibility of a recovery (meaning even disconnecting in a worker-procedure/thread does not work!). The test app cannot be recovered and must be closed with the task manager.

The SSL libraries (32bit x86!) are from here: http://slproweb.com/products/Win32OpenSSL.html but I've tried 5 other versions from different sites, with the same results.

Here is a zip package with source code, compiled executable, and the SSL libraries:

https://www.dropbox.com/s/pd5soxon0qbnnl0/IndyTest.zip

And here is the source code (the form has a button and two memos):

 procedure TForm1.Button1Click(Sender: TObject);
 var IdHTTP1: TIdHTTP;
     sl : TStringList;
     SSL1: TIdSSLIOHandlerSocketOpenSSL;
 begin
   try
    try
      IdHTTP1 := TIdHTTP.Create(nil);
      sl := TStringList.Create;

      SSL1 := TIdSSLIOHandlerSocketOpenSSL.Create(nil);
      SSL1.SSLOptions.Method := sslvSSLv23;

      with IdHTTP1 do
      begin
           ConnectTimeout := 10 * 1000;
           ReadTimeout := 10 * 1000;
           IOHandler := SSL1;

           Request.UserAgent := 'Mozilla/5.0 (X11; U; Linux i586; en-US; rv:1.7.3) Gecko/20040924 Epiphany/1.4.4 (Ubuntu)';
           Memo2.Text := 'connecting...';
           Application.ProcessMessages;
           Memo1.Text := Get('https://www.webtide.com/choose/jetty.jsp');
           Memo1.Lines.Add ('response: '+ResponseText);
           Memo2.Text := 'connected or timeout...';
      end;
    except
      On e: Exception do
           Memo2.Text := 'Exception: '+e.Message;
    end;
   finally
      IdHTTP1.Free;
      SSL1.Free;
      sl.Free;
   end;
 end;

Why does it crash/hang on WindowsXP?

Can't you ask the debugger why it's hung? Pause the program, and the debugger will tell you where it's waiting. Can you use this code to connect to any other sites, or is it just this specific one that makes your program hang? — Rob Kennedy, Mar 12 '13 at 14:17
Debugging indy is very difficult, because when you do the connection in slow motion weird things happen. If I pause the code and use F8 Delphi goes thru all the message handles. Nothing of Indy code is shown. — Casady, Mar 12 '13 at 14:25
I've used the debugger to track the Get() function step by step. In procedure TIdCustomHTTP.DoRequest() there is this connect line: ConnectToHost(Request, Response); Going with F7 into it I land on: function TIdCustomHTTP.GetResponse: TIdHTTPResponse; Result := FHTTPProto.Response; end; If I press F8 one more time and the debugger hangs and I have to stop the app. — Casady, Mar 12 '13 at 14:35
After setting A LOT of break points in Indy code I know where it hangs: unit IdSSLOpenSSL; procedure TIdSSLSocket.Connect(); this line hangs: error := SSL_connect(fSSL); Unfortunately this is an external function from the OpenSLL DLL: {$EXTERNALSYM SSL_connect} SSL_connect : function(ssl: PSSL): TIdC_INT cdecl = nil; — Casady, Mar 12 '13 at 14:53
From what I could test, I see that https://www.webtide.com/choose/jetty.jsp responds very slow. I suspect that Indy is not respecting the connection timeout under XP. Verify with wireshark traces and compare when it works and when not. Maybe the firewall in XP is giving you trouble. — whosrdaddy, Mar 12 '13 at 14:56
You might be onto something. This website is hosted on a Java server, so it's very slow. But I don't think it's an indy problem, since indy is calling an external OpenSSL function and this function never comes back if either the connection or the server is very slow. Is there a way to force a timeout on calling an external DLL function? In this case SSL_connect(fSSL); — Casady, Mar 12 '13 at 15:17
There is also SSL1.ReadTimeOut, but setting it, e.g. to 5000, does not change anything, it still hangs indefinitely, since it does not hang in Indy Code, but inside the open source libeay32.dll. — Casady, Mar 12 '13 at 19:43
Indy's `ConnectTimeout` only applies to the socket API `connect()` function when establishing the underlying TCP/IP connection. `SSL_connect()` is called afterwards to initiate the SSL handshake, which is application data and thus is not subject to the `ConnectTimeout`. For what it is worth, Indy does use its `ReadTimeout` property to assign socket-level read/write timeouts on OpenSSL connections, but only on Vista+ as a workaround for an OpenSSL bug. On XP, default socket read/write timeouts apply. — Remy Lebeau, Mar 12 '13 at 20:45
Default socket read/write timeouts are apparently not applied, as it hangs even after 1 hour! Is there no way to forcibly exit/disregard the SSL_connect() call in case it hangs? Is the application doomed once it happens? — Casady, Mar 12 '13 at 20:49
@Casady: You can manually set the socket read/write timeouts after the socket is connected. I have posted an answer with an example. — Remy Lebeau, Mar 12 '13 at 21:03
Thanks, but it does not help, as apparently it hangs on connect, before receiving any data, so the send/receive timeout does not apply. Or it does apply, but the hanging is in the external OpenSSL dll (i'm 90% sure about it), and the Delphi App loses control. — Casady, Mar 12 '13 at 22:01

score 1 · Answer 1 · answered Mar 12 '13 at 21:02

1

Indy's ConnectTimeout property only applies to the socket API connect() function when establishing the underlying TCP/IP connection. SSL_connect() is called at a later time to initiate the SSL handshake, which is application data and thus is not subject to the ConnectTimeout.

Indy does use its ReadTimeout property to assign socket level read/write timeouts on OpenSSL connections, but only on Vista+ as a workaround for an OpenSSL bug. On XP and earlier, default socket read/write timeouts apply. The ReadTimeout only tells Indy how long to wait when reading data, but it is not applied to the socket itself. If you want to do that, you can do it manually by calling the TIdSocketHandle.SetSockOpt() method after establishing the TCP/IP connection but before beginning the SSL handshake, for example:

procedure TForm1.Button1Click(Sender: TObject);
var
  IdHTTP1: TIdHTTP;
  SSL1: TIdSSLIOHandlerSocketOpenSSL;
begin
  try
    IdHTTP1 := TIdHTTP.Create(nil);
    try
      SSL1 := TIdSSLIOHandlerSocketOpenSSL.Create(IdHTTP1);
      SSL1.SSLOptions.Method := sslvSSLv23;

      with IdHTTP1 do
      begin
        ConnectTimeout := 10 * 1000;
        ReadTimeout := 10 * 1000;
        IOHandler := SSL1;

        OnConnected := IdHTTPConnected;
        OnStatus := IdHTTPStatus;

        Request.UserAgent := 'Mozilla/5.0 (X11; U; Linux i586; en-US; rv:1.7.3) Gecko/20040924 Epiphany/1.4.4 (Ubuntu)';

        Memo1.Text := Get('https://www.webtide.com/choose/jetty.jsp');
        Memo1.Lines.Add('response: '+ ResponseText);

        Memo2.Text := 'finished...';
      end;
    finally
      IdHTTP1.Free;
    end;
  except
    on e: Exception do
      Memo2.Text := 'Exception: ' + e.Message;
  end;
end;

procedure TForm1.IdHTTPStatus(ASender: TObject; const AStatus: TIdStatus; const AStatusText: string);
begin
  case AStatus of
    hsResolving: Memo2.Text := 'resolving...';
    hsConnecting: Memo2.Text := 'connecting...';
    hsConnected: Memo2.Text := 'connected...';
    hsDisconnecting: Memo2.Text := 'disconnecting...';
    hsDisconnected: Memo2.Text := 'disconnected...';
  end;
  Update;
end;

procedure TForm1.IdHTTPConnected(Sender: TObject);
begin
  with TIdHTTP(Sender).Socket.Binding do
  begin
    SetSockOpt(Id_SOL_SOCKET, Id_SO_RCVTIMEO, 10 * 1000);
    SetSockOpt(Id_SOL_SOCKET, Id_SO_SNDTIMEO, 10 * 1000);
  end;
end;

answered Mar 12 '13 at 21:02

Remy Lebeau

555,201
31
458
770

Remy, I've tried your code. Works fine on Win7, but on WindowsXP it does still hang (since 10 minutes now the test app has an hourglass icon). There is no "connecting..." in memo2. Even worse when it hangs you cannot pause the app and use the debugger in the Delphi IDE, as a the IDE caption changes to "stopping..." and hangs there, until you terminate the app! – Casady Mar 12 '13 at 21:22
I think the problem here is that Receive/Send timeouts are not applied, because it hangs already on connect/handshake, not when sending/receiving data. – Casady Mar 12 '13 at 21:38
@Casady: If you are not seeing the `OnStatus` event being triggered with the `hsConnecting` status, then Indy is not attempting to connect the socket to the server at all, and that has nothing to do with SSL. The `OnConnected` event is triggered immediately when the socket is connected, before any data is exchanged, so that is the earliest oppurtunity to set socket timeouts. – Remy Lebeau Mar 12 '13 at 22:18
1

@Casady: Assuming you are at least seeing the `OnStatus` event trigger with the `hsResolving` state, then you are encountering a hang in the DNS resolution of the hostname to an IP address, which occurs before the socket can be connected. That resolution is handled by the OS, not by Indy, so there is no way for Indy to use any timeout for that. If that turns out to be where your hang is actually occuring, then your PC is having DNS issues outside of Indy. – Remy Lebeau Mar 12 '13 at 22:19
@Casady: If the hang is actually on `SSL_connect()`, then you are guaranteed to have received `OnStatus` events for `hsConnecting` and `hsConnected` states, and the `OnConnected` event so you can set your own socket timeouts. `SSL_connect()` simply exchange data back adn forth over the established socket, so socket timeouts DO affect it. – Remy Lebeau Mar 12 '13 at 22:21
Remy, unfortunately I don't receive it on WindowsXP! I've checked it over and over, it hangs before the IdHTTPStatus event. I've checked it by placing a breakpoint at case AStatus of, and the breakpoint is not reached. Pausing the delphi IDE is not possible, only terminating the app. If I debug it line by line it hangs at SSL_connect() – Casady Mar 12 '13 at 22:33
2

let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/26064/discussion-between-remy-lebeau-and-casady) – Remy Lebeau Mar 12 '13 at 22:57
I've added more/new information about the problem to the chat. – Casady Mar 12 '13 at 23:15
Since this issue is unresolved, I've decided to add a 50 points bounty. – Casady Mar 21 '13 at 19:56

Indy 10 + SSL = works in Windows 7, does not work on XP

1 Answers1

Linked