7

I have a web app that makes frequent TIdHTTP calls to the Google Analytics API (around 25,000-50,000 per day). Every so often calls to the API fail with the error message in the subject line (not often - less than 1 out of 1000 times). I have never been able to find a pattern to get it to happen. And retrying the failed call usually works. So it seems entirely random.

I have the latest version of openssl (1.0.2.1 - 03/20/2015). And the latest version of Indy (source code files dated 01/07/2015).

Below is the basic source code for making these calls.

Anyone have any ideas what it could be?

Would making two simultaneous calls to the API affect things (this is taking place in a multi-threaded Web App)?

IdSSLIOHandlerSocket1 := TIdSSLIOHandlerSocketOpenSSL.create(nil);
IdSSLIOHandlerSocket1.PassThrough := True;
IdHTTP := TIdHTTP.create(nil);
IdHTTP.reusesocket := rsTrue;
IdSSLIOHandlerSocket1.reusesocket := rsTrue;
idhttp.handleredirects := True;
with IdSSLIOHandlerSocket1 do begin
  SSLOptions.Method := sslvTLSv1_2;
  SSLOptions.SSLVersions := [sslvTLSv1_2];
  SSLOptions.VerifyMode := [];
  SSLOptions.VerifyDepth := 2;
end;
with IdHTTP do begin
  IOHandler := IdSSLIOHandlerSocket1;
  ProxyParams.BasicAuthentication := False;
  Request.UserAgent := 'EmbeddedAnalytics API Interface';
  Request.ContentType := 'text/html';
  request.connection := 'close';
  Request.Accept := 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8';
  Request.BasicAuthentication := False;
  Request.UserAgent := 'Mozilla/3.0 (compatible; Indy Library)';
  HTTPOptions := [hoForceEncodeParams];
  Request.AcceptEncoding := 'gzip,deflate';
  Request.CustomHeaders.Add('Accept-Language: en-us,en;q=0.5');
  idhttp.Request.CustomHeaders.Add('Authorization: Bearer '+FToken);
end;
idhttp.get(':https://www.googleapis.com/analytics/v3/data/realtime?ids=..........');

Update 1 update some lines of code to:

SSLOptions.Method := sslvSSLv3;
SSLOptions.SSLVersions := [sslvSSLv3];

It works. I will monitor and see if SSL errors go away.

Solution Turns out making the changes to sslVSSLv3 fixed it. I no longer get the errors! This is somewhat surprising seeing that most all other services are adopting TLS instead.

M Schenkel
  • 6,294
  • 12
  • 62
  • 107
  • Not related to your question, but you should 1) not be using `ReuseSocket`, 2) not be manually adding `deflate` or `gzip` in `Request.AcceptEncoding` (set the `TIdHTTP.Compressor` property instead), 2) use `Request.AcceptLanguage` instead of `Request.CustomHeaders.Add()`, and 4) remove the `:` in front of the URL you are passing to `Get()`. And if you are making so many requests, you might consider using `Request.Connection := 'keep-alive'` instead of `'close'` so you can reuse a single HTTP connection for multiple requests. – Remy Lebeau Apr 15 '15 at 01:49
  • Thanks again @RemyLebeau for your active monitoring of Indy issues. I will take a look at your suggestions. I have found that going back to sslvSSLv3 for the Google API calls takes care of the problem. Surprising seeing so many other services are moving away from this because of POODLE. To you have an explanation for why most of the time sslvTLSv1_2 worked, but every so often it would not? – M Schenkel Apr 15 '15 at 17:06
  • @RemyLebeau - Thank you. I have implemented these. I actually have a setting to toggle betwen `close` and `keep-alive`. At the time it was set to "close". Question: will "keep-alive" be effective If after the call I free the idhttp object? Also, regarding the `:`, this was a typo. – M Schenkel Apr 21 '15 at 20:56
  • By default, no. If you free the `TIdHTTP` object, it will close the connection. If you want to re-use the same connection for a new `TIdHTTP` object, you have to 1) save the last `Host`/`Port` and `IOHandler` property values, 2) set the `IOHandler` property to nil without closing the connection, 3) assign the saved `IOHandler` object to another `TIdHTTP` object, and assign the saved `Host`/`Port` values, 5) set the `TIdHTTP.Response.KeepAlive` property to true and the `HTTP.Response.Connection` property to `keep-alive`. That will allow `TIdHTTP` to then re-use the existing connection. – Remy Lebeau Apr 22 '15 at 02:42
  • Check this out: https://stackoverflow.com/questions/50840101/curl-35-error1408f10bssl-routinesssl3-get-recordwrong-version-number – Michel Gokan Khan Nov 13 '18 at 11:21

3 Answers3

6

Problem solved by changing this:

SSLOptions.Method := sslvTLSv1_2;
SSLOptions.SSLVersions := [sslvTLSv1_2];

To this:

SSLOptions.Method := sslvSSLv3;
SSLOptions.SSLVersions := [sslvSSLv3];

You might want to try TLS 1.0 instead, to avoid SSLv3.

There are two things to be mindful of with Google and TLS 1.2. And some of this may have changed by now. (This discussion is very specific, and it only applies to Google servers and TLS 1.2).

First, you have to disable compression if using TLS 1.2 and ECDSA. This weird factoid showed up in a discussion on the OpenSSL mailing list under ECDHE-ECDSA Support. Here's a related support ticket it generated: Bug 3277: OpenSSL s_client doc missing option.

Second, if your are not using the ChaCha20/Poly1305 ciphers, then you have to be mindful of fallback cipher suites for TLS 1.2. I was never able to figure this one out (especially since all the ephemeral DH suites should be supported), but I know it used to be the case from testing. So be sure to include the following for fallback (this is also needed for Microsoft servers running IIS 8 (or maybe 7) and earlier):

  • TLS_RSA_WITH_AES_256_CBC_SHA256
  • TLS_RSA_WITH_AES_256_CBC_SHA
  • TLS_RSA_WITH_AES_128_CBC_SHA256
  • TLS_RSA_WITH_AES_128_CBC_SHA
ppostma1
  • 3,616
  • 1
  • 27
  • 28
jww
  • 97,681
  • 90
  • 411
  • 885
  • Thanks - I will switch back and make the changes as you suggest. And will let you know the results. Remy Lebeau pointed out that compression should not be set. – M Schenkel Apr 15 '15 at 20:38
  • 1
    @MSchenke - I think TLS 1.2 is a good choice because of the AES/GCM (AEAD) or the ChaCah20/Poly1305 cipher suites. You should try to get that to work. If you can't, then fall back to "TLS 1.0 and above". It will take 1.2 if it can be negotiated; otherwise it will take the highest TLS it can get. Here's how to do "TLS 1.0 and above" in C: [How to block SSL protocols in favor of TLS?](http://stackoverflow.com/a/29291287/608639). But I don't know how to do it with Delphi or Indy. – jww Apr 15 '15 at 21:31
3

Problem solved by changing this:

SSLOptions.Method := sslvTLSv1_2;
SSLOptions.SSLVersions := [sslvTLSv1_2];

To this:

SSLOptions.Method := sslvSSLv3;
SSLOptions.SSLVersions := [sslvSSLv3];

This is surprising seeing that most services are moving to TLS instead.

M Schenkel
  • 6,294
  • 12
  • 62
  • 107
  • I've encountered problems with TLS 1.2 *and* Google servers. See the answer below that may explain some of the things you are experiencing, and some potential work arounds. – jww Apr 15 '15 at 17:50
0

I doubt that Google still allows access to their servers using SSLv3 (see Poodle attack).

The POODLE attack (which stands for "Padding Oracle On Downgraded Legacy Encryption") is a man-in-the-middle exploit which takes advantage of Internet and security software clients' fallback to SSL 3.0.

So if your client gets a SSLv3 related error message I would contact a network expert to check if this error message can be caused by a man-in-the-middle attack.

It also could be a simple network problem, as it is not reproducible.

For a deeper diagnosis, a Wireshark recording would be helpful (for the expert, not me).

mjn
  • 36,362
  • 28
  • 176
  • 378
  • Thank you for comment. Within the year I had to update from Indy 9 to Indy 10 for the sole reason of the POODLE attack. In that case it was to get PayPal interface to work. I don't have exact version, but I was using something before sslvTLSv1_2. Upgrading to Indy 10 and using this new version fixed that problem. You say you doubt Google still allows access using SSLv3. If this were the case wouldn't it make sense they would deny for ALL requests? Certainly they wouldn't allow some through but not others would they? – M Schenkel Apr 14 '15 at 15:42
  • *"The POODLE attack ... is a man-in-the-middle..."* - its not a MitM attack. If the man were in the middle, he would have direct access to the material and would not need to use the padding oracle. The attacks I have seen are more like a MitB (Man in the Browser) who is allowed to make unauthenticated requests. In the browser, its simply a [CSRF](https://www.owasp.org/index.php/Cross-Site_Request_Forgery_%28CSRF%29) – jww Apr 14 '15 at 18:40
  • @jww the [original source](https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2014-3566) says SSLv3 `makes it easier for man-in-the-middle attackers to obtain cleartext data via a padding-oracle attack, aka the "POODLE" issue` – mjn Apr 14 '15 at 18:55
  • @mjn - more misinformation. There is no active attacker who is intercepting the channel. A better discussion can be found at [TLS Working Group: Rethink TLS 1.3](https://www.ietf.org/mail-archive/web/tls/current/msg14704.html). – jww Apr 14 '15 at 18:57
  • Yeah - maybe poor networking: https://github.com/google/google-api-python-client/issues/37 – M Schenkel Apr 14 '15 at 19:02
  • @mjn - *"I doubt that Google still allows access to their servers using SSLv3"* - Google has SSLv3 enabled on their servers. See, for example, `$ openssl s_client -ssl3 -connect www.google.com:443`. And it not just Google's main site; its their other properties, like GDrive and Gmail, too. They worship money, and the packet must get through at all costs so the ad can be served. Google did disable SSLv3 in their Chrome browser, though. See https://threatpost.com/google-removes-sslv3-fallback-support-from-chrome/109455. – jww Apr 14 '15 at 19:14