0

I know that ssl_handshake_failure is a generic error, however this is my last resort, since I've been investigating this issue for three weeks.

I'm deploying Apache HTTPD server on an AIX Server. AIX does not have standard repository so I've installed it from RPM packages from perzl.org/aix/

I had a hard-time solving all the dependencies, but eventually I got it to work. I hope RPM would not let me to do that without all the proper dependencies.

The apache server works pretty well when I only use HTTP over port 80. Problems arise when I try to connect to it through port 443.

I test it with the following command:

openssl s_client -state -connect 127.0.0.1:443

As soon as I run this command, I get:

CONNECTED(00000003)
SSL_connect:before/connect initialization
SSL_connect:SSLv2/v3 write client hello A

OpenSSL then waits for about two minutes, then throws the following error:

804401144:error:140790E5:SSL routines:ssl23_write:ssl handshake failure:s23_lib.c:177:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 305 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : 0000
    Session-ID:
    Session-ID-ctx:
    Master-Key:
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1505947800
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)
---

This is very strange, it seems that the apache is not sending any data back, then after some timeout the OpenSSL client "tires" of waiting then throws the err.

I tried to keep my configuration as simple as I could, since I just want to complete a handshake for now.

This is my httpd-ssl.conf

LoadModule ssl_module /opt/freeware/lib/httpd/modules/mod_ssl.so

Listen 443

SSLCipherSuite HIGH:MEDIUM:!MD5:!RC4
SSLProtocol all -SSLv3


<VirtualHost *:443>

DocumentRoot "/usr/local/Tomcat/tomcat1/webapps/ROOT"
ServerName <server_name>:443
ServerAdmin root@localhost
ErrorLog "logs/httpd/ssl_error_log"
LogLevel trace8
TransferLog "logs/httpd/ssl_access_log"
LogLevel trace8

SSLEngine on

SSLCertificateFile "cert.crt"
SSLCertificateKeyFile "cert.key"


<FilesMatch "\.(cgi|shtml|phtml|php)$">
    SSLOptions +StdEnvVars
</FilesMatch>
<Directory "/var/www/cgi-bin">
    SSLOptions +StdEnvVars
</Directory>

BrowserMatch "MSIE [2-5]" \
         nokeepalive ssl-unclean-shutdown \
         downgrade-1.0 force-response-1.0

CustomLog "/var/log/httpd/ssl_request_log" \
          "%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x \"%r\" %b"

</VirtualHost>

I put all the logs in trace8 Level, but that didn't help me, when I try to complete the ssl handshake, this is all I get from the ssl_error_log:

[ssl:info] [client 127.0.0.1:36193] AH01964: Connection to child 66 established (server <server_name>:443)
[ssl:trace2] ssl_engine_rand.c(125): Seeding PRNG with 136 bytes of entropy
[ssl:trace3] ssl_engine_kernel.c(1988): [client 127.0.0.1:36193] OpenSSL: Handshake: start
[ssl:trace3] ssl_engine_kernel.c(1996): [client 127.0.0.1:36193] OpenSSL: Loop: before/accept initialization
[ssl:info] [client 194.10.155.114:52781] AH01964: Connection to child 65 established (server <server_name>:443)
[ssl:trace2] ssl_engine_rand.c(125): Seeding PRNG with 136 bytes of entropy
[ssl:trace3] ssl_engine_kernel.c(1988): [client 194.10.155.114:52781] OpenSSL: Handshake: start
[ssl:trace3] ssl_engine_kernel.c(1996): [client 194.10.155.114:52781] OpenSSL: Loop: before/accept initialization

OpenSSL version:

OpenSSL 1.0.2j-fips  26 Sep 2016

Apache HTTPD Version:

Server version: Apache/2.4.25 (Unix)

Any idea about what I may be missing folks? Why the handshake error takes a considerable time to appear, similar to a timeout error?

I'm afraid this can be something dummy that I missed, or some complex problems with dependencies, since I haven't installed it from any repo.

Any guesses?

vinicius.olifer
  • 5
  • 7
  • I see your OpenSSL version is FIPS-capable; are you actually enabling FIPS mode? I don't see `SSLFIPS` in your httpd config, but there are other ways to enable it, and if enabled it must do a selftest that can be quite slow. Try adding `-state` to `s_client` and see at what point(s?) in the handshake the delay occurs. As to the underlying problem, are there any corefiles from the server (unless something prevents it -- I don't know if anything unusual factors into deciding to take a corefile on AIX) – dave_thompson_085 Sep 21 '17 at 07:38
  • Thanks for your feedback Dave, in regards to FIPS, in fact I haven't done any specific configuration on the httpd in this sense, is it mandatory to do that because the openssl is FIPS-capable? If I add `-state` to openssl, this is the result: `CONNECTED(00000003) SSL_connect:before/connect initialization SSL_connect:SSLv2/v3 write client hello A`. I'm editing my quesiton with this result. It seems that the apache server is not answering at all to the request, then OpenSSL fails due a timeout. Concerninf the CORE files, I haven't found any yet, will keep looking. – vinicius.olifer Sep 21 '17 at 14:03
  • FIPS-capable should _default_ to FIPS-actually-off, but I asked because there are multiple ways (I'm not sure I remember all) to turn it FIPS-actually-on. Hmmm; try adding `-debug` as well to see if `s_client` is actually sending the clienthello at BIO level. If so, then it appears the server either isn't getting it or isn't processing it, because with tracing cranked up you should see more states after 'before/accept initialization'. I'm pretty sure `s_client` doesn't have a timeout, but I'd expect Apache does. ... – dave_thompson_085 Sep 22 '17 at 02:12
  • ... You could also try running `openssl s_server` (on a high port to avoid access problems, it defaults to 4433) similarly with `-state -debug` and connect to that to see if it can reproduce the problem with more useful details. – dave_thompson_085 Sep 22 '17 at 02:15

1 Answers1

0

Use a real server name instead of "ServerName <server_name>:443". https://httpd.apache.org/docs/current/en/mod/core.html#servername

"If no ServerName is specified, the server attempts to deduce the client visible hostname by first asking the operating system for the system hostname, and if that fails, performing a reverse lookup on an IP address present on the system."

MichaEL
  • 1
  • 1