0

I'm trying to run openSSL examples, (source downloaded from here). I've managed to build it and running the 'wserver' example on port 4433 (running on Ubuntu 12.10 virtual machine using VirtualBox)

I then try to launch a client from Windows 8, but I keep getting SSL accept error shows on the server's terminal.

I saw this post and copied the bash script from the accepted answer, using it, I've managed to create a new root.pem and other certificates, but I'm still getting the same error

(Don't know it if matters, I'm using port 4433 because the VM couldn't bind to 443 for some reason, and Yes, I changed it on the client as well)

Also, server written in C.

Any ideas? Thanks!

EDIT

I found that the error happens in this line

r=SSL_accept(ssl);

I checked the error returned into r using

SSL_get_error(ssl,r);

and it appeard to be SSL_ERROR_SYSCALL

I checkd ERR_get_error(); and it returned 0 meaning

If ret == 0, an EOF was observed that violates the protocol

Not sure yet what it means..

EDIT 2

I checked the client return code, and it's error 12045 which, according to MSDN is

ERROR_INTERNET_INVALID_CA

12045

The function is unfamiliar with the Certificate Authority that generated the server's certificate.

But when using the supplied client from the virtual machine, the connection is fine and no certificates problems

Overriding 12045 as shown by Microsoft here, leads to a 12038 error, which is

ERROR_INTERNET_SEC_CERT_CN_INVALID

           SSL certificate common name (host name field) is incorrect.
           For example, if you entered www.server.com and the common
           name on the certificate says www.different.com.

Does this help? Thanks again!

Community
  • 1
  • 1
La bla bla
  • 8,558
  • 13
  • 60
  • 109

2 Answers2

1

Because the error code was SSL_ERROR_SYSCALL and the r was zero, the client did close the underlying TCP connection gracefully (by calling close() or shutdown()).

You should look from the client code, what may cause the closing of TCP connection.

Edit: according to edit2 of the question:

There are some problems with certificates.

I suggest to use self-signed cerificates during testing for avoiding problems with certificate chains. When everything works fine, then start using more complex certificates.

Because you get ERROR_INTERNET_SEC_CERT_CN_INVALID, there may be wrong common nane (CN) in a certificate. Check the Subject: field of certificate. There is CN=xxxxx in the subject. The xxxxx must be same as IP/hostname of the host.

Or you can also remove the common name check from your client code. You have probably the following code from the example:

if(require_server_auth)
  check_cert(ssl,host);

Comment it out or set require_server_auth=0. And then try again.

SKi
  • 8,007
  • 2
  • 26
  • 57
  • Yeah I tried changing the `CN` in my self signed certificate, bu still no good, So I found a way (client is written in WinAPI, not familiar with it) to disable the SSL check for now, and it runs now. Thanks for your help! – La bla bla Dec 13 '12 at 14:03
0

It probably means that there was an error that occurred during the Accept phase of the SSL handshake.

Also, I think there's a verbose option for openSSL. -v I think.

Are you using a packet sniffer (Wireshark, Ethereal) to help in your debugging?

TheQuickBrownFox
  • 116
  • 1
  • 6
  • I'm quite new to this, Not quite sure how can they help me debugging.. Editing my question to add a bit more info. Thanks for your answer – La bla bla Dec 13 '12 at 08:59
  • Packet sniffers help you in your debugging because it a allows you to peek inside the packets that you are sending. Try running a packet sniffer then perform your test. Make sure to filter ssl in your packet sniffer. Since the server reported an ssl accept error, I'm sure it will reflect in the server's reply packet during the ssl handshake. – TheQuickBrownFox Dec 13 '12 at 09:19
  • About the EOF: Your client program probably sent some incomplete data to the server during SSL accept. I still suggest that you perform debugging with packet capture. We always want to make sure that we are actually sending the data we think we are sending. =) – TheQuickBrownFox Dec 13 '12 at 09:27