OpenSSL for 64-bit Windows and "no shared cipher"

Question

I have just compiled and installed OpenSSL for 64-bit Windows. I have created a self-signed certificate and a private key with the command:

openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -days 10000 -nodes

I am now testing the "Simple TLS Server" example found at OpenSSL Wiki with Firefox and a couple of modifications to support Winsock, but I keep getting the error

11216:error:1417A0C1:SSL routines:tls_post_process_client_hello:no shared cipher:ssl\statem\statem_srvr.c:1422:

(the first number always varies) during the execution of the SSL_accept() function. I have checked the list of (11) ciphers sent by Firefox (v 43.0.1) in its TLS v1.2 Client Hello when connecting to some HTTPS server using Wireshark (because capturing on localhost is difficult) and compared it to the ones supported by my installation of OpenSSL (found using openssl.exe ciphers -s -tls1_2 -V). The result is that there are common ciphers, so what am I missing?!

The block containing the line 1422 of statem_srvr.c is the following, starting with 1420:

if (cipher == NULL) {
    SSLerr(SSL_F_TLS_POST_PROCESS_CLIENT_HELLO,
           SSL_R_NO_SHARED_CIPHER);
    goto f_err;
}

The modifications to the original code are before the while loop and in the headers:

#pragma comment(lib,"Ws2_32.lib")
#include <stdio.h>
#include <winerror.h>
#include <WinSock2.h>
#include <openssl/ssl.h>
#include <openssl/err.h>
#include <openssl/applink.c>

and

int sock;
SSL_CTX *ctx;

WSADATA WsaDat;
if (WSAStartup(MAKEWORD(2, 2), &WsaDat) != 0) perror("Winsock fatal startup error");
init_openssl();
ctx = create_context();
configure_context(ctx);

sock = create_socket(4433);

Edit: This is what happens when I try to connect to the server using s_client with TLSv1.2:

CONNECTED(000000F0)
23368:error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure:ssl\record\rec_layer_s3.c:1362:SSL alert number 40
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 7 bytes and written 176 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : 0000
    Session-ID:
    Session-ID-ctx:
    Master-Key:
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1473536238
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: no
---

Are you able to connect to your local server using openssl itself? What cipher does it connect with? `openssl s_client -connect localhost:4433 -tls1_2` — Jason Hoetger, Sep 10 '16 at 19:34
No, I get the same error. I will post the output of the client program. — Æðelstan, Sep 10 '16 at 19:38
@JasonHoetger - He's on a Windows machine. He may need to use TLS 1.0 instead of 1.2 (SChannel lacks TLS 1.2 in may versions of Windows). @Æðelstan - try `openssl s_client -connect localhost:4433 -tls1 -servername localhost`. `-servername` enlists SNI, which is a TLS 1.0 and above feature. The 4096-bit key could also be a problem. Many versions of Windows could only handle 2048-bit keys. — jww, Sep 10 '16 at 19:42
@jww `Many versions of Windows could only handle 2048-bit keys` it is correct for pre-Win2k systems (Windows 98, for example). — Crypt32, Sep 10 '16 at 20:27
@MobileGuy - Windows CE, Windows Mobile and some Windows Phone, too. That was 2008 through 2012. I recall having to make some nasty changes to a protocol because Windows could only perform 2048 Diffie-Hellman. — jww, Sep 10 '16 at 20:32
@jww can't agree with Windows Phone and WinCE, because it (WP) natively (since WP7) supports 4k RSA keys and ECC cryptography. Windows CE (5.0) with Enhanced CSP does support 4k RSA keys. Apparently, only Windows Mobile (which is dead long ago) remains in the list and barely fits "many versions" in 2016. — Crypt32, Sep 10 '16 at 20:57
This message can mean that the server couldn't find its own private key and certificate, and that conjecture is supported by the 'no peer certificate' message. — user207421, Sep 11 '16 at 02:13
@EJP that did it for me as well. In my case, I created an SSL object from my context but added the cert and key to the ctx afterward instead of before, so they were never copied to the ssl object. — MDMoore313, Sep 20 '16 at 20:44

score 0 · Accepted Answer · answered Sep 11 '16 at 11:51

0

As it turns out, there was a problem with finding the certificate and private key. Problem solved.

answered Sep 11 '16 at 11:51

Æðelstan

822
12
23

score 0 · Answer 2 · answered Sep 20 '16 at 20:43

The comment by EJP solved my problem. In my case, I created an SSL object with an SSL_CTX, but set the cert and key for the SSL_CTX after I created the ssl object, so the cert and key was never copied to the SSL object. Moving my cert and key code to before I created my SSL object solved the issue.

Wrong way:

/*Create SSL Context*/
SSL_CTX *ctx = SSL_CTX_new(TLS_server_method());

/*Create SSL Object*/
SSL *ssl = SSL_new(ctx);

/*Load certs into SSL server*/ //<---WRONG; SHOULD NOT BE HERE
if (!SSL_CTX_use_certificate_file(ctx, "C:\\Users\\dzmf39\\Documents\\PPH1261726\\TLS\\tls_cert.pem", SSL_FILETYPE_PEM)) {
    fprintf(stderr, "Error while loading SSL Server Certificate.\n");
    return 0;
}

if (!SSL_CTX_use_PrivateKey_file(ctx, "C:\\Users\\dzmf39\\Documents\\PPH1261726\\TLS\\tls_key.pem", SSL_FILETYPE_PEM)) {
    ERR_print_errors_fp(stderr);
    return 0;
}

SSL_do_handshake(ssl); //Throws handshake error

Right Way

/*Create SSL Context*/
SSL_CTX *ctx = SSL_CTX_new(TLS_server_method());

/*Load certs into SSL server*/ //<---CORRECT; GOES BEFORE SSL object creation
if (!SSL_CTX_use_certificate_file(ctx, "C:\\Users\\dzmf39\\Documents\\PPH1261726\\TLS\\tls_cert.pem", SSL_FILETYPE_PEM)) {
    fprintf(stderr, "Error while loading SSL Server Certificate.\n");
    return 0;
}

if (!SSL_CTX_use_PrivateKey_file(ctx, "C:\\Users\\dzmf39\\Documents\\PPH1261726\\TLS\\tls_key.pem", SSL_FILETYPE_PEM)) {
    ERR_print_errors_fp(stderr);
    return 0;
}

/*Create SSL Object*/
SSL *ssl = SSL_new(ctx);

SSL_do_handshake(ssl);

There are other methods where you can add the key directly to the SSL object, in which case this wouldn't apply, but I didn't go that route.

score 0 · Answer 3 · answered Jul 27 '18 at 09:26

I also encountered same problem. I also resolved it:

berofe(that's error)

SSL_CTX_new
SSL_new
SSL_CTX_use_certificate_chain_file
SSL_CTX_use_PrivateKey_file

after(that's ok)

SSL_CTX_new
SSL_CTX_use_certificate_chain_file
SSL_CTX_use_PrivateKey_file
SSL_new

OpenSSL for 64-bit Windows and "no shared cipher"

3 Answers3