curl SSL protocol error in connection to shbr-staging.surescripts.net

Question

I am trying to connect to: shbr-staging.surescripts.net/

Here are the steps that I have followed:

Step 1: I generated a private key

$openssl genrsa -des3 -out sc-private.key 2048
passphrase: xyz123

Step 2: Using the private key from Step 1 generated a CSR and sent it to surescripts

$openssl req -new -out for-surescripts-generated-on-20-july-2016.csr -key sc-private.key -config ./openssl.conf

Step 3: Surescripts emailed back to me: drone.sc.com.p7b

Step 4: Verified that the file drone.sc.com.p7b has 3 certificates

$openssl pkcs7 -inform der -in drone.sc.com.p7b -print_certs | less

Step 5: From the p7b files extracted the individual certificates

$openssl pkcs7 -inform DER -print_certs -in drone.sc.com.p7b | awk 'split_after==1{n++;split_after=0} /-----END CERTIFICATE-----/ {split_after=1} {print > "cert" n ".pem"}'

ref: stackoverflow.com/questions/6211591/export-p7b-file-with-all-the-certificate-chain-into-cer-file

Step 6: Confirm that cert2.pem is the client certificate.

cert.pem
subject=/C=US/O=Surescripts LLC./OU=Surescripts Certification Authorities/CN=Surescripts Root Certification Authority
issuer=/C=US/O=Surescripts LLC./OU=Surescripts Certification Authorities/CN=Surescripts Root Certification Authority

cert1.pem
subject=/C=US/O=Surescripts LLC./OU=Surescripts Certification Authorities/CN=Surescripts Issuing Certification Authority
issuer=/C=US/O=Surescripts LLC./OU=Surescripts Certification Authorities/CN=Surescripts Root Certification Authority

cert2.pem
subject=/C=US/ST=California/L=Los Altos/O=Savant Care, Inc/OU=SC/CN=drone.sc.com/emailAddress=vk@sc.com
issuer=/C=US/O=Surescripts LLC./OU=Surescripts Certification Authorities/CN=Surescripts Issuing Certification Authority

Step 7: Verify that the private key, certificate and CSR match

$openssl rsa -noout -modulus -in sc-private.key | openssl md5
$openssl req -noout -modulus -in for-surescripts-generated-on-20-july-2016.csr | openssl md5
$openssl x509 -noout -modulus -in cert2.pem | openssl md5

All 3 return a802effa75a7ee2750e7a25dc07a450d

Ref: digitalocean.com/community/tutorials/openssl-essentials-working-with-ssl-certificates-private-keys-and-csrs

Step 8: Confirm that curl is built correctly

$curl -V
curl 7.47.0 (x86_64-pc-linux-gnu) libcurl/7.47.0 GnuTLS/3.4.10 zlib/1.2.8 libidn/1.32 librtmp/2.3
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp smb smbs smtp smtps telnet tftp
Features: AsynchDNS IDN IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz TLS-SRP UnixSockets

Step 9: Using the certificate and the private key trying to connect using curl:

$curl -k -vvvv https://shbr-staging.surescripts.net/ --cert cert2.pem --key sc-private.key
* Trying 69.25.46.22...
* Connected to shbr-staging.surescripts.net (69.25.46.22) port 443 (#0)
* found 173 certificates in /etc/ssl/certs/ca-certificates.crt
* found 1049 certificates in /etc/ssl/certs
* ALPN, offering http/1.1
* error reading X.509 key or certificate file: Decryption has failed.
* Closing connection 0
curl: (35) error reading X.509 key or certificate file: Decryption has failed.

@dave_thompson_085 pointed out curl is not built using openssl hence

Step 10: Rebuilt curl. Confirming that this new curl uses openssl

$curl -V curl 7.50.1 (x86_64-pc-linux-gnu) libcurl/7.47.0 OpenSSL/1.0.2g zlib/1.2.8 libidn/1.32 librtmp/2.3
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp smb smbs smtp smtps telnet tftp
Features: AsynchDNS IDN IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz TLS-SRP UnixSockets

Step 11: Using the certificate and the private key trying to connect using curl:

$ curl -k -vvvvvv https://shbr-staging.surescripts.net/rxhub --cert cert2.pem --key sc-private.key
* Trying 69.25.46.22...
* Connected to shbr-staging.surescripts.net (69.25.46.22) port 443 (#0)
* ALPN, offering http/1.1
Enter PEM pass phrase:
* Cipher selection:
ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: /etc/ssl/certs
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Request CERT (13):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS handshake, CERT verify (15):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* Unknown SSL protocol error in connection to shbr-staging.surescripts.net:443
* Closing connection 0
curl: (35) Unknown SSL protocol error in connection to shbr-staging.surescripts.net:443

To solve the above problem strategies I have tried:

Strategy1: Use a lower version of curl

$curl -V curl 7.45.0 (x86_64-pc-linux-gnu) libcurl/7.47.0 OpenSSL/1.0.2g zlib/1.2.8 libidn/1.32 librtmp/2.3
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp smb smbs smtp smtps telnet tftp
Features: AsynchDNS IDN IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz TLS-SRP UnixSockets

Strategy2: Use a lower version of openssl

$/usr/local/bin/curl -V
curl 7.50.1 (x86_64-pc-linux-gnu) libcurl/7.50.1 OpenSSL/1.0.1e zlib/1.2.3
Protocols: dict file ftp ftps gopher http https imap imaps pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: IPv6 Largefile NTLM NTLM_WB SSL libz UnixSockets

I both strategy 1 and 2 I get the same error as step 11

Step12: Output of

$openssl s_client -connect shbr-staging.surescripts.net:443 -key sc-private-key-without-password.pem -cert cert2.pem -servername drone.savantcare.com -state

CONNECTED(00000003)
SSL_connect:before/connect initialization
SSL_connect:SSLv2/v3 write client hello A
SSL_connect:unknown state
depth=2 C = US, O = Surescripts LLC., OU = Surescripts Certification Authorities, CN = Surescripts Root Certification Authority
verify return:1
depth=1 C = US, O = Surescripts LLC., OU = Surescripts Certification Authorities, CN = Surescripts Issuing Certification Authority
verify return:1
depth=0 C = US, ST = VA, L = Arlington, O = Surescripts, OU = Operations, CN = shbr-staging.surescripts.net, emailAddress = network@surescripts.com
verify return:1
SSL_connect:unknown state
SSL_connect:unknown state
SSL_connect:unknown state
SSL_connect:unknown state
SSL_connect:unknown state
SSL_connect:unknown state
SSL_connect:unknown state
SSL_connect:unknown state
SSL_connect:unknown state
SSL_connect:unknown state
SSL_connect:error in unknown state
SSL_connect:error in unknown state
write:errno=104
---
Certificate chain
 0 s:/C=US/ST=VA/L=Arlington/O=Surescripts/OU=Operations/CN=shbr-staging.surescripts.net/emailAddress=network@surescripts.com
   i:/C=US/O=Surescripts LLC./OU=Surescripts Certification Authorities/CN=Surescripts Issuing Certification Authority
 1 s:/C=US/O=Surescripts LLC./OU=Surescripts Certification Authorities/CN=Surescripts Issuing Certification Authority
   i:/C=US/O=Surescripts LLC./OU=Surescripts Certification Authorities/CN=Surescripts Root Certification Authority
 2 s:/C=US/O=Surescripts LLC./OU=Surescripts Certification Authorities/CN=Surescripts Root Certification Authority
   i:/C=US/O=Surescripts LLC./OU=Surescripts Certification Authorities/CN=Surescripts Root Certification Authority
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIFgzCCBGugAwIBAgIKYMCecAAAAAACazANBgkqhkiG9w0BAQsFADCBjjELMAkG
A1UEBhMCVVMxGTAXBgNVBAoTEFN1cmVzY3JpcHRzIExMQy4xLjAsBgNVBAsTJVN1
cmVzY3JpcHRzIENlcnRpZmljYXRpb24gQXV0aG9yaXRpZXMxNDAyBgNVBAMTK1N1
cmVzY3JpcHRzIElzc3VpbmcgQ2VydGlmaWNhdGlvbiBBdXRob3JpdHkwHhcNMTUw
NDI3MjEwOTUxWhcNMTcwNDI2MjEwOTUxWjCBqDELMAkGA1UEBhMCVVMxCzAJBgNV
BAgTAlZBMRIwEAYDVQQHEwlBcmxpbmd0b24xFDASBgNVBAoTC1N1cmVzY3JpcHRz
MRMwEQYDVQQLEwpPcGVyYXRpb25zMSUwIwYDVQQDExxzaGJyLXN0YWdpbmcuc3Vy
ZXNjcmlwdHMubmV0MSYwJAYJKoZIhvcNAQkBFhduZXR3b3JrQHN1cmVzY3JpcHRz
LmNvbTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMio25K7vFgmM2v9
VL726msqmZAeAiq2PswOy+KDhRtBJWFfEYRKrhpNU2JwCB7WpnqHzzhsQSDuBTFv
mO6D3mEa5Oo04nBfxg7HDX3INeWWZ5wGzoenD8yc9tvNj157FMbO1I+J4ij2r1sF
f7geo02hctuZMj9qRVlVLZrQrZfRH2XNmyccNtSbX+D+wSqBx1uyqEJQpEsDA4Kj
ZyY1eM1G76y7CvIg2fZ58bD5K5DIBncLRh7oOXgJ/8PBPo27w/QXnn3Vr3g0xJiV
KUbJK4eHBhOStJnvXfDBrubqQ0oqcNxf+USzqX3ZUc26Fg2R7IS/KBhabpS8rCtc
K5UhlScCAwEAAaOCAcUwggHBMB0GA1UdDgQWBBQn9ZUw3rtN8vjOyWe1Od2vGKs8
uTAfBgNVHSMEGDAWgBRUkp4XGGrE3wF7f0vzTnl/B9TzTjBIBgNVHR8EQTA/MD2g
O6A5hjdodHRwOi8vcGtpLnN1cmVzY3JpcHRzLm5ldC9yZXBvc2l0b3J5L1N1cmVz
Y3JpcHRzQ0EuY3JsMIGABggrBgEFBQcBAQR0MHIwQwYIKwYBBQUHMAKGN2h0dHA6
Ly9wa2kuc3VyZXNjcmlwdHMubmV0L3JlcG9zaXRvcnkvU3VyZXNjcmlwdHNDQS5j
cnQwKwYIKwYBBQUHMAGGH2h0dHA6Ly9wa2kuc3VyZXNjcmlwdHMubmV0L29jc3Aw
DgYDVR0PAQH/BAQDAgWgMD4GCSsGAQQBgjcVBwQxMC8GJysGAQQBgjcVCITQ+E+D
3v80h/2ZLILPsHmFp4wLgX2FyskYhtbgSAIBZAIBBzAdBgNVHSUEFjAUBggrBgEF
BQcDAgYIKwYBBQUHAwEwGgYDVR0gBBMwETAPBg0rBgEEAYKlT4N9MgEDMCcGCSsG
AQQBgjcVCgQaMBgwCgYIKwYBBQUHAwIwCgYIKwYBBQUHAwEwDQYJKoZIhvcNAQEL
BQADggEBAE0vZrTHr8L3UuZrsewuAB2ON9m574UkaUFfOZX8SLUVLHM9UfdKpS/I
vo9fCYG7u4K+itvUqtdWfXKz2zhKCrWMjEh1s4cNWIcnccwCnRt7rFMYl8fGlemO
JzfCnIBYmSYi6QGCmycCtn9rK9Ifgzht3oL0odVsWSPS0XuyexDe5qVwm3gZYYUK
rcRPMx/q8Vads4xB62Y0typC7xRjMkPphUOTz2mvkAuo7BsBV4iogHVt3r7Ie4hz
DzOKokPBnuY7G/qzd+rlMixF3ZhIMogUPdPFHvHCv9D8Rx+Bh5fwsmRf6D4gj11M
rj8qrhfE/hLbhWTNAbra/vcjvhpLRyI=
-----END CERTIFICATE-----
subject=/C=US/ST=VA/L=Arlington/O=Surescripts/OU=Operations/CN=shbr-staging.surescripts.net/emailAddress=network@surescripts.com
issuer=/C=US/O=Surescripts LLC./OU=Surescripts Certification Authorities/CN=Surescripts Issuing Certification Authority
---
Acceptable client certificate CA names
/C=US/O=Surescripts LLC./OU=Surescripts Certification Authorities/CN=Surescripts Issuing Certification Authority
/C=US/O=Surescripts LLC./OU=Surescripts Certification Authorities/CN=Surescripts Root Certification Authority
/C=US/ST=Arizona/L=Scottsdale/O=GoDaddy.com, Inc./OU=http://certs.godaddy.com/repository//CN=Go Daddy Secure Certificate Authority - G2
/C=US/ST=Arizona/L=Scottsdale/O=GoDaddy.com, Inc./CN=Go Daddy Root Certificate Authority - G2
/C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO High-Assurance Secure Server CA
/C=SE/O=AddTrust AB/OU=AddTrust External TTP Network/CN=AddTrust External CA Root
/C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO RSA Organization Validation Secure Server CA
/C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO RSA Certification Authority
Client Certificate Types: RSA sign, DSA sign, ECDSA sign
Requested Signature Algorithms: RSA+SHA256:DSA+SHA256:ECDSA+SHA256:RSA+SHA384:DSA+SHA384:ECDSA+SHA384:RSA+SHA512:DSA+SHA512:ECDSA+SHA512:RSA+SHA1:DSA+SHA1:ECDSA+SHA1
Shared Requested Signature Algorithms: RSA+SHA256:DSA+SHA256:ECDSA+SHA256:RSA+SHA384:DSA+SHA384:ECDSA+SHA384:RSA+SHA512:DSA+SHA512:ECDSA+SHA512:RSA+SHA1:DSA+SHA1:ECDSA+SHA1
Peer signing digest: SHA256
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 5394 bytes and written 4213 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES256-GCM-SHA384
    Session-ID: 
    Session-ID-ctx: 
    Master-Key: 3C3375DE1FAA01C19C37DC6FA4038B4FEA350FF32A4C83F55444A6B63CC95D49F6601B01D9D69BA118BF56EDA9D30709
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1472462285
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)
---

What am i doing wrong ?

(1) are you sure cert2 is your end entity (client) cert? There is no standard order for p7b chains but upward is common. Show subject and issuer for each. (2) are you sure your curl uses OpenSSL? If not file format may be different. Show `curl -V` (uppercase). — dave_thompson_085, Aug 25 '16 at 15:54
(1) Subject and issuer for each certificate has been added in step 6 (2) Show curl -V output has been added in step 8 .. thank you @dave_thompson_085 for pointing me in the right direction. I pass the handshake phase now but I am stuck at "Unknown SSL protocol error " I have also updated the error message in Step 9 — Vikas kedia, Aug 25 '16 at 17:41
(Sorry for delay; power out here, took a while to catch up.) Reject by server after client's second flight is _likely_ to mean server doesn't like your cert, although it _should_ use an alert for that not 'unknown error'. Try `openssl s_client -connect host:port -key file -cert file -servername host -state` to see if it gives a more specific error (like "EOF"=FIN or RST). Also `s_client` will log `Acceptable client certificate CA names`; check the value(s) under that match the cert/chain you are using. If it matches the root (only), it _might want you to provide the intermediate ... — dave_thompson_085, Aug 29 '16 at 03:27
... (even though that's logically unnecessary); if so add cert1 to your default truststore (either in the combined-file or with a hashnamed-link or filename in the directory), or a copy of said truststore specified with `s_client -CAfile/-CApath` or possibly `curl --cacert/--capath` as applicable; `s_client` will definitely add from truststore to the client chain (even though that seems backwards) and I'm pretty sure curl-using-openssl will also (but haven't tested). PS: If `s_client` does connect successfully, just type `Q` (uppercase Q return) to exit without sending any bad request. — dave_thompson_085, Aug 29 '16 at 03:31
In step12 I have added the output of $openssl s_client -connect shbr-staging.surescripts.net:443 -key sc-private-key-without-password.pem -cert cert2.pem -servername drone.savantcare.com -state — Vikas kedia, Aug 29 '16 at 09:26
(1) `s_client ... -servername host` should be the host you are connecting _to_, although it depends on the server whether it's needed and here it appears yours doesn't (2) The `unknown state` and `error in unknown state` definitely shouldn't happen; this suggests something is wrong with the OpenSSL build you are using. I notice you refer to some different OpenSSL versions; if these are on the same system, make sure you are using a consistent combination of executable and (dynamic) libraries. If that doesn't help, give details about your (distro&) package or other build source/method. ... — dave_thompson_085, Aug 30 '16 at 05:45
... (3) On the good(?) side, `errno=104` on Linux is definitely 'connection reset' aka RST, an abnormal close by the server. (4) The `client CAs` info does look right to me. So in short, I don't see any obvious reason the server should be rejecting this. Can you obtain or request information from the server or server admins, like log entries? Is there a pre-staging or alternate or backup server you can try instead? Usual practice is you can't go to prod without succeeding in staging first, but is there is any option here? — dave_thompson_085, Aug 30 '16 at 05:53
Oh, and can you try adding the intermediate cert to the truststore for `s_client` as I suggested before? I don't think it's _likely_ it to help, but it's cheap and fast to try. — dave_thompson_085, Aug 30 '16 at 05:56

curl SSL protocol error in connection to shbr-staging.surescripts.net

0 Answers0