Socket self signed certificate handshake failure

Question

I am using Starscream socket library and am trying to use WSS however I am having a handshake failure. I got my self signed certificate, I converted it to a .der file. Here is the code I am trying

 var socket = WebSocket(url: URL(string: "wss://192.168.1.130:6223")!, protocols: [])

    override func viewDidLoad() {
        super.viewDidLoad()
        do
        {
            let urlPath     = Bundle.main.path(forResource: "my_cert", ofType: "der")
            let url         = NSURL.fileURL(withPath: urlPath!)
            let certificateData = try Data(contentsOf: url)

            let certificate: SecCertificate =
                SecCertificateCreateWithData(kCFAllocatorDefault, certificateData as CFData)!

            var trust: SecTrust?
            let policy = SecPolicyCreateBasicX509()
            let status = SecTrustCreateWithCertificates(certificate, policy, &trust)
            if status == errSecSuccess {
                let key = SecTrustCopyPublicKey(trust!)!;
                let ssl =  SSLCert(key: key)
                socket.security = SSLSecurity(certs: [ssl], usePublicKeys: true)
                socket.delegate = self
                socket.connect()
            }

        }catch let error as NSError
        {
            print(error)
        }
    }

So when I try to connect, I get the following error message

2017-07-07 11:06:26.590 CertificateTesting[5180:81661] CFNetwork SSLHandshake failed (-9807) websocket is disconnected: The operation couldn’t be completed. (OSStatus error -9807.)

The certificate should work fine, my Android colleague has tried it on his side and has had no issues. The only way I can get it working on my side is if I disable SSL validation like so

 socket.disableSSLCertValidation = true

Does anyone have any experience using self signed SSL with sockets. Any information would be much appreciated.

Edit:

I called verify ssl command, it returns

➜  CertificateTesting git:(master) ✗ openssl verify -my_cert.der ca-cert.pem server-cert.pem
usage: verify [-verbose] [-CApath path] [-CAfile file] [-purpose purpose] [-crl_check] [-engine e] cert1 cert2 ...
recognized usages:
    sslclient   SSL client
    sslserver   SSL server
    nssslserver Netscape SSL server
    smimesign   S/MIME signing
    smimeencrypt    S/MIME encryption
    crlsign     CRL signing
    any         Any Purpose
    ocsphelper  OCSP helper

Does that look okay?

`openssl verify -CAfile ca-cert.pem server-cert.pem` returns ok? — Ulug Toprak, Jul 07 '17 at 10:16
I updated my question with the verify SSL response, can you check if that is okay? — AdamM, Jul 07 '17 at 10:27
Yes actually, which is strange. You could check if you can access the server from a web browser perhaps? Also i would try to uninstalling the device configuration profile and create a fresh one before i would investigate further. — Ulug Toprak, Jul 07 '17 at 10:34

score 0 · Answer 1 · answered Mar 15 '20 at 16:35

You can give a try using the common name in the WebSocket instead of IP.

var socket = WebSocket(url: URL(string: "wss://192.168.1.130:6223")!, protocols: [])

You can verify the common name in the certificate using the command

openssl x509 -in <certificate file> -text

Validate the SSL handshake using following command

openssl s_client -host <common name mentioned in the cert> -port <port> -cert <client_cert file> -key <client_key file> -CAfile <ca_cert file>

Giuseppe Iacobucci · Answer 2 · 2020-03-16T12:23:19.757

in an old swift app, i had the same problem using REST API and WS exposed by a self signed certificate.

Websocket protocol is embedded in HTTP/S protocol, also the handshake.

So generate the certificate .cert: echo -n | openssl s_client -connect yoururl:443 | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > name_youwant.cert

Import the .cert file in the asset of the app that will use it.

In the class that estabilish the websocket connection, implement URLSessionDelegate.

And then use this logic to validate the self signed certificate:

Note: NSBundle.mainBundle().pathForResource(Config.certificate, ofType: ".cert") -> Config.certificate is a static string that indicate the name of file.

    // uncomment if self signed certificate is used on the backend
    public func URLSession(session: NSURLSession, didReceiveChallenge challenge: NSURLAuthenticationChallenge, completionHandler: (NSURLSessionAuthChallengeDisposition, NSURLCredential?) -> Void) {
        let serverTrust = challenge.protectionSpace.serverTrust
        let certificate = SecTrustGetCertificateAtIndex(serverTrust!, 0)
        let certificateData = SecCertificateCopyData(certificate!)
        let remoteCertificateData = certificateData as NSData
        let cerPath = NSBundle.mainBundle().pathForResource(Config.certificate, ofType: ".cert")

        let localCertData = NSData(contentsOfFile: cerPath!)!
        if localCertData.isEqualToData(remoteCertificateData) {
            completionHandler(NSURLSessionAuthChallengeDisposition.UseCredential,NSURLCredential(forTrust:
                challenge.protectionSpace.serverTrust!))
        }else{
            completionHandler(NSURLSessionAuthChallengeDisposition.CancelAuthenticationChallenge,nil)
       }

    }

Hope this helps. Regards,

Socket self signed certificate handshake failure

2 Answers2