C++ OpenSSL: libssl fails to verify certificates on Windows

Question

I've done a lot of looking around but I can't seem to find a decent solution to this problem. Many of the StackOverflow posts are regarding Ruby, but I'm using OpenSSL more or less directly (via the https://gitlab.com/eidheim/Simple-Web-Server library) for a C++ application/set of libraries, and need to work out how to fix this completely transparently for users (they should not need to hook up any custom certificate verification file in order to use the application).

On Windows, when I attempt to use the SimpleWeb HTTPS client, connections fail if I have certificate verification switched on, because the certificate for the connection fails to validate. This is not the case on Linux, where verification works fine.

I was advised to follow this solution to import the Windows root certificates into OpenSSL so that they could be used by the verification routines. However, this doesn't seem to make any difference as far as I can see. I have dug into the guts of the libssl verification functions to try and understand exactly what's going on, and although the above answer recommends adding the Windows root certificates to a new X509_STORE, it appears that the SSL connection context has its own store which is set up when the connection is initialised. This makes me think that simply creating a new X509_STORE and adding certificates there is not helping because the connection doesn't actually use that store.

It may well be that I've spent so much time debugging the minutiae of libssl that I'm missing what the actual approach to solving this problem should be. Does OpenSSL provide a canonical way of looking up system certificates that I'm not setting? Alternatively, could the issue be the way that the SimpleWeb library/ASIO is initialising OpenSSL? I know that the library allows you to provide a path for a "verify file" for certificates, but I feel like this wouldn't be an appropriate solution since I as a developer should be using the certificates found on the end user's system, rather than hard-coding my own.

EDIT: For context, this is the code I'm using in a tiny example application:

#define MY_ENCODING_TYPE  (PKCS_7_ASN_ENCODING | X509_ASN_ENCODING)

static void LoadSystemCertificates()
{
    HCERTSTORE hStore;
    PCCERT_CONTEXT pContext = nullptr;
    X509 *x509 = nullptr;
    X509_STORE *store = X509_STORE_new();

    hStore = CertOpenSystemStore(NULL, "ROOT");

    if (!hStore)
    {
        return;
    }

    while ((pContext = CertEnumCertificatesInStore(hStore, pContext)) != nullptr)
    {
        const unsigned char* encodedCert = reinterpret_cast<const unsigned char*>(pContext->pbCertEncoded);
        x509 = d2i_X509(nullptr, &encodedCert, pContext->cbCertEncoded);

        if (x509)
        {
            X509_STORE_add_cert(store, x509);
            X509_free(x509);
        }
    }

    CertCloseStore(hStore, 0);
}

static void MakeRequest(const std::string& address)
{
    using Client = SimpleWeb::Client<SimpleWeb::HTTPS>;

    Client httpsClient(address);
    httpsClient.io_service = std::make_shared<asio::io_service>();

    std::cout << "Making request to: " << address << std::endl;

    bool hasResponse = false;
    httpsClient.request("GET", [address, &hasResponse](std::shared_ptr<Client::Response> response, const SimpleWeb::error_code& error)
    {
        hasResponse = true;

        if ( error )
        {
            std::cerr << "Got error from " << address << ": " << error.message() << std::endl;
        }
        else
        {
            std::cout << "Got response from " << address << ":\n" << response->content.string() << std::endl;
        }
    });

    while ( !hasResponse )
    {
        httpsClient.io_service->poll();
        httpsClient.io_service->reset();

        std::this_thread::sleep_for(std::chrono::milliseconds(20));
    }
}

int main(int, char**)
{
    LoadSystemCertificates();
    MakeRequest("google.co.uk");

    return 0;
}

The call returns me: Got error from google.co.uk: certificate verify failed

Answer 1

OK, to anyone who this might help in future, this is how I solved this issue. This answer to a related question helped.

It turns out that the issue was indeed that the SSL context was not making use of the certificate store that I'd set up. Everything else was OK, bu the missing piece of the puzzle was a call to SSL_CTX_set_cert_store(), which takes the certificate store and provides it to the SSL context.

In the context of the SimpleWeb library, the easiest way to do this appeared to be to subclass the SimpleWeb::Client<SimpleWeb::HTTPS> class and add the following to the constructor:

#define WIN32_LEAN_AND_MEAN
#include <Windows.h>
#include <wincrypt.h>

class MyClient : public SimpleWeb::Client<SimpleWeb::HTTPS>
{
public:
    MyClient( /* ... */ ) :
        SimpleWeb::Client<SimpleWeb::HTTPS>( /* ... */ )
    {
        AddWindowsRootCertificates();
    }

private:
    using OpenSSLContext = asio::ssl::context::native_handle_type;

    void AddWindowsRootCertificates()
    {
        // Get the SSL context from the SimpleWeb class.
        OpenSSLContext sslContext = context.native_handle();
        
        // Get a certificate store populated with the Windows root certificates.
        // If this fails for some reason, the function returns null.
        X509_STORE* certStore = GetWindowsCertificateStore();

        if ( sslContext && certStore )
        {
            // Set this store to be used for the SSL context.
            SSL_CTX_set_cert_store(sslContext, certStore);
        }
    }

    static X509_STORE* GetWindowsCertificateStore()
    {
        // To avoid populating the store every time, we keep a static
        // pointer to the store and just initialise it the first time
        // this function is called.
        static X509_STORE* certificateStore = nullptr;

        if ( !certificateStore )
        {
            // Not initialised yet, so do so now.

            // Try to open the root certificate store.
            HCERTSTORE rootStore = CertOpenSystemStore(0, "ROOT");

            if ( rootStore )
            {
                // The new store is reference counted, so we can create it
                // and keep the pointer around for later use.
                certificateStore = X509_STORE_new();
                
                PCCERT_CONTEXT pContext = nullptr;

                while ( (pContext = CertEnumCertificatesInStore(rootStore, pContext)) != nullptr )
                {
                    // d2i_X509() may modify the pointer, so make a local copy.
                    const unsigned char* content = pContext->pbCertEncoded;
                    
                    // Convert the certificate to X509 format.
                    X509 *x509 = d2i_X509(NULL, &content, pContext->cbCertEncoded);

                    if ( x509 )
                    {
                        // Successful conversion, so add to the store.
                        X509_STORE_add_cert(certificateStore, x509);
                        
                        // Release our reference.
                        X509_free(x509);
                    }
                }

                // Make sure to close the store.
                CertCloseStore(rootStore, 0);
            }
        }

        return certificateStore;
    }
};

Obviously GetWindowsCertificateStore() would need to be abstracted out to somewhere platform-specific if your class needs to compile on multiple platforms.