Misunderstanding CSR and X.509 functioning

Question

After spending two days searching and finding answers to my questions only to realize these answers were only creating more questions and problems I couldn't solve, I resolved to ask for help here hoping I hadn't missed the information I'm looking for somewhere.

I've just began a project in which I have to exchange X.509 certificates with a client before starting to work with this client. I've managed to understand well how and why X.509 certificate were made :

• Make a CSR from a private key;
• This CSR contains digital identity and a public key extracted from private key;
• CSR is self-signed or sent to a CA who signs it with its own private key in order to validate the identity of the sender;
• The CA sends back a certificate matching X.509 standard and proving you're not faking an identity and thus that the public key contained in X.509 is reliable.

So X.509 certificate is a way to exchange your public key with someone in order to counter a potential man-in-the-middle attack. Using OpenSSL I've been able to simulate alone those operations in order to understand the functioning of these steps.

My first problem is that when I make a CSR from a private key (privateKey.pem) and then check if this private key matches the CSR, it doesn't.
I then self-sign the CSR with a different private key (signingPrivateKey.pem), check again if the X.509 certificate matches the private key, and it still does not (which seems normal because the CSR didn't either but both should match, shouldn't they ?).

Here are the OpenSSL commands :

@echo off

echo "Generates private key and puts it in SigningPrivateKey.pem"
echo.
openssl genrsa -out SigningPrivateKey.pem 1024

echo "Generates private key and puts it in privateKey.pem, then create CSR"
echo.
openssl req -newkey rsa:1024 -keyout privateKey.pem -out CSR_TEST.csr -nodes


echo "Checks if CSR and privateKey matches
echo.
openssl x509 -noout -modulus -in CSR_TEST.csr | openssl md5
openssl rsa -noout -modulus -in privateKey.pem | openssl md5

echo "Self sign CSR with private key"
echo.
openssl x509 -in CSR_TEST.csr -out CSR_TEST.pem -req -signkey 
SigningPrivateKey.pem -days 1

echo. "Checks again if certificate and private key matches
openssl x509 -noout -modulus -in CSR_TEST.pem | openssl md5
openssl rsa -noout -modulus -in privateKey.pem | openssl md5  

Here's the summarized input :

stdin = d41d8....8427e  
stdin = f3213....4538c  
Signature ok  
stdin = 82baf...a0863  
stdin = f3213...4538c

My second problem is the way to verify a X.509. Let's say I manage to make a X.509 certificate from an input private key and the extracted public key matches with it. I send it to my client, who sends a certificate to. How do we know this X.509 is reliable ? I've read somewhere that we need to "unsign" the X.509 certificate with the CA's public key, and compare the output with a hashed certificate, but everyone seems to say that an X.509 certificate is self-sufficient and that it proves by itself your identity. We'd just need to check its authenticity with an OpenSSL command ? Please lighten me on this point.

Thanks for taking the time to read and understand my problems, once again I hope I'm not the only one facing those problems and that this post will help someone else. Sorry for I'm not allowed yet to put more links in this post, otherwise I'd have been more accurate.

Answer 1

CSR/Key comparison

$ openssl req -in test.csr -modulus -noout | openssl md5
(stdin)= 76d44c1a05f535f5e78a648b41bdaf73
$ openssl rsa -in test.key -modulus -noout | openssl md5
(stdin)= 76d44c1a05f535f5e78a648b41bdaf73

You used openssl x509 on a CSR. They're not the same. And you clearly didn't look at the output before running it through MD5:

$ openssl x509 -in test.csr -modulus -noout
unable to load certificate
139958187611800:error:0906D06C:PEM routines:PEM_read_bio:no start line:pem_lib.c:701:Expecting: TRUSTED CERTIFICATE

You haven't made it into a certificate yet, so the x509 command doesn't work.

Your after you do make the certificate and you check again, you're getting a different answer because you used -signkey. The documentation says that -signkey is used as a self-sign key, meaning it replaces the public key that was in the original request. If you use -signkey privateKey.pem your commands should show alignment.

Integrity

Each X.509 certificate contains the presented data (including the public key) and a signature. For "real" CAs there's usually a note (the Authority Information Access extension) which says how to find the CA's certificate. Whether you have it on-hand or you have to go retrieve it from the internet, the public key in the signing certificate can be used to validate that the data in the original certificate hasn't changed.

The signing CA itself had a certificate, so you can go another step and prove that it hasn't been altered.

Eventually you get to a root/self-signed certificate. Either you already have it and consider it trusted, or you don't, and you call the whole thing malarkey.

The only thing that can really be verified about a self-signed certificate which isn't part of your inbuilt trust store is that it wasn't modified after it was created. The only security benefit of this is repudiation: If someone edited a certificate you created and added an extension which says something to the effect of "Your mother is a smelly donkey salesman!" you can demonstrate that the self-signed signature is invalid.

$ openssl x509 -in test.cer -text -noout
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 11195357966677484939 (0x9b5de6c15126a58b)
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: C=US, ST=Washington, L=Redmond, O=Microsoft Corporation, OU=.NET Framework (CoreFX), CN=localhost
        Validity
            Not Before: Mar  2 01:48:00 2016 GMT
            Not After : Mar  2 01:48:00 2017 GMT
        Subject: C=US, ST=Washington, L=Redmond, O=Microsoft Corporation, OU=.NET Framework (CoreFX), CN=localhost
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)
                Modulus:
                    00:af:81:c1:cb:d8:20:3f:62:4a:53:9e:d6:60:81:
                    75:37:23:93:a2:83:7d:48:90:e4:8a:19:de:d3:69:
                    73:11:56:20:96:8d:6b:e0:d3:da:a3:8a:a7:77:be:
                    02:ee:0b:6b:93:b7:24:e8:dc:c1:2b:63:2b:4f:a8:
                    0b:bc:92:5b:ce:62:4f:4c:a7:cc:60:63:06:b3:94:
                    03:e2:8c:93:2d:24:dd:54:6f:fe:4e:f6:a3:7f:10:
                    77:0b:22:15:ea:8c:bb:5b:f4:27:e8:c4:d8:9b:79:
                    eb:33:83:75:10:0c:5f:83:e5:5d:e9:b4:46:6d:df:
                    be:ee:42:53:9a:ef:33:ef:18:7b:77:60:c3:b1:a1:
                    b2:10:3c:2d:81:44:56:4a:0c:10:39:a0:9c:85:cf:
                    6b:59:74:eb:51:6f:c8:d6:62:3c:94:ae:3a:5a:0b:
                    b3:b4:c7:92:95:7d:43:23:91:56:6c:f3:e2:a5:2a:
                    fb:0c:14:2b:9e:06:81:b8:97:26:71:af:2b:82:dd:
                    39:0a:39:b9:39:cf:71:95:68:68:7e:49:90:a6:30:
                    50:ca:77:68:dc:d6:b3:78:84:2f:18:fd:b1:f6:d9:
                    ff:09:6b:af:7b:eb:98:dc:f9:30:d6:6f:cf:d5:03:
                    f5:8d:41:bf:f4:62:12:e2:4e:3a:fc:45:ea:42:bd:
                    88:47
                Exponent: 8589935681 (0x200000441)
        X509v3 extensions:
            X509v3 Subject Key Identifier:
                78:A5:C7:5D:51:66:73:31:D5:A9:69:24:11:4C:9B:5F:A0:0D:7B:CB
            X509v3 Authority Key Identifier:
                keyid:78:A5:C7:5D:51:66:73:31:D5:A9:69:24:11:4C:9B:5F:A0:0D:7B:CB

            X509v3 Basic Constraints:
                CA:TRUE
    Signature Algorithm: sha256WithRSAEncryption
         77:75:6d:05:ff:a6:ad:fe:d5:b6:d4:af:b5:40:84:0c:6d:01:
         cf:6b:3f:a6:c9:73:df:d6:1f:ca:a0:a8:14:fa:1e:24:69:01:
         9d:94:b1:d8:56:d0:7d:d2:b9:5b:85:50:df:d2:08:59:53:a4:
         94:b9:9e:fc:ba:a7:98:2c:e7:71:98:4f:9d:4a:44:5f:fe:e0:
         62:e8:a0:49:73:6a:39:fd:99:4e:1f:da:0a:5d:c2:b5:b0:e5:
         7a:0b:10:c4:1b:c7:fe:6a:40:b2:4f:85:97:73:02:59:3e:60:
         b9:8d:d4:81:1d:47:d9:48:ed:f8:d6:e6:b5:af:80:a1:82:74:
         96:e2:0b:fd:24:0e:46:76:74:50:4d:4e:47:03:33:1d:64:70:
         5c:36:fb:6e:14:ba:bf:d9:cb:ee:c4:4b:33:a8:d7:b3:64:79:
         90:0f:3c:5b:ba:b6:9c:5e:45:3d:18:07:83:e2:50:80:51:b9:
         98:c0:38:e4:62:25:71:d2:ab:89:1d:89:8e:54:58:82:8c:f1:
         86:79:51:7d:28:db:ca:bf:72:e8:13:07:bf:d7:21:b7:3d:db:
         17:51:12:3f:99:d8:fc:0d:53:37:98:c4:db:d1:47:19:d5:d8:
         a8:5b:00:a1:44:a3:67:67:7b:48:89:1a:9b:56:f0:45:33:48:
         11:ba:cb:7a

And

$ openssl x509 -in alsotest.cer -text -noout
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 11195357966677484939 (0x9b5de6c15126a58b)
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: C=US, ST=Washington, L=Redmond, O=Microsoft Corporation, OU=.NET Framework (CoreFX), CN=localhost
        Validity
            Not Before: Mar  2 01:48:00 2016 GMT
            Not After : Mar  2 01:48:00 2019 GMT
        Subject: C=US, ST=Washington, L=Redmond, O=Microsoft Corporation, OU=.NET Framework (CoreFX), CN=localhost
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)
                Modulus:
                    00:af:81:c1:cb:d8:20:3f:62:4a:53:9e:d6:60:81:
                    75:37:23:93:a2:83:7d:48:90:e4:8a:19:de:d3:69:
                    73:11:56:20:96:8d:6b:e0:d3:da:a3:8a:a7:77:be:
                    02:ee:0b:6b:93:b7:24:e8:dc:c1:2b:63:2b:4f:a8:
                    0b:bc:92:5b:ce:62:4f:4c:a7:cc:60:63:06:b3:94:
                    03:e2:8c:93:2d:24:dd:54:6f:fe:4e:f6:a3:7f:10:
                    77:0b:22:15:ea:8c:bb:5b:f4:27:e8:c4:d8:9b:79:
                    eb:33:83:75:10:0c:5f:83:e5:5d:e9:b4:46:6d:df:
                    be:ee:42:53:9a:ef:33:ef:18:7b:77:60:c3:b1:a1:
                    b2:10:3c:2d:81:44:56:4a:0c:10:39:a0:9c:85:cf:
                    6b:59:74:eb:51:6f:c8:d6:62:3c:94:ae:3a:5a:0b:
                    b3:b4:c7:92:95:7d:43:23:91:56:6c:f3:e2:a5:2a:
                    fb:0c:14:2b:9e:06:81:b8:97:26:71:af:2b:82:dd:
                    39:0a:39:b9:39:cf:71:95:68:68:7e:49:90:a6:30:
                    50:ca:77:68:dc:d6:b3:78:84:2f:18:fd:b1:f6:d9:
                    ff:09:6b:af:7b:eb:98:dc:f9:30:d6:6f:cf:d5:03:
                    f5:8d:41:bf:f4:62:12:e2:4e:3a:fc:45:ea:42:bd:
                    88:47
                Exponent: 8589935681 (0x200000441)
        X509v3 extensions:
            X509v3 Subject Key Identifier:
                78:A5:C7:5D:51:66:73:31:D5:A9:69:24:11:4C:9B:5F:A0:0D:7B:CB
            X509v3 Authority Key Identifier:
                keyid:78:A5:C7:5D:51:66:73:31:D5:A9:69:24:11:4C:9B:5F:A0:0D:7B:CB

            X509v3 Basic Constraints:
                CA:TRUE
    Signature Algorithm: sha256WithRSAEncryption
         77:75:6d:05:ff:a6:ad:fe:d5:b6:d4:af:b5:40:84:0c:6d:01:
         cf:6b:3f:a6:c9:73:df:d6:1f:ca:a0:a8:14:fa:1e:24:69:01:
         9d:94:b1:d8:56:d0:7d:d2:b9:5b:85:50:df:d2:08:59:53:a4:
         94:b9:9e:fc:ba:a7:98:2c:e7:71:98:4f:9d:4a:44:5f:fe:e0:
         62:e8:a0:49:73:6a:39:fd:99:4e:1f:da:0a:5d:c2:b5:b0:e5:
         7a:0b:10:c4:1b:c7:fe:6a:40:b2:4f:85:97:73:02:59:3e:60:
         b9:8d:d4:81:1d:47:d9:48:ed:f8:d6:e6:b5:af:80:a1:82:74:
         96:e2:0b:fd:24:0e:46:76:74:50:4d:4e:47:03:33:1d:64:70:
         5c:36:fb:6e:14:ba:bf:d9:cb:ee:c4:4b:33:a8:d7:b3:64:79:
         90:0f:3c:5b:ba:b6:9c:5e:45:3d:18:07:83:e2:50:80:51:b9:
         98:c0:38:e4:62:25:71:d2:ab:89:1d:89:8e:54:58:82:8c:f1:
         86:79:51:7d:28:db:ca:bf:72:e8:13:07:bf:d7:21:b7:3d:db:
         17:51:12:3f:99:d8:fc:0d:53:37:98:c4:db:d1:47:19:d5:d8:
         a8:5b:00:a1:44:a3:67:67:7b:48:89:1a:9b:56:f0:45:33:48:
         11:ba:cb:7a

To spare you the eye strain: One of them has an expiration in the current past, another has an expiration date in the current future. (I was too lazy to inject an extension).

Which one is correct?

$ openssl verify test.cer
test.cer: C = US, ST = Washington, L = Redmond, O = Microsoft Corporation, OU = .NET Framework (CoreFX), CN = localhost
error 18 at 0 depth lookup:self signed certificate
C = US, ST = Washington, L = Redmond, O = Microsoft Corporation, OU = .NET Framework (CoreFX), CN = localhost
error 10 at 0 depth lookup:certificate has expired
OK

$ openssl verify alsotest.cer
alsotest.cer: C = US, ST = Washington, L = Redmond, O = Microsoft Corporation, OU = .NET Framework (CoreFX), CN = localhost
error 18 at 0 depth lookup:self signed certificate
OK

Drat, they're both correct. Or are they? Wait, openssl verify doesn't check self-signed signatures normally, because it usually doesn't matter.

$ openssl verify -check_ss_sig test.cer
test.cer: C = US, ST = Washington, L = Redmond, O = Microsoft Corporation, OU = .NET Framework (CoreFX), CN = localhost
error 18 at 0 depth lookup:self signed certificate
C = US, ST = Washington, L = Redmond, O = Microsoft Corporation, OU = .NET Framework (CoreFX), CN = localhost
error 10 at 0 depth lookup:certificate has expired
OK

$ openssl verify -check_ss_sig alsotest.cer
alsotest.cer: C = US, ST = Washington, L = Redmond, O = Microsoft Corporation, OU = .NET Framework (CoreFX), CN = localhost
error 18 at 0 depth lookup:self signed certificate
C = US, ST = Washington, L = Redmond, O = Microsoft Corporation, OU = .NET Framework (CoreFX), CN = localhost
error 7 at 0 depth lookup:certificate signature failure
140450704717464:error:04091068:rsa routines:INT_RSA_VERIFY:bad signature:rsa_sign.c:278:
140450704717464:error:0D0C5006:asn1 encoding routines:ASN1_item_verify:EVP lib:a_verify.c:218:

There we go. alsotest.cer was modified (without being resigned) to extend its expiration date.