Openssl DSA sign

Question

I am trying to sign some data using a DSA certificate. I am saving the certificate in memory (it was generated with the openssl gendsa command).

My function looks like this, and my problem is with res = EVP_SignFinal. Here the function returns 0, and sets the signature_len on 0.

bool my_dsa_sign(const Certificate& certificate, const char* messageData, size_t messageLength, Signature &outSignature) {
    bool resultOk = false;
    BIO* bio = BIO_new_mem_buf((void*) certificate.getPEMData(), certificate.getSize());
    if(NULL != bio) {
        EVP_PKEY *pkey = NULL;
        if(NULL != (pkey = PEM_read_bio_PrivateKey(bio, NULL, NULL, NULL))) {
            unsigned int signature_len;
            EVP_MD_CTX ctx;
            int res = EVP_SignInit(&ctx, EVP_sha512());
            if(1 == res) {
                res = EVP_SignUpdate(&ctx, messageData, messageLength);
                if(1 == res) {
                    unsigned char* s = (unsigned char*)malloc(EVP_PKEY_size(pkey));

                    // problem here
                    res = EVP_SignFinal(&ctx, s, &signature_len, pkey);
                    if(1 == res) {
                        resultOk = true;
                        signature.setData(s, signature_len);
                    }
                }
            }
            EVP_PKEY_free(pkey);
            EVP_MD_CTX_cleanup(&ctx);
        }
        BIO_free(bio);
    }
    return resultOk;
}

any idea what the problem might be ? I have checked with the debugger, the value of the certificate is correct, in the form:

-----BEGIN DSA PRIVATE KEY-----
MIID....
.......
-----END DSA PRIVATE KEY-----

Also the length is correct, matches with that of the file.

Just for terminology: A *certificate* is a (more or less public) piece of data containing the public key, some describing data and a signature (usually by someone else), while your private key is separate from this (and not contained in the certificate). — Paŭlo Ebermann, Dec 04 '11 at 20:13

Paŭlo Ebermann · Accepted Answer · 2016-04-02T15:56:12.583

Not knowing much of OpenSSL, I don't see anywhere that you are using DSA in your code.

int res = EVP_SignInit(&ctx, EVP_sha512());

The manual page to EVP_SignInit says:

EVP_SignInit() initializes a signing context ctx to use the default implementation of digest type.

The manual page to EVP_sha512 says:

[...], EVP_sha512(), [...] return EVP_MD structures for the [...], SHA512 , [...] digest algorithms respectively. The associated signature algorithm is RSA in each case.

Highlight by me. So it looks like you are trying to do an RSA signature with a DSA key, which will not work (or spit out nonsense).

After some browsing through the documentation, I didn't quite find a way to use SHA512 together with DSA, though, even though the documentation now states:

The link between digests and signing algorithms was fixed in OpenSSL 1.0 and later, so now EVP_sha1() can be used with RSA and DSA, there is no need to use EVP_dss1() any more.

You are right, SHA512 cannot be used with DSA, I didn't think the hashing algorithm will be important. I changed it to SHA1 and it works. Thanks a lot, I spent almost an entire day looking in the wrong direction. — , Dec 04 '11 at 22:57
Also the fact that I use DSA is only important at the end when using EVP_SignUpdate since the EVP_KEY determines if I do DSA or RSA signing. The PEM_read_bio_PrivateKey method handles all formats transparently. — , Dec 04 '11 at 23:24

Openssl DSA sign

1 Answers1