How to Sign with RSA private key by CNG

Question

1. How to sign with RSA private key? (SOLVED)

I followed this sample to sign data with private key. I only reserve hash and sign functions, and use ECDSA P-256 private key like the sample.

Here are my steps and the program work fine:

BCryptOpenAlgorithmProvider()
Hash some data
NCryptOpenStorageProvider()
NCryptCreatePersistedKey()
NCryptFinalizeKey()
NCryptSignHash()
NCryptSignHash()

I tried to sign with RSA private key, replace NCRYPT_ECDSA_P256_ALGORITHM with NCRYPT_RSA_ALGORITHM and NCRYPT_RSA_SIGN_ALGORITHM, but it failed in second NCryptSignHash() with NTE_INVALID_PARAMETER.

    //create a persisted key
    if (FAILED(secStatus = NCryptCreatePersistedKey(
        hProv,
        &hKey,
        NCRYPT_RSA_ALGORITHM,
        L"my RSA key",
        0,
        0)))
    {
        wprintf(L"**** Error 0x%x returned by NCryptCreatePersistedKey\n", secStatus);
        goto Cleanup;
    }

2. Why the program can't find the private key after first run?

The private key is created in the program, I tried to sign with X509 certificate which has private key(ECDSA P-256) in My certifacate store.

Here are my steps and the program only work in the first run. It failed in CryptAcquireCertificatePrivateKey() with NTE_BAD_KEYSET error

BCryptOpenAlgorithmProvider()
Hash some data
CertOpenStore()
CertFindCertificateInStore()
CryptAcquireCertificatePrivateKey()
NCryptSignHash()
NCryptSignHash()

   // Open the certificate store.
   if (!(hCertStore = CertOpenStore(
       CERT_STORE_PROV_SYSTEM,
       0,
       NULL,
       CERT_SYSTEM_STORE_CURRENT_USER,
       CERT_STORE_NAME)))
   {
       MyHandleError(const_cast<LPTSTR>("The MY store could not be opened."));
   }
   swprintf(wMY_SIGNER_NAME, 100, L"%hs", MY_SIGNER_NAME);
   if (pSignerCert = CertFindCertificateInStore(
       hCertStore,
       MY_ENCODING_TYPE,
       0,
       CERT_FIND_SUBJECT_STR,
       wMY_SIGNER_NAME,
       NULL))
   {
       //_tprintf(TEXT("The signer's certificate was found.\n"));
   }
   else
   {
       MyHandleError(const_cast<LPTSTR>("Signer certificate not found."));
   }

   if (CryptAcquireCertificatePrivateKey(
       pSignerCert, 
       CRYPT_ACQUIRE_PREFER_NCRYPT_KEY_FLAG, NULL, 
       &hKey,
       &dwKeySpec, 
       false))
   {
       printf("Get Priv OK\n");
   }
   else
   {
       if (GetLastError() == NTE_BAD_KEYSET)
           printf("NTE_BAD_KEYSET\n");
   }

UPDATE:

    if (FAILED(secStatus = NCryptSignHash(
        hKey,
        0,
        pbHash,
        cbHash,
        NULL,
        0,
        &cbSignature,
        0)))
    {
        wprintf(L"**** Error 0x%x returned by NCryptSignHash1\n", secStatus);
        goto Cleanup;
    }

    //allocate the signature buffer
    pbSignature = (PBYTE)HeapAlloc(GetProcessHeap(), 0, cbSignature);
    if (NULL == pbSignature)
    {
        wprintf(L"**** memory allocation failed\n");
        goto Cleanup;
    }

    if (FAILED(secStatus = NCryptSignHash(
        hKey,
        0,
        pbHash,
        cbHash,
        pbSignature,
        cbSignature,
        &cbSignature,
        0)))
    {
        wprintf(L"**** Error 0x%x returned by NCryptSignHash2\n", secStatus);
        goto Cleanup;
    }

UPDATE2:

The code work after removing NCryptDeleteKey(hKey, 0). I think this API delete my EC key! But I don't know why RSA key not be deleted?

You open the algorithm provider but you don't close it. Could you try to skip the "open" call after the first run? The store probably can be kept open as well and once you've found your certificate / key pair, then that can also be cached. — Maarten Bodewes, Feb 26 '20 at 02:00
For the rest it is unclear where you go wrong, you need to post your code. Replace `BCRYPT_SHA256_ALGORITHM` with `BCRYPT_RSA_ALGORITHM` can certainly not be right, RSA signature generation requires a hash. — Maarten Bodewes, Feb 26 '20 at 02:04
Hi Maarten, I ran the program one time and called ```BCryptCloseAlgorithmProvider()``` in the end. I don't think it will be cached? — Assam, Feb 26 '20 at 03:18
Sorry for my typo, not the ```BCRYPT_SHA256_ALGORITHM```. The correct is ```NCRYPT_ECDSA_P256_ALGORITHM```. I post my code, please take a look. — Assam, Feb 26 '20 at 03:20
It seems to me that you've mainly got problems with the `NCryptSignHash` method, however exactly that code is missing. — Maarten Bodewes, Feb 26 '20 at 22:35
#1 question Solved, it need add ```NCryptSetProperty()``` to set key length. I post my ```NCryptSignHash()``` code. — Assam, Feb 27 '20 at 03:42

How to Sign with RSA private key by CNG

0 Answers0