0

I am looking through the Internet trying to find the source of the PK11_GenerateRandom() function to see why would the function fail. I have a program that perfectly uses this function but when we moved to a new flavor of Linux, it fails after forking (fork()) Since I do not believe there is a problem with NSS, I suspect that we are doing something incorrectly which was disregarded in the older versions of Linux but with the new one there is an issue.

The OpenSSL package is the same on the 'good' and the 'bad' server:

OpenSSL 0.9.8e-fips-rhel5 01 Jul

NSS rpm differs though. The 'good' has 

nss-3.12.2.0-2.el5

and the bas has this version

nss-3.15.3-4.el5_10

The 'good' server uses quite obsolete Linux:

Linux GOOD 2.6.18-128.el5 #1 SMP Wed Jan 21 08:45:05 EST 2009 x86_64 x86_64 x86_64 GNU/Linux
Enterprise Linux Enterprise Linux Server release 5.3 (Carthage)
Red Hat Enterprise Linux Server release 5.3 (Tikanga)

The 'bad' server is newer:

Linux BAD 2.6.18-371.4.1.el5 #1 SMP Wed Jan 29 11:05:49 PST 2014 x86_64 x86_64 x86_64 GNU/Linux
Oracle Linux Server release 5.10
Red Hat Enterprise Linux Server release 5.10 (Tikanga)

Any clue as to where could I find the source or what could be the reason for failure (like side effect coming from fork()) would be greatly appreciated.

  • Greg

Edit

Here is the code, which is so simple, that I did not think it is needed.

/* random points to properly allocated memory, let=32 */
SECStatus rv = PK11_GenerateRandom((unsigned char *)random, (int)len);
if ( rv != SECSuccess )
    printf( "PK11_GenerateRandom error = %d\n", PR_GetError()) ;

and the output message is, of course:

PK11_GenerateRandom error = -8023
  • Greg
tshepang
  • 12,111
  • 21
  • 91
  • 136
Grzegorz
  • 3,207
  • 3
  • 20
  • 43
  • Please show the command as you enter it and the error message – sabbahillel Mar 10 '14 at 14:43
  • @sabbahillel: Added the code to the original posting, but I do not believe you will find any problem there, as the issue is not with calling. The same code worked for years at thousands of sites. It is the new Linux flavor that exposed something which we are investigating now. Thanks for the interest, though. – Grzegorz Mar 10 '14 at 14:50
  • 1
    @Grzegorz: I don't understand why did you provide OpenSSL version when PK11_GenerateRandom() seems to be NSS function. Is there any relation between your problem and OpenSSL you forget to mention? Could you please provide also version of NSS you are using? And maybe you are using any PKCS#11 enabled cryptographic hardware (HSM module etc) which could be used by PK11_GenerateRandom? – jariq Mar 10 '14 at 15:10
  • @jariq - Your questions give me hope you can help, as I am being added to the project, and they were telling me about OpenSLL and NSS as if all was interconnected. How can I check NSS version? How can I know if we use enabled cryptographic hardware functions? Thank you for your help! – Grzegorz Mar 10 '14 at 15:20
  • @jariq - The old nss rpm is nss-3.12.2.0-2.el5, and the new one is nss-3.15.3-4.el5_10. – Grzegorz Mar 10 '14 at 15:42

1 Answers1

1

The source of the PK11_GenerateRandom() function: http://mxr.mozilla.org/mozilla-central/source/security/nss/lib/pk11wrap/pk11slot.c#2285

Based on my calculation the -8023 corresponds error SEC_ERROR_PKCS11_DEVICE_ERROR

The reason (thanks to jariq's hints) is described here: https://bugzilla.mozilla.org/show_bug.cgi?id=331096

It is that in the past, it was okay to fork and continue using PKCS11 functions. They decided that it cannot be like that, and now, conclusion is that the parent should not initialize these functions if a child after forking is expected to use them.

PKCS11 internal functions are checking if there was forking (they use various methods dependently on what platform the code is built.) For example, they stored pid of the process in an internal memory, and in some expensive functions or called not so often they compare this preserved pid with current getpid().

The fix for our problem will require redesigning the code.

Grzegorz
  • 3,207
  • 3
  • 20
  • 43
  • 1
    And the error description for SEC_ERROR_PKCS11_DEVICE_ERROR is "A PKCS #11 module returned CKR_DEVICE_ERROR, indicating that a problem has occurred with the token or slot." PK11_GenerateRandom() calls C_GenerateRandom() on internal NSS SoftToken or on external PKCS#11 module (in many cases something like a driver for cryptographic hardware). If you are not using any HSM or smartcard than I believe you should look into the code of NSS SoftToken. – jariq Mar 10 '14 at 17:36
  • 1
    You should continue here: [C_GenereateRandom](http://mxr.mozilla.org/mozilla-central/source/security/nss/lib/softoken/pkcs11c.c#3369) source in NSS SoftToken. BTW 3.12.2.0-2 released Dec 10 2008 and 3.15.3-4 released Dec 13 2013 - the difference is five years. – jariq Mar 10 '14 at 17:38
  • @jariq - Thank you very much. I am looking in to this page now. – Grzegorz Mar 10 '14 at 18:00
  • @jariq - I have found a place that is related to forking. I have also found that the PKCS#11 standard says that after a fork all PKCS#11 modules must be reinitialized again. I am now searching for a way to reinitialize PKCS#11 modules or whatever I need to do... – Grzegorz Mar 10 '14 at 18:25