Generate elliptic curve key pairs (EC_KEY_generate_key) in multiple threads using OpenSSL C library

Question

I want to generate many ec key pairs. Speeding up the process a bit, I rewrote my appication to use multiple threads for this job. Here is a code snippet of the way each thread wants to generate the keys:

(...)
EC_KEY* _ec_key = EC_KEY_new(); 
EC_GROUP* ec_group_new = EC_GROUP_new_by_curve_name(NID_X9_62_prime256v1); 
const EC_GROUP* ec_group = ec_group_new; 
if (!EC_KEY_set_group(ec_key,ec_group)) 
  DieWithError("Error in initializeCrypto, EC_KEY_set_group failed!");

// Segfault at this position
if(!EC_KEY_generate_key(ec_key))
  DieWithError ("Error in generateKeys, EC_KEY_generate_key failed!");

(...)
EC_GROUP_free(ec_group_new); 
EC_KEY_free(ec_key);

Ok at the first glance, everything seemed to work fine. The applications ran twice as fast using four threads on my i5 520m. But then after 3-4 E6 key generations it suddenly segfaults. If I lock the EC_KEY_generate_key operation there is no segfault anymore, but the advantage of using multiple threads is gone. Now my questions. Is it possible split the creation of keys into multiple threads without corrupting memory? I didn't found any information using google. The SSL Docu doesn't mention anything about thread-safety, though. Any help is highly appreciated. thx

jww · Accepted Answer · 2013-12-04T11:27:48.707

// Segfault at this position
if(!EC_KEY_generate_key(ec_key))
  DieWithError ("Error in generateKeys, EC_KEY_generate_key failed!");
...

... But then after 3-4 E6 key generations it suddenly segfaults.

You are using OpenSSL's random number generator, and its not thread safe. Below is from cryptlib.c around line 125. Notice the random number generators and the elliptic curve gear make the list.

/* real #defines in crypto.h, keep these upto date */
static const char* const lock_names[CRYPTO_NUM_LOCKS] =
    {
    "<<ERROR>>",
    "err",
    "ex_data",
    "x509",
    "x509_info",
    "x509_pkey",
    "x509_crl",
    "x509_req",
    ...
    "ssl_ctx",
    "ssl_session",
    "ssl",
    "ssl_method",
    "rand",
    "rand2",
    ...
    "ecdsa",
    "ec",
    "ecdh",
    "bn",
    "ec_pre_comp",
    ...
    };

You have to explicitly set the locks. See OpenSSL's threads(3).

Is it possible split the creation of keys into multiple threads without corrupting memory?

Yes, but you have to use OpenSSL's locking mechanism.

Here's what my OpenSSL initialization routine looks like in C++. It initializes the locks and sets the callbacks.

pthread_mutex_t s_locks[CRYPTO_NUM_LOCKS] = { };

void Initialize()
{    
    static once_flag init;
    std::call_once(init, []() {      

        // Standard OpenSSL library init
        OPENSSL_no_config();
        SSL_library_init();

        SSL_load_error_strings();
        OpenSSL_add_ssl_algorithms();

        // Lock setup
        LOCK_setup();
        CALLBACK_setup();
    });
}

void LOCK_setup()
{    
    ASSERT(CRYPTO_NUM_LOCKS == CRYPTO_num_locks());
    if(CRYPTO_NUM_LOCKS != CRYPTO_num_locks())
        throw runtime_error("CRYPTO_NUM_LOCKS mismatch");

    for(unsigned i = 0; i < CRYPTO_NUM_LOCKS; ++i)
    {
        int rc = pthread_mutex_init(&s_locks[i], NULL);
        ASSERT(rc == 0);
        if(!(rc == 0))
            throw runtime_error("pthread_mutex_init");
    }
}

void CALLBACK_setup()
{    
    CRYPTO_set_id_callback(&ThreadIdFnc);
    CRYPTO_set_locking_callback(&LockingFnc);
}

void LockingFnc(int mode, int idx, const char* file, int line)
{
    ASSERT(mode == CRYPTO_LOCK || mode == CRYPTO_UNLOCK);
    ASSERT(CRYPTO_NUM_LOCKS == CRYPTO_num_locks());
    ASSERT(idx >= 0 && idx < CRYPTO_NUM_LOCKS);

    if(!(idx >= 0 && idx < CRYPTO_NUM_LOCKS))
    {    
        ostringstream oss;
        oss << "LockingFnc: lock failed with bad index ";
        oss << idx << ". File: " << (file ? file : "Unknown");
        oss << ", line: " << line;

        // Log oss.str()
        return;
    }

    if((mode & CRYPTO_LOCK) == CRYPTO_LOCK)
    {
        int rc = pthread_mutex_lock(&s_locks[idx]);
        int err = errno;
        ASSERT(rc == 0);

        if(!(rc == 0))
        {
            ostringstream oss;
            oss << "LockingFnc: lock failed with error ";
            oss << err << ". File: " << (file ? file : "Unknown");
            oss << ", line: " << line;          

            throw runtime_error(oss.str());
        }
    }
    else if((mode & CRYPTO_UNLOCK) == CRYPTO_UNLOCK)
    {
        int rc = pthread_mutex_unlock(&s_locks[idx]);
        int err = errno;
        ASSERT(rc == 0);

        if(!(rc == 0))
        {
            ostringstream oss;
            oss << "LockingFnc: unlock failed with error ";
            oss << err << ". File: " << (file ? file : "Unknown");
            oss << ", line: " << line;

            throw runtime_error(oss.str());
        }
    }
}

unsigned long ThreadIdFnc()
{
#if defined(AC_OS_APPLE)
    ASSERT(sizeof(unsigned long) >= sizeof(pid_t));
    return static_cast<unsigned long>(pthread_mach_thread_np(pthread_self()));
#elif defined(AC_OS_STARNIX)
    ASSERT(sizeof(unsigned long) >= sizeof(pid_t));
    return static_cast<unsigned long>(gettid());
#else
# error "Unsupported platform"
#endif
}

If you are not using libssl, then forgo the call to SSL_library_init. All libcrypto needs is the call to OpenSSL_add_all_algorithms to initialize.

The SSL Documentation doesn't mention anything about thread-safety, though.

Yeah, the docs leave something to be desired at times. I know a bunch of folks are working on improving it through a wiki run by the OpenSSL Foundation. Matt Caswell has done a lot of work in simply documenting the elliptic curve stuff at http://wiki.openssl.org/index.php/Elliptic_Curve_Cryptography. He's also responsible for the POD files and MAN pages. Keep in mind that Matt did not write any of the code - he's just documenting it for others.

There's a page on initialization, but it does not have the code for the locks. Its on my TODO list. See http://wiki.openssl.org/index.php/Library_Initialization.

Thx for your good answer! Your hints in the first version already helped me finding an example which solves my problem. I found this one http://curl.haxx.se/libcurl/c/opensslthreadlock.html and adapted my code to it. However there are still some uncertainties left (due to my curiosity). First, I don't get how CRYPTO_num_locks() works, because it seems "to know" how much threads I'll use in my program? Second, the OpenSSL thread docu proposes another ThreadIdFnc() with other arguments and return type (see int CRYPTO_THREADID_set_callback(void (*threadid_func)(CRYPTO_THREADID *));). — Arthur Giss, Dec 04 '13 at 11:47
OpenSSL locks their components (for example, the random number generator). The locking occurs around shared data, and the number of threads is irrelevant. `CRYPTO_num_locks` is a runtime function. It returns the maximum number of locks needed in the array you provide. The compile time constant is `CRYPTO_NUM_LOCKS`. There's no need to `malloc` at runtime when you know the size at compile time. — jww, Dec 04 '13 at 20:25
The `threadid_func` vs `ThreadIdFnc` naming does not matter. You could name it `Foo`, `bar`, or `Foo_bar`. What matters is the function's signature and calling convention - the function's parameters and return type. The function has to take arguments `int,int,char*,int` and return `void`. — jww, Dec 04 '13 at 20:29
The sample code in `/crypto/threads/mttest.c` might be helpful for you. — jww, Dec 04 '13 at 20:36

Generate elliptic curve key pairs (EC_KEY_generate_key) in multiple threads using OpenSSL C library

1 Answers1