Python 2.7 Test_SSL fails - RAND_status is 0 (insufficient randomness)

Question

I've embedded my embedded python application into marmalade C++ for cross-platform use (Android, iOS, Windows phone 8, BlackBerry 10). I'm having trouble connecting to websites over HTTPS (HTTP sockets work)

A hunch

• I had to write a custom random function since it's supposed to be cross platform, using /dev/random is out of the question ( posix issues too probably )
• With the marmalade api there is a built in cross-platform random function that I've had good results with when using it to generate iv bits for AES encryption.

If anyone could shed some light on how much randomness is required for ssl, in bits or if it actually checks for degree of entropy????

test_ssl.py [Traceback]

test_random (test_ssl.BasicTests) ...
 RAND_status is 0 (insufficient randomness)


ok
test_refcycle (test_ssl.BasicTests) ... ERROR
test_sslwrap_simple (test_ssl.BasicTests) ... ok

======================================================================
ERROR: test_refcycle (test_ssl.BasicTests)
----------------------------------------------------------------------
----------------------------------------------------------------------
Ran 8 tests in 2.000s

test.test_support.TestFailed: Traceback (most recent call last):
  File "./test_ssl.py", line 149, in test_refcycle
    ss = ssl.wrap_socket(s)
  File "/pythonHome/Lib/ssl.py", line 344, in wrap_socket
    ciphers=ciphers)
  File "/pythonHome/Lib/ssl.py", line 108, in __init__
    socket.getpeername(self)
  File "/pythonHome/Lib/socket.py", line 226, in meth
    return getattr(self._sock,name)(*args)
error: [Errno 0] Error

Next I tested the obvious:

test_random.py [Traceback]

test_jumpahead (test_random.MersenneTwister_TestBasicOps) ... FAIL


======================================================================
FAIL: test_jumpahead (test_random.MersenneTwister_TestBasicOps)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "./test_random.py", line 58, in test_jumpahead
    self.assertRaises(TypeError, self.gen.jumpahead, "ick")  # wrong type
AssertionError: TypeError not raised

----------------------------------------------------------------------
Ran 60 tests in 34.000s

FAILED (failures=1)

test.test_support.TestFailed: Traceback (most recent call last):
  File "./test_random.py", line 58, in test_jumpahead
    self.assertRaises(TypeError, self.gen.jumpahead, "ick")  # wrong type
AssertionError: TypeError not raised

From what i've read the 'test_jumpahead' function is used for when you need entropy in parallel processes.. even though I patched the random function, this is the only issue it comes up with.

C++ custom random function

PyObject* s3eRand(PyObject* self, PyObject* pArgs)
{
    char* str2;
    Py_ssize_t count;
    if (!PyArg_ParseTuple(pArgs, "s#", &str2, &count)) return NULL;

    IwRandSeed((int32)s3eTimerGetMs());
    char str[5];

    sprintf(str,"%d",IwRandMinMax(0, 255));
    return Py_BuildValue("s",str);
}

static PyMethodDef PYs3eRandomMethods[] = {
    { "s3eUrandom", s3eRand, METH_VARARGS, "randomintiger" },
    { NULL, NULL, 0, NULL }
};

os.py [modification]

import s3ePY
...
...
if not _exists("urandom"):
    def urandom(n):
        """urandom(n) -> str

        Return a string of n random bytes suitable for cryptographic use.
        _urandomfd = open("raw:///dev/urandom", O_RDONLY) doesnt work

        """
        #from s3ePY import s3eUrandom

        bs=b""
        while n - len(bs) >= 1:
            bs+=chr(int(s3ePY.s3eUrandom()))
        return bs

---

Answer 1

RAND_status is 0 (insufficient randomness)

When RAND_status fails, you should call RAND_poll to reseed the generator, or you should call RAND_add to add your own entropy. See Random Numbers on the OpenSSL wiki.

---

If anyone could shed some light on how much randomness is required for ssl..

Its the underlying generator; not SSL.

It depends on the generator. The default generator is md_rand, and its source code can be found in <openssl src>/crypto/rand/md_rand.c. If you are using md_rand, then you need to seed/reseed the generator with at least 32 bytes:

$ grep -R ENTROPY_NEEDED *
...
crypto/rand/md_rand.c:  if (entropy < ENTROPY_NEEDED) /* stop counting when we have enough */
crypto/rand/md_rand.c:  ok = (entropy >= ENTROPY_NEEDED);
crypto/rand/md_rand.c:  ret = entropy >= ENTROPY_NEEDED;
crypto/rand/rand_lcl.h:#define ENTROPY_NEEDED 32  /* require 256 bits = 32 bytes of randomness */
...

Some generators don't allow entropy input. For example, the RDRAND engine no-ops RAND_add and always returns success. (But that does not seem to be your problem).

The size required for seeding (i.e., the 32 bytes) is purposefully hidden from you. See RAND_status/RAND_seed on the OpenSSL mailing list. And notice the 128-bits/16-bytes is no longer the minimum.

---

C++ custom random function...

You should create an engine and plug it into OpenSSL. Then OpenSSL will use your engine rather than the md_rand engine.

But, this does not instill a lot of confidence:

IwRandSeed((int32)s3eTimerGetMs());

Nobody will ever think to brute force calls to time()... Wait, Wagner and Goldberg did it to Netscape in 1996. See Randomness and the Netscape Browser.

Maybe you should mix in the PID so its as least as bad as Netscape's implementation, and not worse than :)

---

You did not say what platform you are on. Below is the code I use to gather entropy from sensors on iOS. I have similar code for Android. Using sensor data allows me to ensure sufficient entropy even on low entropy devices.

After the sensor data is fetched, its then sent into the random number generator before each use of the generator. The underlying generator then performs entropy expansion and extraction (unlike your s3eRand).

The code is written for Crypto++, but all you need to do is swap Crypto++'s IncorporateEntropy for OpenSSL's RAND_add.

static CryptoPP::RandomPool& GetRandomPool()
{
    static CryptoPP::RandomPool pool;
    static dispatch_once_t once = 0;

    dispatch_once(&once, ^{
        CryptoPP::SecByteBlock seed(32);
        CryptoPP::OS_GenerateRandomBlock(true, seed.data(), seed.size());
        pool.IncorporateEntropy(seed.data(), seed.size());
    });

    // First, send in all the uninitialized data. Then:
    //  sesnors[0,1,2] uses accelerometer, if available
    //  sesnors[3,4,5] uses gyroscope, if available
    //  sesnors[6,7,8] uses magnetometer, if available
    CryptoPP::SecBlock<double> sensors(3 * 3);
    pool.IncorporateEntropy(sensors.BytePtr(), sensors.SizeInBytes());

    // See Event Handling Guide for iOS, Event Motions
    // https://developer.apple.com/library/ios/documentation/EventHandling/Conceptual/EventHandlingiPhoneOS/motion_event_basics/motion_event_basics.html
    CMMotionManager* mgr = [[CMMotionManager alloc] init];
    if(mgr == nil) { return pool; }

    // Start motion services
    if([mgr isAccelerometerAvailable] != NO)
        [mgr startAccelerometerUpdates];
    if([mgr isGyroAvailable] != NO)
        [mgr startGyroUpdates];
    if([mgr isMagnetometerAvailable] != NO)
        [mgr startMagnetometerUpdates];

    // The components starts slowly... A pause less than about 0.150s
    //  will cause individual sensor data to return nil.    
    [NSThread sleepForTimeInterval:0.150f];

    if([mgr isAccelerometerAvailable] != NO) {
        CMAccelerometerData* accelData = [mgr accelerometerData];
        if(accelData) {
            sensors[0] = [accelData acceleration].x;
            sensors[1] = [accelData acceleration].y;
            sensors[2] = [accelData acceleration].z;
        }
        [mgr stopAccelerometerUpdates];
    }

    if([mgr isGyroAvailable] != NO) {
        CMGyroData* gyroData = [mgr gyroData];
        if(gyroData) {
            sensors[3] = [gyroData rotationRate].x;
            sensors[4] = [gyroData rotationRate].y;
            sensors[5] = [gyroData rotationRate].z;
        }
        [mgr stopGyroUpdates];
    }

    if([mgr isMagnetometerAvailable] != NO) {
        CMMagnetometerData* magnetData = [mgr magnetometerData];
        if(magnetData) {
            sensors[6] = [magnetData magneticField].x;
            sensors[7] = [magnetData magneticField].y;
            sensors[8] = [magnetData magneticField].z;
        }
        [mgr stopMagnetometerUpdates];
    }

    pool.IncorporateEntropy(sensors.BytePtr(), sensors.SizeInBytes());

    [mgr release], mgr = nil;

#if !defined(NDEBUG)
    NSLog(@"\n  A: %f, %f, %f" \
          @"\n  G: %f, %f, %f" \
          @"\n  M: %f, %f, %f",
          sensors[0], sensors[1], sensors[2],
          sensors[3], sensors[4], sensors[5],
          sensors[6], sensors[7], sensors[8] );
#endif

    return pool;
}

If interested, the call to CryptoPP::OS_GenerateRandomBlock wraps a call to /dev/urandom. But as you can see, its called once and I don't depend on it. If it were missing, the routine would still be able to produce bits.

Answer 2

[Python-Dev] what platforms require RAND_add() before using SSL?

In general, anything that does not have /dev/[u]random;
older Solaris releases and HP-UX in particular.

The SSL Random function has to be pre-seeded before calling

RAND_add() Documentation

This is applied to socket.

ex: OpenSSL -- SSL method >> RAND_add(string, entropy)

import socket
socket.RAND_add("this is a random string", 75.0) #Note: do not use in production

import urllib2
response = urllib2.urlopen('https://twitter.com') #Note: 'https'
html = response.read()
print html

NOTE: do not use "this is a random string" in production, pre-seed with your random function