Lazy boolean evaluation in Python when using Map-Reduce

Question

I will use my example: I want to create a list of primes using the sieve of Eratosthenes. For each number, I check if it is composite, and if not, I append it to the list.

Using "standard" programming:

primes = [2]
start = time.time()
for i in xrange(3,primeRange,2):
    isPrime = True
    for p in primes:
        if(i % p == 0):
            isPrime = False
            break;
    if(isPrime):        
        primes.append(i)
print "Using C++-style: ", time.time() - start, " seconds"

Using reduce function:

start = time.time()
for i in xrange(3,primeRange,2):
    if(reduce(lambda x,y:x and y,[i % p != 0 for p in primes])):
        primes.append(i)
print "Using map-reduce: ", time.time() - start, " seconds"

The results for primeRange = 100000:

Using map-reduce:  54.1150000095  seconds
Using C++-style:  4.62000012398  seconds

The 2nd case makes the code more compact, but the condition will be evaluated for the entire list and then reduced to True / False. Is there a way to avoid that?

score 3 · Accepted Answer · answered May 31 '14 at 18:41

3

You could use all with a generator expression:

if all(i % p != 0 for p in primes)

The generator expression will lazily evaluate one term at a time, and all will return early if it hits a value that doesn't satisfy the condition.

answered May 31 '14 at 18:41

BrenBarn

242,874
37
412
384

score 2 · Answer 2 · answered May 31 '14 at 18:41

yes, you should do two things.

First, if you want to avoid fully evaluating a list comprehension, don't use a list comprehension, use a generator expression:

... [i % p != 0 for p in primes] ...

should be

... (i % p != 0 for p in primes) ...

second, reduce() doesn't understand that lambda x,y:x and y can fail, and there's no way to teach it that. But that doesn't matter, because python has a builtin function for exactly the fold over the conjunction monoid, all(), so the if should be

if all(i % p != 0 for p in primes):

score 0 · Answer 3 · answered Aug 09 '18 at 13:15

Use generator,

def prime_iter(sz):
    '''
    Iterate through all the prime number in range [2,sz]
    '''
    bucket = [0]*sz
    for i in range(2,sz):
        if bucket[i]:
            continue
        for j in range(i+i,sz,i):
            bucket[j] = i
        yield i

# get all the primes from 2 to 100.
primes = list(prime_iter(100))

Lazy boolean evaluation in Python when using Map-Reduce

3 Answers3