Python - Project Euler #35 -- what is wrong with this approach?

Question

Problem 35 of Project Euler is as so:

The number, 197, is called a circular prime because all rotations of the digits: 197, 971, and 719, are themselves prime.

There are thirteen such primes below 100: 2, 3, 5, 7, 11, 13, 17, 31, 37, 71, 73, 79, and 97.

How many circular primes are there below one million?

My, rather crude, approach is to first generate all the primes below one million then filter out all the primes which contain even digits or the number 5 (as there will always be a non-prime permutation). Then, for each element in this reduced list of primes to return all the possible permutations of the number using the permutations() function in the itertools module, then to check if any of these permutations are not prime and if so to remove the element from the list of primes.

from itertools import permutations

def gen_primes(limit):
    D = {}
    q = 2
    while q <= limit:
        if q not in D:
            yield q
            D[q * q] = [q]
        else:
            for p in D[q]:
                D.setdefault(p + q, []).append(p)
            del D[q]       
        q += 1 

def odd_primes(limit):
    r = list(gen_primes(limit))
    for i in r[:]:
        for j in str(i):
            if any(int(j)%2 == 0 or int(j) == 5 for j in str(i)):
                r.remove(i)
                break
    r.extend([2,5])
    return r    

def circular_list():
    prime_list = odd_primes(1000000)
    for i in prime_list[:]:
        perm = [''.join(j) for j in permutations(str(i))]
        if any(int(j) not in prime_list for j in perm):
            prime_list.remove(i)
            break
    return prime_list

print len(circular_list)

The output yields a value which is incorrect by some margin. I've really been struggling to find the mistake in either the logic or code (or both). Is the permutations() function a viable approach?

I understand that there are more efficient approaches, but I'd be grateful if someone could point me in a direction to make this one work.

Answer 1

First issue is you need rotations, not permutations. A deque can be rotated so we can substitute that. Second issue is that odd_primes() is a speed optimization that shouldn't be added until the basic code is working, so we'll leave it out for now and have circular_list() call gen_primes() directly. Of course, a prime generator isn't optimal, as you discovered, since we have to look back into the list of primes and a generator can only be iterated once.

So here's what is hopefully working code with a deque and sans odd_primes():

from collections import deque

def gen_primes(limit):
    D = {}
    q = 2
    while q <= limit:
        if q not in D:
            yield q
            D[q * q] = [q]
        else:
            for p in D[q]:
                D.setdefault(p + q, []).append(p)
            del D[q]       
        q += 1 

def circular_list(limit):
    circular = []

    primes = list(gen_primes(limit))

    for prime in primes:
        string = str(prime)
        digits = deque(string)

        for rotation in range(1, len(string)):
            digits.rotate(1)

            if int("".join(digits)) not in primes:
                break
        else:
            circular.append(prime)

    return circular

print(circular_list(1000000))

Which outputs:

[2, 3, 5, 7, 11, 13, 17, 31, 37, 71, 73, 79, 97, 113, 131, 197, 199, 311, 337, 373, 719, 733, 919, 971, 991, 1193, 1931, 3119, 3779, 7793, 7937, 9311, 9377, 11939, 19391, 19937, 37199, 39119, 71993, 91193, 93719, 93911, 99371, 193939, 199933, 319993, 331999, 391939, 393919, 919393, 933199, 939193, 939391, 993319, 999331]

If that's valid output, now go back and slip in odd_primes() to see if there are any tricks you can play to optimize the speed.

Answer 2

Using the great code from cdlane and introducing the speed optimization based on that a circular prime with at least two digits can only consist of combinations of the digits 1, 3, 7 or 9, because having 0, 2, 4, 6 or 8 as the last digit makes the number divisible by 2, and having 0 or 5 as the last digit makes it divisible by 5 (from https://en.wikipedia.org/wiki/Circular_prime) you get:

from collections import deque
import re
import time

def gen_primes(limit):
    D = {}
    q = 2
    while q <= limit:
        if q not in D:
            yield q
            D[q * q] = [q]
        else:
            for p in D[q]:
                D.setdefault(p + q, []).append(p)
            del D[q]
        q += 1

def exclude_primes(primes_list):
    regex = re.compile("[024568]")
    included_primes_list = [str(prime) for prime in primes_list if prime < 10 or regex.search(str(prime)) is None]
    return included_primes_list

def circular_list(limit):
    circular_count = 0

    primes = set(exclude_primes(gen_primes(limit)))

    for prime in primes.copy(): # need copy to process allowing update original primes list
        digits = deque(prime)

        rotations = set()
        for rotation in range(len(digits)):
            digits.rotate(1)
            rotations.add("".join(digits))
        # check all rotations at once
        if rotations.issubset(primes):
            circular_count += len(rotations)
        # remove all rotations already checked from primes list
        primes.difference_update(rotations)

    return circular_count

start_cpu_time = time.clock()
print("Number of primes:", circular_list(1000000))
end_cpu_time = time.clock()
print("CPU seconds:", end_cpu_time - start_cpu_time)

This is a lot faster than without the optimization, but can probably be optimized further.

Answer 3

Here's how I solved this, takes about 1.1s to run on my laptop...

It preloads the return list with 2 and 5.

Basically, it runs through all odd numbers between 3 and 1e6.

First, it runs all of the digits to see if any of them are divisible by 2 or 5.

If there are no such digits, it checks if the number is prime. First, it maintains a list of primes that it has found, and then runs the 6k+1 algorithm to test after that.

For numbers that are prime, it then runs all the rotations (using a quick string rotation function) to check if all of them are prime. If they are, it adds that number to the list.

Once all the numbers up to 1e6 are checked, it kicks out the count and the number of seconds it took to run.

It ran quickly enough I extended it to 100,000,000 (130s runtime), and found no additional matches.

"""
Circular primes
Problem 35 
The number, 197, is called a circular prime because all rotations of the digits: 197, 971, and 719, are themselves prime.

There are thirteen such primes below 100: 2, 3, 5, 7, 11, 13, 17, 31, 37, 71, 73, 79, and 97.

How many circular primes are there below one million?
"""
import time
start_time = time.time()


foundprimes = []

def isPrime(n):
    """Determine if cand is a prime number"""
    if n in foundprimes:
        return True

    if n <= 3:
        return n>1
    if n%2 == 0:
        return False
    if n%3 == 0:
        return False

    i=5
    while (i*i <= n):
        if n%i == 0:
            return False
        if n%(i+2) == 0:
            return False
        i+=6

    foundprimes.append(n)
    return True

def rotations(num):
    """Return all rotations of the given number"""

    if abs(num) < 10:
        return [num]

    numstr = str(num)
    strlen = len(numstr)
    returnarray = []
    for i in range(strlen):
        if i==0:
            pass
        else:
            start = numstr[i:]
            end = numstr[0:i]
            returnarray.append(int(start+end))


    return returnarray

def QCheck(num):
    """Do a quick check if there is an even number in the set...at some point it will be an even number in rotation"""
    numstr = str(num)
    for i in range(len(numstr)):
        if int(numstr[i]) % 2 == 0:
            return False
        if int(numstr[i]) % 5 == 0:
            return False

    return True


allrotations = [2,5]

for i in range(3,1000000,2):
    if QCheck(i):
        prime = isPrime(i)
        if prime:
            rotcheck = True
            for rot in rotations(i):
                if isPrime(rot) == False:
                    rotcheck = False

            if rotcheck:
                allrotations.append(i)

print(len(allrotations))
print("--- %s seconds ---" % (time.time() - start_time))