Using python for frequency analysis

Question

I am trying to use python to help me crack Vigenère ciphers. I am fairly new to programming but I've managed to make an algorithm to analyse single letter frequencies. This is what I have so far:

Ciphertext = str(input("What is the cipher text?"))
Letters = "ABCDEFGHIJKLMNOPQRSTUVWXYZ"

def LetterFrequency():
    LetterFrequency = {'A': 0, 'B': 0, 'C': 0, 'D': 0, 'E': 0, 'F': 0, 'G': 0, 'H': 0, 'I': 0, 'J': 0, 'K': 0, 'L': 0, 'M': 0, 'N': 0, 'O': 0, 'P': 0, 'Q': 0, 'R': 0, 'S': 0, 'T': 0, 'U': 0, 'V': 0, 'W': 0, 'X': 0, 'Y': 0, 'Z': 0}
    for letter in Ciphertext.upper():
        if letter in Letters:
            LetterFrequency[letter]+=1
    return LetterFrequency

print (LetterFrequency())

But is there a way for me to print the answers in descending order starting from the most frequent letter? The answers are shown in random order right now no matter what I do.

Also does anyone know how to extract specific letters form a large block of text to perform frequency analysis? So for instance if I wanted to put every third letter from the text “THISISARATHERBORINGEXAMPLE” together to analyse, I would need to get:

T   H   I  
S   I   S  
A   R   A  
T   H   E  
R   B   O  
R   I   N  
G   E   X  
A   M   P  
L   E

Normally I would have to do this by hand in either notepad or excel which takes ages. Is there a way to get around this in python?

Thanks in advance,
Tony

coder · Accepted Answer · 2016-11-19T19:31:22.990

2

For the descending order you could use Counter:

>>> x = "this is a rather boring example"
>>> from collections import Counter
>>> Counter(x)
Counter({' ': 5, 'a': 3, 'e': 3, 'i': 3, 'r': 3, 'h': 2, 's': 2, 't': 2, 'b': 1, 'g': 1, 'm': 1, 'l': 1, 'o': 1, 'n': 1, 'p': 1, 'x': 1})

As for the second question you could iterate per 3.

To exclude spaces you can try what @not_a_robot suggests in the comment or delete it manually like:

>>> y = Counter(x)
>>> del y[' ']
>>> y
Counter({'a': 3, 'e': 3, 'i': 3, 'r': 3, 'h': 2, 's': 2, 't': 2, 'b': 1, 'g': 1, 'm': 1, 'l': 1, 'o': 1, 'n': 1, 'p': 1, 'x': 1})

edited Nov 19 '16 at 19:31

answered Nov 19 '16 at 19:20

coder

12,832
5
39
53

Thanks! That seems to work, but can I not include the spaces in the counter? – Tony Zhang Nov 19 '16 at 19:24
Try `Counter(letter for letter in x if letter != ' ')` – blacksite Nov 19 '16 at 19:25

score 1 · Answer 2 · answered Nov 19 '16 at 19:25

Another approach, although the collections.Counter example from @coder is your best bet.

from collections import defaultdict
from operator import itemgetter

Letters = "ABCDEFGHIJKLMNOPQRSTUVWXYZ"

Ciphertext = "this is a rather boring example"

def LetterFrequency():
    LetterFrequency = {letter: 0 for letter in Letters}
    for letter in Ciphertext.upper():
        if letter in Letters:
            LetterFrequency[letter]+=1
    return LetterFrequency

def sort_dict(dct):
    return sorted(dct.items(), key = itemgetter(1), reverse = True)

print(sort_dict(LetterFrequency()))

Which prints this, a list of tuples sorted descendingly by frequency:

[('A', 3), ('I', 3), ('E', 3), ('R', 3), ('T', 2), ('S', 2), ('H', 2), ('L', 1), ('G', 1), ('M', 1), ('P', 1), ('B', 1), ('N', 1), ('O', 1), ('X', 1), ('Y', 0), ('J', 0), ('D', 0), ('U', 0), ('F', 0), ('C', 0), ('Q', 0), ('W', 0), ('Z', 0), ('K', 0), ('V', 0)]

Using python for frequency analysis

2 Answers2