2

I am trying to generate a sentence in the style of the bible. But whenever I run it, it stops at a KeyError on the same exact word. This is confusing as it is only using its own keys and it is the same word every time in the error, despite having random.choice.

This is the txt file if you want to run it: ftp://ftp.cs.princeton.edu/pub/cs226/textfiles/bible.txt

import random

files = []
content = ""
output = ""

words = {}

files = ["bible.txt"]
sentence_length = 200

for file in files:
    file = open(file)
    content = content + " " + file.read()

content = content.split(" ")

for i in range(100):  # I didn't want to go through every word in the bible, so I'm just going through 100 words
    words[content[i]] = []
    words[content[i]].append(content[i+1])

word = random.choice(list(words.keys()))

output = output + word

for i in range(int(sentence_length)):
    word = random.choice(words[word])
    output = output + word

print(output)
falcon user
  • 717
  • 1
  • 6
  • 10
  • What's the exact text of the error? – Patrick Haugh Oct 06 '17 at 22:27
  • Midst is a value in your dictionary, not a key. –  Oct 06 '17 at 22:29
  • If you increase the range to 200, it resolves itself, but it falls into a loop and prints the same thing over and over. You might want to rework this code to avoid this .Try to increase the range to 200 and see. –  Oct 06 '17 at 22:32

2 Answers2

2

The KeyError happens on this line:

word = random.choice(words[word])

It always happens for the word "midst".

How? "midst" is the 100th word in the text. And the 100th position is the first time it is seen. The consequence is that "midst" itself was never put in words as a key. Hence the KeyError.

Why does the program reach this word so fast? Partly because of a bug here:

for i in range(100):
    words[content[i]] = []
    words[content[i]].append(content[i+1])

The bug here is the words[content[i]] = [] statement. Every time you see a word, you recreate an empty list for it. And the word before "midst" is "the". It's a very common word, many other words in the text have "the". And since words["the"] is ["midst"], the problem tends to happen a lot, despite the randomness.

You can fix the bug of creating words:

for i in range(100):
    if content[i] not in words:
        words[content[i]] = []
    words[content[i]].append(content[i+1])

And then when you select words randomly, I suggest to add a if word in words condition, to handle the corner case of the last word in the input.

janos
  • 120,954
  • 29
  • 226
  • 236
0

"midst" is the 101st word in your source text and it is the first time it shows up. When you do this:

words[content[i]].append(content[i+1])

you are making a key:value pair but you aren't guaranteed that that value is going to be equivalent to an existing key. So when you use that value to search for a key it doesn't exist so you get a KeyError.

If you change your range to 101 instead of 100 you will see that your program almost works. That is because the 102nd word is "of" which has already occurred in your source text.

It's up to you how you want to deal with this edge case. You could do something like this:

    if i == (100-1):
        words[content[i]].append(content[0])
    else:
        words[content[i]].append(content[i+1])

which basically loops back around to the beginning of the source text when you get to the end.

wsad597
  • 53
  • 8