Python: Create Dictionary from Text/File that's in Dictionary Format

Question

I'd like to create a dictionary from a text file that I have, who's contents are in a 'dictionary' format. Here's a sample of what the file contains:

{'fawn': [1], 'sermersheim': [3], 'sonji': [2], 'scheuring': [2]}

It's exactly this except it contains 125,000 entries. I am able to read in the text file using read(), but it creates a variable of the literal text of the file even when I initialize the variable with

dict = {}

@JBernardo +1 as long as you have Python 2.6 or newer, that's the way to go. The `ast` module was introduced in 2.5, but didn't have the helper functions (such as `literal_eval`). Those came in 2.6. — Steven T. Snyder, Feb 16 '12 at 17:48

Steven T. Snyder · Accepted Answer · 2013-07-30T17:43:38.120

38

You can use the eval built-in. For example, this would work if each dictionary entry is on a different line:

dicts_from_file = []
with open('myfile.txt','r') as inf:
    for line in inf:
        dicts_from_file.append(eval(line))    
# dicts_from_file now contains the dictionaries created from the text file

Alternatively, if the file is just one big dictionary (even on multiple lines), you can do this:

with open('myfile.txt','r') as inf:
    dict_from_file = eval(inf.read())

This is probably the most simple way to do it, but it's not the safest. As others mentioned in their answers, eval has some inherent security risks. The alternative, as mentioned by JBernardo, is to use ast.literal_eval which is much safer than eval since it will only evaluate strings which contain literals. You can simply replace all the calls to eval in the above examples with ast.literal_eval after importing the ast module.

If you're using Python 2.4 you are not going to have the ast module, and you're not going to have with statements. The code will look more like this:

inf = open('myfile.txt','r')
dict_from_file = eval(inf.read())
inf.close()

Don't forget to call inf.close(). The beauty of with statements is they do it for you, even if the code block in the with statement raises an exception.

edited Jul 30 '13 at 17:43

answered Feb 16 '12 at 16:19

Steven T. Snyder

5,847
4
27
58

What if it's just a wall o' text? It's literally just a solid string starting with { and ending with }. – Jared Feb 16 '12 at 16:22
It will work if there's only one line, though it won't work if the dictionary is spread out over more than one line. I added additional code for that case. – Steven T. Snyder Feb 16 '12 at 16:27
I'm getting an 'invalid syntax' with the > "with open('myfile.txt','r') as inf: line. – Jared Feb 16 '12 at 16:35
Are you using Python 2.5 or newer? – Steven T. Snyder Feb 16 '12 at 16:38
@Jared I edited my answer to include some notes about using a version of Python older than 2.5, which it seems you may have. Basically, you won't have `with` statements (not too important), and you won't have the `ast` module. – Steven T. Snyder Feb 16 '12 at 16:45
Yes, you win the Pickle @Series8217. I have 2.4 on this computer and 2.5 at my other (primary), so I was unaware "with" would not work! – Jared Feb 16 '12 at 17:13
1

FYI: it looks like the method is now `ast.literal_eval`, not `ast.literaleval` – Taylor D. Edmiston Jul 30 '13 at 16:20
Got error: "ValueError: source code string cannot contain null bytes" – Peter.k Apr 26 '22 at 11:54

score 3 · Answer 2 · answered Feb 16 '12 at 16:18

3

Use the eval function.

For example,

dict = eval(open("yourfile.txt").read())

answered Feb 16 '12 at 16:18

1

+1 `eval` is definitely the simplest thing to do. The comment made by @JBernardo about `ast.literal_eval` is safer though as it only allows for strings, numbers, tuples, lists, dicts, booleans, and None. Just something to consider. – istruble Feb 16 '12 at 16:28

score 3 · Answer 3 · answered Feb 16 '12 at 16:24

3

Using eval might be dangerous. If json doesn't work, then I'd recommend using yaml which seems to work fine with your example input:

>>> import yaml
>>> yaml.load("{'fawn': [1], 'sermersheim': [3], 'sonji': [2], 'scheuring': [2]}")
{'fawn': [1], 'scheuring': [2], 'sermersheim': [3], 'sonji': [2]}

answered Feb 16 '12 at 16:24

jcollado

39,419
8
102
133

Ah, I'd like to use this but I don't have the module and the system the code is being run on has no internet access. Thanks anyway! – Jared Feb 16 '12 at 16:33
You might want to consider installation from source anyway. According to the documentation, it should be as easy as: `python setup.py install`. – jcollado Feb 16 '12 at 16:38

score 2 · Answer 4 · answered Feb 19 '17 at 10:21

It's not a production ready solution and may not work well with a file of your size, but if you need a simple way and can prepend you file to

my_dict = {'fawn': [1], 'sermersheim': [3], 'sonji': [2], 'scheuring': [2]}

then you can rename it to a python file and simply import

from my_file import my_dict

score 1 · Answer 5 · answered Feb 16 '12 at 16:23

1

I highly discourage using eval though. It may result in security issues if you don't have full control on the input file. Just import your dictionary and save them by using the json or pickle module.

answered Feb 16 '12 at 16:23

nopper

825
11
18

I agree that eval is a giant security risk if used on a regular basis or in production code, I interpreted the original question to be dealing with an isolated bit of code that comes from a trusted source and is already in a text file. Given such a situation, there's not much else you can do. – Feb 16 '12 at 16:26
It is in a static file from a trusted source, yes! – Jared Feb 16 '12 at 16:31
And I've had no luck with pickle files. I have this same information in a .pickle file, but can't seem to get it to load back in. This code is being ran on a restricted system with no internet access. – Jared Feb 16 '12 at 16:33
@Jared Perhaps you should post a question about your problems with `pickle` since that seems to be the real problem. – jcollado Feb 16 '12 at 16:36

Marcin · Answer 6 · 2012-02-16T16:27:47.030

0

~~This looks like json to me. Use the json module if so.~~

This looks like yaml to me. Use the pyyaml module if so. (As suggested by @jcollado).

You can't use the json module because it is strict about its input.

edited Feb 16 '12 at 16:27

answered Feb 16 '12 at 16:16

Marcin

48,559
18
128
201

4

It's not JSON: it uses single-quotes instead of double-quotes. – Ned Batchelder Feb 16 '12 at 16:18
@NedBatchelder: Yes. I didn't realise that the `json` module was strict about its input. – Marcin Feb 16 '12 at 16:26

Python: Create Dictionary from Text/File that's in Dictionary Format

6 Answers6

Linked

Related