Python: How to create a dictionary from properties file while omitting comments

Question

I've searched for the answer to this here for awhile and haven't found it, so hope this isn't a dupe.

I have a properties file that mostly contains key=value pairs, but also contains #comments. I need to put it in a dictionary so I can grab values at will. In a file without #comments, the following works perfectly.

myprops = dict(line.strip().split('=') for line in open('/Path/filename.properties'))
print myprops['key']

But not so when there are comments present. If there's #comment present, dictionary says

"ValueError: dictionary update sequence element #x has length 1, 2 is required"

I've tried wrapping the dictionary creation in conditionals with

if not line.startswith('#'):

But I can't seem to get that to work. Suggestions? Thanks!

Answer 1

To address your newest constraint about blank lines, I would try something like:

myprops = {}
with open('filename.properties', 'r') as f:
    for line in f:
        line = line.rstrip() #removes trailing whitespace and '\n' chars

        if "=" not in line: continue #skips blanks and comments w/o =
        if line.startswith("#"): continue #skips comments which contain =

        k, v = line.split("=", 1)
        myprops[k] = v

It's very clear and it's easy to add on extra constraints, whereas using a dict comprehension will get quite bloated. However, you could always format it nicely

myprops = dict(line.strip().split('=') 
               for line in open('/Path/filename.properties'))
               if ("=" in line and 
                   not line.startswith("#") and
                   <extra constraint> and
                   <another extra constraint>))

Answer 2

You should just use the built-in configparser which is made to read ini-style configuration files. It allows comments using ; and # by default, so it should work for you.

For .properties files you might need to trick a bit as the configparser generally expects section names. You can do this easily by adding a dummy section while reading it though:

>>> from configparser import ConfigParser
>>> config = ConfigParser()
>>> with open(r'C:\Users\poke\Desktop\test.properties') as f:
        config.read_string('[config]\n' + f.read())

>>> for k, v in config['config'].items():
        print(k, v)

foo bar
bar baz
baz foo

(Using the same example file as mtitan8)

For Python 2, use from ConfigParser import ConfigParser instead.

Answer 3

Given a properties file test.txt as you've described:

foo=bar
#skip me
bar=baz
baz=foo
#skip me too!

You can do the following:

>>> D = dict( l.rstrip().split('=') for l in open("test.txt")
              if not l.startswith("#") )
>>> D
{'baz': 'foo', 'foo': 'bar', 'bar': 'baz'}

This seems just like the code you said you tried using if not line.startswith('#'), so hopefully this working example will help you pinpoint the bug.

Answer 4

None of the solutions take into account that the value can contain an = sign.

Imagine the following file:

baz=foo
foo="foo=bar"
bar=baz

Therefor, I suggest to use the following code:

>>> D = dict( l.rstrip().split('=', maxsplit=1) for l in open("test.txt") if not l.startswith("#") )
>>> D
{'baz': 'foo', 'foo': '"foo=bar"', 'bar': 'baz'}

Answer 5

Why force this into one line? Two weeks from now a user will put a space somewhere, or want to use quotes and you have to go unwind the code. Just make a function now which handles the input and be done with it. It also means you can use unit tests to ensure it works and stays working.

Given this input:

foo=bar
#skip me

bar=baz
baz=foo

#skip me too!

The following code will handle it all nicely.

import sys

def parse_line(input):
    key, value = input.split('=')
    key = key.strip()  # handles key = value as well as key=value
    value = value.strip()

    return key, value

if __name__ == '__main__':
    data = {}

    with open(sys.argv[1]) as fp:
        for line in fp:
            line = line.strip()
            if not line or line.startswith('#'):
                continue

            key, value = parse_line(input)
            data[key] = value

    print data

BTW, I like poke's suggestion of using ConfigParser. But the hack of adding a section may or may not work for everyone.

If you want to move the comment checking into the parse_line() function you could return None, None and check for that before putting the key/value pair into the dictionary.

Answer 6

shouldn't line.startswith('#') better read line.strip().startswith('#')?

dict(line.strip().split('=') for line in open('/Path/filename.properties') 
                             if not line.strip().startswith('#'))