Modifying python variables based on config file entries

Question

Relative Python newbie, and I'm writing a script that takes as its input a csv file, splits it into its constituent fields line-by-line and spits it out in another format. What I have so far generally works very well.

Each incoming csv line has specific fields that are read into variables 'txnname' and 'txnmemo' respectively. Before I write out the line containing these, I need to test them for specific string sequences and modify them as necessary. At the moment I have these tests hard coded into the script as follows

if ('string1' in txnname) and ('string2' in txnmemo):
    txnname = 'string3'
if 'string4' in txnname:
    txnname = 'string5 ' + txnmemo

'stringx' is being used here replacing the actual text I'd like to use.

What I'd really like to do is to remove the search strings, variable names and modifier strings from the script altogether and place them in a config file that already exists alongside the script and contains various parameters read into the script via ConfigParser. That way I won't have to modify the code every time I want to test a new condition. The 'hacky' way of doing this would seem to be using eval/exec in some combination on the literal Python statements read in from the config file, however just about everybody says that there is nearly always a better alternative to eval/exec. However I can't come up with what this alternative would be despite much research. I've read mention of using dictionaries somehow, but can't get my head round how this would work, especially on an 'If' statement with more than one condition to be tested (as above)

Any advice gratefully received.

Answer 1

There are a few options.

The simplest way is to create a global config module with your string variables and modifiers defined and import it into this module. Basically that will just centralize all these variables into a single location so when you need to modify, you just change the config module.

The other option is by using configparser as you noted. In that case, you'd need to create an INI file with a format something like below. There are different sections and keys in each section.

example.ini

[DEFAULT]
variable_to_access = value

[INPUT]
txnname_mod = string1
txnmemo_mod = string2

[MODIFY]
txnname_mod = string3
txnmemo_mod = string4

Now you can read the INI file and access variables like a dictionary.

Python

>>> import configparser
>>> config = configparser.ConfigParser()
>>> config.sections()
[]
>>> config.read('example.ini')
>>> config['INPUT']['txnname_mod']
string1

For more info, check out the documentation here.

Answer 2

Creating a config file and using configparser is pretty simple and easy to do https://docs.python.org/2/library/configparser.html. The configparser already puts everything into a dictionary for you. Convert ConfigParser.items('') to dictionary

The way to retrieve a value from a dictionary is pretty simple and probably the fastest approach.

Example usage

renames = {"string1": "string3",
           "string2": "string3",
           "string4": "string5 " + txnmemo}

# new_value = renames[txnname] # raises KeyError if txnname not in renames
new_value = renames.get(txnname, txnname) # txname is the default value if txnname is not in renames

If you are using the configparser approach then you will not be able to create "string5 " + txnmemo as a dictionary result. For this case make your config file result be "string5 *unique_text*" then replace that text.

new_value = renames.get(txnname, txname).replace("*unique_text*", txnmemo)

I would just make the configfile as simple as possible

[Renames]
string1 = string3
string2 = string3
string4 = string5 *unique_text*

configparser

import configparser
config = configparser.ConfigParser(allow_no_value=True, inline_comment_prefixes=(";"))
config.optionxform = str # Prevents key case change
config.read('rename.ini')

with open("file.csv", "r") as file:
    for line in file:
        txnname, txnmemo = line.split(",")
        txnname = config["Renames"].get(txnname, txnname)
        txnname = txnname.replace("*unique_text*", txnmemo)

Answer 3

You can indeed use ConfigParser to read from .cfg files. That way you can read and write data on the .cfg file using config.get() and config.write()