How to design a program with many configuration options?

Question

Lets say I have a program that has a large number of configuration options. The user can specify them in a config file. My program can parse this config file, but how should it internally store and pass around the options?

In my case, the software is used to perform a scientific simulation. There are about 200 options most of which have sane defaults. Typically the user only has to specify a dozen or so. The difficulty I face is how to design my internal code. Many of the objects that need to be constructed depend on many configuration options. For example an object might need several paths (for where data will be stored), some options that need to be passed to algorithms that the object will call, and some options that are used directly by the object itself.

This leads to objects needing a very large number of arguments to be constructed. Additionally, as my codebase is under very active development, it is a big pain to go through the call stack and pass along a new configuration option all the way down to where it is needed.

One way to prevent that pain is to have a global configuration object that can be freely used anywhere in the code. I don't particularly like this approach as it leads to functions and classes that don't take any (or only one) argument and it isn't obvious to the reader what data the function/class deals with. It also prevents code reuse as all of the code depends on a giant config object.

Can anyone give me some advice about how a program like this should be structured?

Here is an example of what I mean for the configuration option passing style:

class A:
    def __init__(self, opt_a, opt_b,  ..., opt_z):
        self.opt_a = opt_a
        self.opt_b = opt_b
        ...
        self.opt_z = opt_z

    def foo(self, arg):
        algo(arg, opt_a, opt_e)

Here is an example of the global config style:

class A:
    def __init__(self, config):
        self.config = config

    def foo(self, arg):
        algo(arg, config)

The examples are in Python but my question stands for any similar programming langauge.

Answer 1

matplotlib is a large package with many configuration options. It use a rcParams module to manage all the default parameters. rcParams save all the default parameters in a dict.

Every functions will get the options from keyword argurments:

for example:

def f(x,y,opt_a=None, opt_b=None):
    if opt_a is None: opt_a = rcParams['group1.opt_a']

Answer 2

A few design patterns will help

• Prototype
• Factory and Abstract Factory

Use these two patterns with configuration objects. Each method will then take a configuration object and use what it needs. Also consider applying a logical grouping to config parameters and think about ways to reduce the number of inputs.

psuedo code

// Consider we  can run three different kinds of Simulations. sim1, sim2, sim3

ConfigFactory configFactory = new ConfigFactory("/path/to/option/file");
....
Simulation1 sim1;
Simulation2 sim2;
Simulation3 sim3;

sim1.run( configFactory.ConfigForSim1() );
sim2.run( configFactory.ConfigForSim2() );
sim3.run( configFactory.ConfigForSim3() );

Inside of each factory method it might create a configuration from a prototype object (that has all of the "sane" defaults) and the option file becomes just the things that are different from default. This would be paired with clear documentation on what these defaults are and when a person (or other program) might want to change them.

** Edit: ** Also consider that each config returned by the factory is a subset of the overall config.

Answer 3

Pass around either the config parsing class, or write a class that wraps it and intelligently pulls out the requested options.

Python's standard library configparser exposes the sections and options of an INI style configuration file using the mapping protocol, and so you can retrieve your options directly from that as though it were a dictionary.

myconf = configparser.ConfigParser()
myconf.read('myconf.ini')
what_to_do = myconf['section']['option']

If you explicitly want to provide the options using the attribute notation, create a class that overrides __getattr__:

class MyConf:
    def __init__(self, path):
        self._parser = configparser.ConfigParser()
        self._parser.read('myconf.ini')
    def __getattr__(self, option):
        return self._parser[{'what_to_do': 'section'}[option]][option]

myconf = MyConf()
what_to_do = myconf.what_to_do

Answer 4

Have a module load the params to its namespace, then import it and use wherever you want.
Also see related question here