ConfigObj option validation

Question

I am using ConfigObj and Validator to parse a configuration file in python. While I like this tool a lot, I am having trouble with validation using a configSpec file. I am using the option() configSpec type that forces the value to be chosen from a controlled vocabulary:

output_mode = option("Verbose", "Terse", "Silent")

I want my code to know when the user enters an option that's not in the CV. From what I have fond, Validator only seems to say which config key failed validation, but not why it failed:

from configobj import ConfigObj, flatten_errors
from validate import Validator

config = ConfigObj('config.ini', configspec='configspec.ini')
validator = Validator()
results = config.validate(validator)

if results != True:
    for (section_list, key, _) in flatten_errors(config, results):
        if key is not None:
            print 'The "%s" key in the section "%s" failed validation' % (key, ', '.join(section_list))
        else:
            print 'The following section was missing:%s ' % ', '.join(section_list)

That code snippet works but there are any number of reasons why a key might have failed validation, from not being in an integer range to not being in the CV. I don't want to have to interrogate the key name and raise a different kind of exception depending on the failure cases for that key. Is there a cleaner way to handle specific types of validation errors?

Long time stackoverflow reader, first time poster :-)

Answer 1

Update: I think this does what I want to do. The key is that config obj stores errors as Exceptions which can then be checked against those that subclass ValidateError. Then you just have to do one check per subclass rather than one check per parameter value. It might be nicer if validate just threw an exception if validation failed but maybe you would lose other functionality.

self.config = configobj.ConfigObj(configFile, configspec=self.getConfigSpecFile())
validator = Validator()
results = self.config.validate(validator, preserve_errors=True)

for entry in flatten_errors(self.config, results):

   [sectionList, key, error] = entry
   if error == False:
      msg = "The parameter %s was not in the config file\n" % key
      msg += "Please check to make sure this parameter is present and there are no mis-spellings."
      raise ConfigException(msg)

   if key is not None:
      if isinstance(error, VdtValueError):
         optionString = self.config.configspec[key]
         msg = "The parameter %s was set to %s which is not one of the allowed values\n" % (key, self.config[key])
         msg += "Please set the value to be in %s" % optionString
         raise ConfigException(msg)

OptionString is just a string of the form option("option 1", "option 2") rather than a list so to get this to look nice, you need to grab the substring in the ()'s.

Answer 2

For future reference for anyone interested, you could also check for extraneous data. This can be handled with the get_extra_values function. The complete example shown below hence does:

• load the configuration with validator
• look for all the validated errors
• verify extra values

from configobj import ConfigObj, ConfigObjError, flatten_errors, get_extra_values
from validate import Validator, VdtValueError


def load_config(configfile, configspec, raise_exception=True):
    "Load and check configvale acccording to spec"

    config = ConfigObj(configfile, file_error=True, configspec=configspec)
    validator = Validator()
    results = config.validate(validator, preserve_errors=True)

    msg = ""
    fatalerr = False

    for entry in flatten_errors(config, results):

        [sectionList, key, error] = entry
        if error is False:
            msg += f"\n{key:>30s} missing in section [{']['.join(sectionList)}]"
            fatalerr = True

        if key is not None:
            if isinstance(error, VdtValueError):
                optionString = config.configspec[key]
                msg += f"\nThe parameter {key} was set to {[config[s][key] for s in sectionList]} which is not one of the allowed values\n"
                msg += "    Please set the value to be in %s" % optionString
                fatalerr = True

    # verifying extra values below

    wmsg = ""
    for sections, name in get_extra_values(config):

        # this code gets the extra values themselves
        the_section = config
        for section in sections:
            the_section = the_section[section]

        # the_value may be a section or a value
        the_value = the_section[name]

        section_or_value = 'value'
        if isinstance(the_value, dict):
            # Sections are subclasses of dict
            section_or_value = 'section'

        section_string = '[' + (']['.join(sections) or "TOP LEVEL") + ']'

        wmsg += f"\n{name:>30s}: Extra {section_or_value} on section {section_string}"

    if wmsg != "":
        print(f"\nWARNINGS found in configuration file {configfile}")
        print(wmsg)

    if fatalerr:
        print(f"\nERRORS found in configuration file {configfile}")
        if raise_exception:
            raise RuntimeError(msg)
        else:
            print("Fatal errors found, but no exception raised, as requested")
            print(msg)

    print(f'Configuration {configfile} validated successfully')
    return config


if __name__ == "__main__":

    configfile="xt_default.cfg"
    configspec="xt_default_spec.cfg"
    
    config = load_config(configfile, configspec)