How to raise exception instead of returning None in Python "ConfigParser" section?

Question

I have a problem using the configparser module.

import configparser

config = configparser.ConfigParser()
config.read_dict({"foo": {}})

foo = config["foo"]
foo.getboolean("missing_field")

I would like my code to raise an exception if the parsed configuration is missing a required field. However, getboolean() returns None in this case instead of raising a KeyError as expected.

I could possibly use foo["missing_field"] which does raise an exception. However, in such case, I loose the boolean conversion.

I could explicitly test for if res is None: and throw the exception manually but I have a lot of config fields so that would be cumbersome.

Does Python provide an elegant way to force strict config parsing?

Kemp · Accepted Answer · 2020-12-08T15:46:21.637

4

You can use getboolean on the config object directly:

config.getboolean("foo", "missing_field")

which will raise a NoOptionError if missing_field doesn't exist:

configparser.NoOptionError: No option 'missing_field' in section: 'foo'

The different behavior for getboolean on the proxy is because it calls the relevant getter with a default fallback=None. The problem is that the "regular" get uses a special _UNSET object as the default fallback and then does:

if fallback is _UNSET:
    raise NoOptionError(option, section)
else:
    return fallback

So, as an alternative (which came up in the discussion for @Tomerikoo's answer and @chepner suggested in a comment to this answer), you can pass in _UNSET as the fallback value when using the section proxy. As per your original code:

foo.getboolean("missing_field", fallback=configparser._UNSET)

edited Dec 08 '20 at 15:46

answered Dec 08 '20 at 14:19

Kemp

3,467
1
18
27

2

You can (re)set the fallback to force the exception: `foo.config("missing_field", fallback=configparser._UNSET)`. – chepner Dec 08 '20 at 14:55
(Oh, didn't see that Tomerikoo suggested the same thing while I was researching it.) – chepner Dec 08 '20 at 14:56
Both options still require to change every access in the code so I still think my class is the best solution ;D Just kidding... – Tomerikoo Dec 08 '20 at 14:58

Tomerikoo · Answer 2 · 2020-12-08T14:45:37.253

4

Well... @Kemp found a tweaky behaviour that solves this easily, but as I already went through the following trouble, I might as well just post it...

The default behavior doesn't raise an exception, but why not "fix" it according to your needs? You can create a sub-class of ConfigParser that only wraps getboolean with a check if the option actually exists and then returns the original getboolean value. I think the nice thing about this option is that you don't need to change your code at all - just change the initial config = ... line to use the new class:

import configparser

class MyConfig(configparser.ConfigParser):
    def getboolean(self, section, option, *, raw=False, vars=None,
                   fallback=configparser_UNSET, **kwargs):
        if self.get(section, option, raw=raw, vars=vars, fallback=fallback) == fallback:
            raise configparser.NoOptionError(option, section)

        return super().getboolean(section, option, raw=raw, vars=vars, fallback=fallback)

Now doing:

config = MyConfig()
config.read_dict({"foo": {'existing_field': '1'}})

foo = config["foo"]
print(foo.getboolean("existing_field"))
print(foo.getboolean("missing_field"))

Will give (truncated):

True
Traceback (most recent call last):
...
configparser.NoOptionError: No option 'missing_field' in section: 'foo'

As opposed to:

True
None

With the regular configparser.ConfigParser.

edited Dec 08 '20 at 14:45

answered Dec 08 '20 at 14:27

Tomerikoo

18,379
16
47
61

1

The `getboolean` implementation on `ConfigParser` already has this behaviour, it's the proxy class it returns when you access a section via the mapping protocol that (strangely) doesn't have this behaviour. *Edit: I've seen your update now and I'm intrigued as to how this works. Taking a look.* – Kemp Dec 08 '20 at 14:30
Problem is that it will also require implementing `get()`, `getint()` and `getfloat()` which is quite cumbersome just to circumvent a poor API design. But yeah, that's a possible workaround. – Delgan Dec 08 '20 at 14:33
1

Well it definitely works, though I'm not exactly sure why. One modification I would suggest is to raise `configparser.NoOptionError(option, section)` to match the behaviour of the base `ConfigParser`. – Kemp Dec 08 '20 at 14:39
1

@Kemp I couldn't find `getboolean` in the `SectionProxy` class, but `get` has `fallback=None` as opposed to `fallback=_UNSET` in the `RawConfigParser`... Seems to be the problem. Under the section proxy the fallback is always None. So another possible workaround to call `foo.getboolean("missing_field", fallback=configparser._UNSET)` – Tomerikoo Dec 08 '20 at 14:41
@Kemp yep the above works as well... setting `fallback=_UNSET` – Tomerikoo Dec 08 '20 at 14:43
Ah, the proxy uses a different default fallback than the config parser, I didn't catch that. That definitely feels like a bug to me. I think in your comment you've just hit on an answer that better matches the original question than either of our actual answers :) It has the drawback of pulling in a private module variable, which can feel wrong (and your IDE will likely complain), but it is a valid answer in my opinion. – Kemp Dec 08 '20 at 14:51
@Kemp I was going to offer you to add it in your answer as it seems more relevant. The two options are call the `config`'s `get` and pass the section, or call the `section`'s `get` and pass a `fallback` – Tomerikoo Dec 08 '20 at 14:53
I've added it to mine as an alternative. I think you're right that it makes sense to keep similar answers together. – Kemp Dec 08 '20 at 15:05
@Kemp mind if I edit in to your answer some clarifications? I already started writing something up. Wouldn't want it to go to waste. Of course you can feel free to change it after – Tomerikoo Dec 08 '20 at 15:07
@Kemp by the way the reason why my class even works, is because the implementation chosen for `getboolean` under the proxy's `get` is the one from `MyConfig` instead of the original faulty in `RawConfigParser` – Tomerikoo Dec 08 '20 at 15:21
Looking at the code, `RawConfigParser.get` already uses `_UNSET` as the default fallback. `SectionProxy.get` uses `None` though, and it provides an implementation of `getboolean` through some magic I haven't dissected that ends up calling its own `get` with the `None` fallback. I guess now I'm back to not understanding why your solution works :D – Kemp Dec 08 '20 at 15:38
Oh, because you check if the returned value is equal to the passed in fallback value, while the original class only checks if it's equal to _UNSET, which it won't be when the proxy is involved! Ok, I'm getting there eventually. – Kemp Dec 08 '20 at 15:42
@Kemp lol. I figured it out. In the proxy `init`, there is a loop doing `setattr` to all different getters back to the single `get` with a specific implementation (using `partial`). Then, in its turn, `get` calls that specific implementation that somewhere along the way calls the `RawConfigParser`'s `get`. I "injected" my own `getboolean` as the implementation passed to `get` for the `getboolean` – Tomerikoo Dec 08 '20 at 15:42

How to raise exception instead of returning None in Python "ConfigParser" section?

2 Answers2