Python logging file config KeyError: 'formatters'

Question

I'm currently working on a python project and I set up logging using a config file. It has already worked and was logging my messages as wanted.

But then, after rearranging some of the packages and modules, I only get a key error.

Full Traceback:

    Traceback (most recent call last):
  File "/Volumes/Daten/Eclipse/workspace/Carputer/src/pyboard/__init__.py", line 42, in <module>
    logging.config.fileConfig('../logging.conf', disable_existing_loggers=False)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/logging/config.py", line 70, in fileConfig
    formatters = _create_formatters(cp)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/logging/config.py", line 103, in _create_formatters
    flist = cp["formatters"]["keys"]
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/configparser.py", line 937, in __getitem__
    raise KeyError(key)
KeyError: 'formatters'

Here is my logging file:

[loggers]
keys=root,pyBoard

[handlers]
keys=consoleHandler

[formatters]
keys=detailedFormatter

[logger_root]
level=DEBUG
handlers=consoleHandler

[logger_pyBoard]
level=DEBUG
handlers=consoleHandler
qualname=pyBoard
propagate=0

[handler_consoleHandler]
class=StreamHandler
level=DEBUG
formatter=detailedFormatter
args=(sys.stdout,)

[formatter_detailedFormatter]
format=%(asctime)s - %(name)s - %(levelname)s : Line %(lineno)s - %(message)s
datefmt=

And the relevant code:

if __name__ == '__main__':

    logging.config.fileConfig('../logging.conf', disable_existing_loggers=False)
    logger = logging.getLogger(__name__)

    obc = Onboard_computer('/dev/ttys001')
    obc.run()

It is almost the same as from the Python Logging Tutorial. I really don't get why it is not working and it drives crazy. It worked, I changed nothing on the code nor on the setup, and it just stopped working and Python throws this KeyError.

My setup: Mac OS X 10.9.2, Eclipse Kepler with PyDev and Python 3.3. I also tested it on a Raspberry Pi with Raspbian Wheezy and Python 3.2 and in Eclipse with Python 2.7 (same error).

Does anyone of you guys have a clue?

The error suggests it is looking for a different file `logging.config.fileConfig('../logging.ini', disable_existing_loggers=False)` It is looking for one ending in `.ini` where as your code has one ending in `.conf`? — shaktimaan, Apr 18 '14 at 20:35
Typo, sorry... I just installed PyCharm and tested it there with the same installation of Python 3.3.5. And: it works! Any ideas? — moritzrupp, Apr 18 '14 at 20:56
Hi moritz, did you manage to resolve this issue? I'm currently having the same problem when trying to run my application with logging on the RPi. — Joern, Mar 30 '15 at 12:16

d512 · Answer 1 · 2016-07-31T15:25:23.657

110

I had this issue because Python couldn't find my config file, though you would never know it by the error message. Apparently it does not look for the config file relative to the file in which the code is running, but rather relative to the current working directory (which you can get from os.getcwd()). I used the following code to initialize the logger. The log.config file is in the same directory as the file running this code:

from os import path
log_file_path = path.join(path.dirname(path.abspath(__file__)), 'log.config')
logging.config.fileConfig(log_file_path)

edited Jul 31 '16 at 15:25

answered Jul 30 '16 at 23:27

d512

32,267
28
81
107

4

Also check to make sure your logging config file really exists where you think it does. – hughes Mar 23 '19 at 03:41
Yeah, the error message is useless. Thanks to both @d512 and @hughes - what was missing in my case was `include path/to/logging.ini` in my manifest file, and hence the file wasn't included in my wheel package! And it worked on my local machine but died as soon as deployed to AWS Lambda! – BIOStheZerg Jul 30 '19 at 11:13
Thanks, that was it. Up voted your response :). I was able to specify a relative path and it worked. Just for the people who use VS Code with the Python extension from Microsoft: no matter where you class or script is, the default current working directory is the top folder of your workspace. – George Smith Aug 14 '19 at 04:00
1

This was beautiful, it was unable to find proper path no matter what i tried. This worked like a charm. – Biplob Biswas Oct 16 '19 at 13:30

score 6 · Answer 2 · edited Jun 21 '23 at 11:23

d512 was correct. And when running the code and based on the PYTHONPATH, Python treats your project root directory as the 'current path' no matter where is your running file located. So another way is to add the relative path either as following:

import logging
logging.config.fileConfig('root_path_of_project/.../logging.conf')

or relative to the current file:

LOGGING_CONFIG = Path(__file__).parent / 'logging.conf'
...
logging.config.fileConfig(LOGGING_CONFIG)

Hope it helps

score 4 · Answer 3 · edited Oct 22 '21 at 10:03

In my case, I was missing the following from my logging.conf file -

[formatters]
keys = json

Below is the structure of the entire file -

[loggers]
keys = root,__main__

[logger_root]
handlers =

[logger___main__]
level = INFO
handlers = __main__
qualname = __main__

[handlers]
keys = __main__

[handler___main__]
class = StreamHandler
level = INFO
formatter = json
args = (sys.stdout,)

[formatters]
keys = json

[formatter_json]
format=%(asctime)f %(module)s %(levelname)s %(message)s %(funcName)s
class = pythonjsonlogger.jsonlogger.JsonFormatter

Python file that uses the logger -

import logging.config
from os import path

log_file_path = path.join(path.dirname(path.abspath('logging.conf')), 'logging.conf')

logging.config.fileConfig(log_file_path, disable_existing_loggers=False)
logger = logging.getLogger(__name__)

logger.info('random message')  
#{"asctime": "2021-10-22 02:48:26,532", "module": "jsonLogger", "levelname": "INFO", "message": "random message", "funcName": "<module>"}

Don't forget to install python-json-logger btw.

score 2 · Answer 4 · answered Apr 29 '14 at 15:20

Try to replace

logging.config.fileConfig('../logging.conf', disable_existing_loggers=False)

with this

logging.config.fileConfig('logging.conf', disable_existing_loggers=False)

Not sure, but probably your logging.conf is in your current working directory, and with the .. the file cannot be found.

score 0 · Answer 5 · answered Jul 07 '21 at 04:35

Some of my logging settings were in migrations/alembic.ini.

# Logging configuration
[loggers]
keys = root,sqlalchemy,alembic

Instead of messing with it, I deleted the migrations folder, deleted the alembic_version table in my db, and then ran flask db stamp heads and then could do my normal db actions when I add a column again.

# flask db init         # only needed when you don't have any db setup
flask db stamp head
flask db migrate
flask db upgrade

score 0 · Answer 6 · answered Mar 13 '23 at 09:58

In addition to the most voted version a note if you get this error only when installing your package from Pypi:

you need to add your logging file to setup.py package_data field like so:


setup(
    name="...",
    #...
    packages=["mypkg"],
    package_data={
        "mypkg": [
            "logging.conf", # <- otherwise it will not be included in your package
        ]
    },

score -1 · Answer 7 · edited Nov 08 '20 at 16:04

-1

in my case this error was showing up when I was trying to run my flask app on Docker container.

What helped me is adding:

flask db init

into the boot.sh that runs at the beginning of container creation.

edited Nov 08 '20 at 16:04

Dharman

30,962
25
85
135

answered Nov 08 '20 at 15:58

Aleke Coder

1
1

Python logging file config KeyError: 'formatters'

7 Answers7

Linked