Logging module not writing to file

Question

I'm using logging module, and I've passed in the same parameters that I have on other jobs that are currently working:

import logging
from inst_config import config3

logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s [%(levelname)s] - %(message)s',
    filename=config3.GET_LOGFILE(config3.REQUESTS_USAGE_LOGFILE))
logging.warning('This should go in the file.')

if __name__ == '__main__':
    logging.info('Starting unload.')

Using this method to create the filename:

REQUESTS_USAGE_LOGFILE = r'C:\RunLogs\Requests_Usage\requests_usage_runlog_{}.txt'.format(
        CUR_MONTH)
def GET_LOGFILE(logfile):
    """Truncates file and returns."""
    with open(logfile, 'w'):
        pass
    return logfile

When I run it, however, it is creating the file, and then still outputting the logging info to the console. I'm running in Powershell.

Just tried putting it inside the main statement like this:

if __name__ == '__main__':
    logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s [%(levelname)s] - %(message)s',
    filename=config3.GET_LOGFILE(config3.REQUESTS_USAGE_LOGFILE))

    logging.warning('This should go in the file.')

Still no luck.

can you show us the code you're usign to write to the log file? — dot.Py, Mar 09 '16 at 17:17
Have you created the log file before the `if __name__ == '__main__':` runs? I mean... if you run your program, it'll know where's the log file? Can you make a test by putting the `logging.basicConfig()` and creating the log file inside the `if __name__ == '__main__':` block? — dot.Py, Mar 09 '16 at 17:28
Sure I'll try that. EDIT: I placed it inside the main statement, and still printing to the console:/ — flybonzai, Mar 09 '16 at 17:29

score 34 · Answer 1 · edited Aug 14 '18 at 14:54

34

I add the following lines before the logging.basicConfig() and it worked for me.

for handler in logging.root.handlers[:]:
    logging.root.removeHandler(handler)

edited Aug 14 '18 at 14:54

Barry

3,303
7
23
42

answered Aug 14 '18 at 14:27

yue dong

597
5
6

1

@yue dong - Can you explain why this solved the problem (p.s. - it solved for me as well) ? – Guy Avraham Sep 12 '19 at 07:19
@yue dong, I'm second. Could you explain, why removing handlers from root logger helps? – wl2776 May 14 '20 at 13:35
It works, but what kind of dark magic is in there? – Joel Carneiro Jan 05 '21 at 19:06
Thanks, but this behavior is more like a logging module bug than anything? – Sriram May 15 '21 at 08:25
This didn't work for me and instead broke my code :/ – Mr Pablo Jun 11 '21 at 16:19
So I have a program that often crashed with a segfault (due to c++ externals), and I need to do this. I suspect it's something that's needed when the code doesn't exit cleanly or with a regular python error. – Clumsy cat May 17 '22 at 14:35

Forge · Accepted Answer · 2021-08-14T01:12:30.950

10

You can try running this snippet in your main file.

import logging 
logging.basicConfig(
    level=logging.INFO, 
    format='%(asctime)s [%(levelname)s] - %(message)s',
    filename='filename.txt')  # pass explicit filename here 
logger = logging.get_logger()  # get the root logger
logger.warning('This should go in the file.')
print logger.handlers   # you should have one FileHandler object

edited Aug 14 '21 at 01:12

answered Mar 09 '16 at 17:29

Forge

6,538
6
44
64

1

I didn't include my imports, but it is indeed loaded at the top of the file. I've added my imports to the OP. – flybonzai Mar 09 '16 at 17:32
Still didn't work. It's not creating the file since it's never writing to it. Could the problem be powershell? The weird thing is I have other jobs with the exact same configuration that run without a hitch. – flybonzai Mar 09 '16 at 17:42
1

You say here that _it's not creating the file_, but in the main post you say _When I run it, however, it is creating the file_. Which is it? – John Gordon Mar 09 '16 at 18:15
That led me to the solution @Forge, for some reason it defaulted to a streaming handler this time around, so I added a `fileHandler`. – flybonzai Mar 09 '16 at 20:16

score 3 · Answer 3 · answered Jul 13 '21 at 23:09

In addition to Forge's answer on using logging.basicConfig(), as of Python 3.8 a parameter to basicConfig() got added. To quote the docs:

"""
This function does nothing if the root logger already has handlers 
configured, unless the keyword argument *force* is set to ``True``.
...
force     If this keyword  is specified as true, any existing handlers
          attached to the root logger are removed and closed, before
          carrying out the configuration as specified by the other
          arguments.
"""

This is why yue dong's answer to remove all handlers has worked for some as well as Alexander's answer to reset logger.handlers to [].

Debugging logger.handlers (as Forge's answer suggests) led me to see a single StreamHandler there (so basicConfig() did nothing for me until I used force=True as parameter).

Hope that helps in addition to all the other answers here too!

Alexander Borochkin · Answer 4 · 2019-01-23T10:14:07.683

If you are using 'root' logger which is by default has name "", than you can do this trick:

logging.getLogger().setLevel(logging.INFO)    
logger = logging.getLogger('')
logger.handlers = []

In addition you may want to specify logging level as in code above, this level will persist for all descendant loggers.

If instead, you specified particular logger, than do

logger = logging.getLogger('my_service')
logger.handlers = []
fh = logging.FileHandler(log_path)
fh.setLevel(logging.INFO)
# create console handler
ch = logging.StreamHandler()
ch.setLevel(logging.INFO)
logger.addHandler(fh)
logger.addHandler(ch)
logger.info('service started')

The above code will create new logger 'my_service'. In case that logger has been already created it clears all handles. That it adds handles for writing in specified file and console. See official documentation as well.

You can also use hierarchical loggers. It is done directly.

logger = logging.getLogger('my_service.update')
logger.info('updated successfully')

bhargav3vedi · Answer 5 · 2023-08-04T10:53:24.933

0

import logging

logging.basicConfig(filename="logs.log", filemode="w", format="%(asctime)s %(message)s", level=0, force=True)

for i in range(2):
    print(f"hello {i}")
    logging.info(f"hello {i}")

set level=0, filemode="a" or "w". if you set filemode w then it will overwrite previous logs but if you want to append logs, you must use filemode="a".

edited Aug 04 '23 at 10:53

answered Aug 03 '23 at 05:15

bhargav3vedi

521
1
6
11

Logging module not writing to file

5 Answers5

Linked