Handling logrotate in open()

Question

I am using the following function to follow and process a logfile line by line:

def tail(f):
    f.seek(0,2)
    while True:
        line = f.readline()
        if not line:
            time.sleep(0.1)
            continue
        yield line

with open("/var/log/some.log") as f:
    for line in tail(f):
    ...

When logrotate rotates the file, that function is stuck at the last line. I guess logrotate moves the file and creates a new one, leaving the program still waiting for lines from the old one.

What is a good way to handle this - reopen the file when rotation occurs?

rolf82 · Accepted Answer · 2020-04-18T12:50:35.883

The standard usual way is to tell logrotate to send the HUP signal to the service/program that handles the logs.

In logrotate configuration this is done with a postrotate script:

   postrotate
       /usr/bin/killall -HUP your_process
   endscript

Then in your python program you'll have to handle the signal HUP. You have an example in python's signal doc. You'll want to handle signal.SIGHUP

Another, less standard way to do that would be to compare the inode number to the (maybe) new inode number. If these numbers differ then you know that the file has changed and you need to reopen it.

I tried something like this and it seemed to work:

import os
import time

logname = "logfile"
logfile = open(logname, "r")

def tail(f):
    f.seek(0,2)
    inode = os.fstat(f.fileno()).st_ino

    while True:
        line = f.readline()
        if not line:
            time.sleep(0.1)
            if os.stat(logname).st_ino != inode:
                f.close()
                f = open(logname, 'r')
                inode = os.fstat(f.fileno()).st_ino
            continue
        yield line

for line in tail(logfile):
    ...

That's the way i am doing it at the moment, i was hoping for a more pythonic way. — tilkov, Apr 10 '20 at 16:47
You could check for inode number of the file with stat. If the inode number has changed when your loop exits then you would re-run the loop on the new file. I don't know if it is more pythonic as it is very close from system. If that's what you were thinking I'll edit my answer to elaborate on that — rolf82, Apr 11 '20 at 07:37

Handling logrotate in open()

1 Answers1