0

Lets say I create a rotating file logger with python using RotatingFileHandler. I set number of files to 10, and each file size to 1 MB. My question is related to how the rotation happens. Is it that the rotation happens by keeping all the 10 files open and ensuring the contents of each file is modified as and when a line gets added to the latest file? If that is so, then there should be at least 10 file descriptors that are always open.

Say each file has 100 KB chunk rows, so each has just 10 rows. So when another row (of 100 KB) is inserted through this rotating log handler, wouldn't the last row of each file be put at the top of the next file (from newer to older)? So it makes sense to keep all the file descriptors open all the time, doesn't it?

Ouroboros
  • 1,432
  • 1
  • 19
  • 41

1 Answers1

1

The handler only ever has one open file. The other files are renamed when rotating.

First, the current file is closed. Then, rotation renames files in reverse order; so the file with extension .9 is renamed to .10 first, deleting an already existing .10 if it is there. Then .8 is renamed to .9, etc. Finally, the 'current' file is renamed to append the .1 extension.

Depending on the delay flag, at the end a new file is opened, either when rotating or when the next log entry is to be written.

All this is included in the logging.handlers.RotatingFileHandler() documentation

When the size is about to be exceeded, the file is closed and a new file is silently opened for output. [...] If backupCount is non-zero, the system will save old log files by appending the extensions ‘.1’, ‘.2’ etc., to the filename. For example, with a backupCount of 5 and a base file name of app.log, you would get app.log, app.log.1, app.log.2, up to app.log.5. The file being written to is always app.log. When this file is filled, it is closed and renamed to app.log.1, and if files app.log.1, app.log.2, etc. exist, then they are renamed to app.log.2, app.log.3 etc. respectively.

Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
  • Thanks, can you let me know what happens if I don't call the Logger.removeHandler(..) for this rotating file handler? Would the garbage collection happen ultimately for the open file descriptor? – Ouroboros Jan 23 '15 at 15:24
  • @P.Prasad: yes, if nothing else references the handler anymore the object is cleared up, together with the open file object it holds. – Martijn Pieters Jan 23 '15 at 15:27