How can I specify output file size which I want after rotation? For example lets say I want to run logrotate when log file reaches 10MB and I want to split this log into chunks which are 1MB max. I have found options "size" but this sets limit above which logrotate is run and not the size of desired chunk. Thanks
1 Answers
logrotate by itself doesn't support the splitting of the rotated log/s, only compression.
You may be able to get the desired behaviour by writing a script of your own that handles the splitting, and call that as a postrotate script in your logrotate configuration for that particular file/s.
update: based on your comment ("it is too late to split files, because they will be splitted by logrotate"), it seems you still have the idea that logrotate is doing splitting of files. This isn't correct, what actually happens is that logrotate renames the existing file and creates a new empty file with the original name.
An example:
/var/log/syslog
{
rotate 7
daily
missingok
notifempty
delaycompress
compress
postrotate
reload rsyslog >/dev/null 2>&1 || true
endscript
}
When logrotate runs:
- it renames /var/log/syslog to /var/log/syslog.1
- it then creates a new empty file /var/log/syslog.
- renaming a file has no effect on any processes that had an open file handle, so after doing this, the rsyslog process is still writing to the old file, syslog.0. Typically, the way to get around this is to reload the process/es that write to the file, which is what happens in the postrotate script. rsyslog is told to reload, it closes its existing file handle to /var/log/syslog.0, and opens a new file handle using the filename /var/log/syslog, the new empty file created by logrotate.
Hopefully you have a better understanding of what logrotate is doing now.
update 2: I'll address your comments here, as formatting options in the comments section are limited.
So when I understand right in the post processing part it is necessary to reload rsyslog to enshure that old file descriptor (before rotation) will be closed and new file descriptor (after rotation) will be opened (because of: mv syslog syslog.1)?
100% correct. After logrotate moves the file, rsyslog is still writing to the old file descriptor. A process performs I/O on the file descriptor, the name is only of interest when performing the initial open on the file.
I thought that logrotate has some mechanism to achieve this automatically.
No, hence the need to do it in a postrotate script. Remember that not all logging is necessarily done by a process which keeps an open file descriptor. If you have a process which opens a file, writes to it and then closes (the file descriptor) repeatedly, then you needn't fuss with anything in the postrotate.
PS: Did you mean "syslog.0" not "syslog" in the following senetence: "the rsyslog process is still writing to the old file, syslog.0."? Becuse "old" log is /var/log/syslog not /var/log/syslog.0. Thank you
No. I mean that it's still writing to the existing file descriptor, which logrotate has renamed to syslog.0. When dealing with processes like rsyslog, always remember that the file name is only relevant at the point when rsyslog opens it, i.e. when it creates the file descriptor. After that you can rename, move, even delete the file, and the process continues performing I/O on the file descriptor unhindered.

- 15,473
- 12
- 53
- 79
-
I do not understand you well. How can I write postprocessing script which handles the splitting? I need to split files first and then compress them. When logrotate is done it is too late to split files, because they will be splitted by logrotate. Or am I missing something, what is the basic idea of logrotate? It just compress files after they reaches specific size and renames them to {file}.0.log (very basic overview)? I have read several tutorials about logrotate but now I am a little bit confused. – Wakan Tanka Jun 07 '13 at 12:46
-
I updated my answer with some more info, hope it clears up the confusion. – ThatGraemeGuy Jun 07 '13 at 13:17
-
So if I understand right in the post processing part it is necessary to reload rsyslog to enshure that old file descriptor (before rotation) will be closed and new file descriptor (after rotation) will be opened (because of syslog was moved to syslog.1 and syslog was created and thus is in fact new file)? I thought that logrotate has some mechanism to achieve this automatically. PS: Did you mean "syslog.0" not "syslog" in the following senetence: "the rsyslog process is still writing to the old file, syslog.0."? Becuse "old" log is /var/log/syslog not /var/log/syslog.0. Thank you – Wakan Tanka Jun 13 '13 at 15:53
-
One more note: is it safe to move file on which is opened file descroptor? In unix terminology I hope that logrotate did not do "mv syslog syslog.0 && touch syslog" but rather "cat syslog > syslog.0 && echo > syslog", is the reload of rsyslog needed in this case? Thank you – Wakan Tanka Jun 13 '13 at 15:54
-
One more question: Does logrotate somehow tracks the changes of the log so it is smart enought to know that if it was no change from last time logs was rotated there is no need to do it again? – Wakan Tanka Jun 13 '13 at 16:03
-
I addressed your questions in my answer, since the comments section is quite limited. – ThatGraemeGuy Jun 13 '13 at 16:06
-
Thank you for reply, can you please clarify how files are moved (my 3rd comment, I have several times edited comments last time, sorry for that). For clarification in your reply "100% correct. After logcheck moves the file," what does logcheck to do with logrotate? It is something completely different AFAIK. – Wakan Tanka Jun 13 '13 at 16:29
-
One more note: logrotate does have option called "copytruncate" which will preserve the original file name, I think what it does is something like that was described in my 3rd comment. If it was not that way there may be serious problems with deleted files and opened file descriptors which may cause the full of file system, correct me if I am wrong. Thanks for all help. – Wakan Tanka Jun 13 '13 at 16:30
-
Sorry, not I meant logrotate, not logcheck. I had a brainfart. :-) They way it moves the file is pretty much the same way 'mv' works. 'copytruncate' does work the way you described, but depending on the volume of logging, you run the risk of losing lines in your log. This is mentioned in the logrotate man page. Yes, you can run out of space if you use the default method of rotation and do not take care to clean up in a postrotate script (unless you opt to use 'copytruncate' which avoids the problem). – ThatGraemeGuy Jun 14 '13 at 08:04