What are the best practices for implementing a file writer/logger in Java that is compatible with logrotate? The goal would be allow logrotate to be used for all log management, instead of using built-in rotation/management of a logging API (Log4J, etc).
I'd be interested in hearing comments/answers for other development platforms, aside from Java.
You simply need to periodically close and re-open the log file inside your application. You need a handler that keeps last close time. The handler should close and reopens the file if (for example) 20 seconds passed since last close and log entry is about to be written. It should make such check just before writing the log entry
If you don't do it, the logs will be written to the old file even if it is renamed by logrotate (!) (file descriptor remains the same) and then later the log entries will disappear when the log is compressed and removed (in such case java will silently drop such logs).
Closing and reopening the log (using file name) will make sure that if the file has been renamed a new one will be created. Closing and reopening the file every time log is written is an overkill because opening a file is a costly operation.
I looked at some other apps and apparently there is a postrotate option and a copytruncate option. The copytruncate option is easier but I think it is less reliable as buffers may not be flushed (ie follow @Jarek Potuik answer) and even if they are you may get duplicate entries or dropped entries.
Thus lets assume you want the postrotate option:
The postrotate option allows you to run a command to tell your application to reopen the file (close and then open). While opening and closing the file in Java you would do this with a synchronized block or some sort of lock and of course flush any buffers.
Lets assume your app is called myapp:
You would have a file in /etc/logrotate.d/myapp with something like:
/var/log/myapp/*.log {
weekly
missingok
rotate 20
compress
delaycompress
notifempty
sharedscripts
postrotate
/etc/init.d/myapp rotate-logs > /dev/null
endscript
}
The command /etc/init.d/myapp rotate-logs would need to signal to the app to close and reopen the log files.
The signaling part is fairly tricky and depends on what kind of app you have because Java doesn't support IPC that well (ie unix signals) so you may have to open a port or use an existing port such as 8080 (ie HTTP REST command in the case of a servlet container) to tell your app to close and reopen the log files.
