10

I have a long running process, that writes its log file to stdout. I would like to save this output to different files, automatically maintain these files (like deleting/archiving the old ones), without restarting the main process.

On solution would be to send the output to a file (process > log.txt), and use logrotate on it, but logrotate needs to restart the program, which is not possible.

The other one is to pipe the output to cronolog (process | cronolog), but in this case older files won't get deleted / archived, meaning I have to make a program that will do the maintenance for me.

The best would be to be able to use both utilities, because with cronolog I don't need to restart the process, and logrotate will maintain the old log files exactly as I want. Is there a way to get these two programs to work with each other? If not, what is a good solution to this problem?

SztupY
  • 265
  • 3
  • 13
  • 1
    This http://serverfault.com/questions/189477/rotate-logs-of-a-dumb-non-interactive-application/189880#189880 probably answers your question – user9517 Nov 03 '12 at 19:00
  • [systemd](http://www.freedesktop.org/software/systemd/) and [the journal](http://0pointer.de/blog/projects/the-journal.html) should be able to fix this when they'll be available in production. systemd will redirect `stdout` to the journal and the journal will rotate the logs when needed. – Cristian Ciupitu Nov 03 '12 at 19:22

1 Answers1

7

Multilog from DJB's daemontools can do (almost) exactly what you are asking. The only downside that I am aware of is that not many distributions carry a package for daemontools.

If you are not managing your application with svc (part of the daemontools) you will have to find a way to pipe the output to a command like

multilog t s1048576 n100 ./my_log_directory

This translates to:

  • t: insert a tai64n timestamp (that can be translated to readable time with tai64nlocal)
  • s1048576: rotate the log file when it grows to 1MiB
  • n100: keep no more than 100 rotated files
  • ./my_log_directory: anything starting with . or / - write the log to that directory

The log that is written will have the filename current, and when multilog rotates the log, it will be renamed to @<tai64n timestamp>.s where the filename shows the time when the file was rotated. The extension can be .s if the file was safely flushed, or .u if it may have been truncated.

For more info, just check the links.

Update:

svlogd from runit is a replacement for multilog with similar command line usage.

s6-log from s6 is another fork with more features and compatible command line usage.

chutz
  • 7,888
  • 1
  • 29
  • 59
  • 1
    Seems like the command I'm searching for. Fortunately it's available under Ubuntu: https://launchpad.net/ubuntu/+source/daemontools – SztupY Nov 05 '12 at 15:41