Proper way to rotate Nginx logs

Question

I would like to achieve rotation of nginx logs that:

would work without any extra software (i.e. - best if without "logrotate")
would create rotated files with names based on date

Best approach is something like PostgreSQL has - i.e. in its log_filename config variable I can specify strftime-style %Y-%m-%d, and it will automatically change log on date (or time) change.

Another approach from apache - sending logs via pipe to rotatelogs program.

As far as I was able to search - no such approach exists. All I can do, is to use logrotate with dateext option, but it has its own set of drawbacks, and I'd rather use something that works like |rotatelogs or log_filename in PostgreSQL.

This [blog article](http://pjkh.com/articles/nginx-and-cronolog/) describes a possible solution for your problem. But I have a question: Why don't you want to use logrotate? It does the job very well, it has almost no dependencies and is proven to work (battle-hardened if you will). Why jump through hoops and use a homegrown solution which may be inferior and error-prone, if you just could use logrotate (which may also be useful to rotate some other logs on that machine)? — joschi, Sep 14 '09 at 17:57
logrotate (with dateext) *nearly* works, but it i don't like it because it has to be run via cron, and this has some drawbacks. — , Sep 14 '09 at 19:47
Since nginx doesn't support piping its logs to other programs, doesn't support log rotation by itself and you do not like a cron-based approach, you might not quite get what you want. Sometimes "nearly works" is as good as it gets. ;) Unless, of course, you want to patch nginx yourself. — joschi, Sep 14 '09 at 22:32

score 8 · Answer 1 · answered Dec 29 '10 at 09:31

While the world is divided on whether the humble named pipe is friend or foe, it is probably the simplest solution to your problem. It does have a few drawbacks (in that you need to create the pipes ahead of time), but it eliminates the need for a cron and allows you to used the logging pipe-filter of your choice.

Here's an example using cronolog on access.log:

Pick a path for our named pipe. I intend to keep my logs in /var/log/nginx, so I'll put my pipes there as well. The name is up to you; I append .fifo, and it's access.log, so mine will be at /var/log/nginx/access.log.fifo.
Delete the file if it exists.
Make a named pipe for the logfile:
```
mkfifo /var/log/nginx/access.log.fifo
```
Configure nginx.conf to point the log at the pipe you just made:
```
access_log /var/log/nginx/access.log.fifo;
```
Modify your init.d script to start the log rotator listening to the pipe before we start the server:
```
LOGS="/var/log/nginx"
pkill -f "/usr/sbin/cronolog --symlink $LOGS/access.log"
( cat $LOGS/access.log.fifo | /usr/sbin/cronolog --symlink $LOGS/access.log "$LOGS/%Y/%m/%d/access.log" ) &
```
A similar commandline would be used for rotatelogs should you prefer it to cronolog -- see their docs for the syntax.

If your distrobution has a start-stop-daemon, you should be using that instead, as it theoretically has whatever special knoweldge about your platform there is, and taking care of pkill for you. Simply wrap the command in a script, and pass it as --exec to start-stop-daemon in your init.d/nginx.

I love cronolog; it's good to see more people using/recommending it. — natacado, Jul 18 '11 at 07:07

score 2 · Answer 2 · answered Jul 18 '11 at 02:35

I've written a simple program, datelog, to split common logs based on the logged date, as opposed to current system time when the log line is seen by the program. This may or may not be exactly what cronolog or another log splitter does already but it was quicker to write my own than to find out what others do.

Using the year and month in the logged request, the line is then written to a file or pipe which includes the YYYYMM computed from the logged data. Yes this is somewhat specific for the common log format. The first [ is assumed to delimit the date. Beware of IPv6 addresses. :)

For log analysis it is important that each log really only contains the requests for each respective month, and each log should ideally be complete for correct analysis results. It's not enough to determine filename based on current time within the log splitter, because a slow request starting at 23:59:59 will then end up in the log file for the wrong month.

I use this with nginx by way of a named fifo which is checked to exist before nginx is started. Note that there is a tradeoff in the program between error detection and buffered output, where datelog currently prefers buffered output for performance reasons, so please make sure that your setup really works, especially when using shell pipes, in order to not lose any log data.

Source code: http://stuge.se/datelog.c

Please feel free to send me any feedback and of course patches!

score 2 · Answer 3 · answered Feb 27 '13 at 11:39

You can achieve this using a simple bash script and cron:

#!/bin/bash
DATE=$(date +%Y-%m-%d-%H%M)
mv /var/log/nginx/access.log /var/log/nginx/nginx.access.log.$DATE
mv /var/log/nginx/error.log /var/log/nginx/nginx_error.log.$DATE
kill -USR1 `cat /var/run/nginx.pid`
sleep 1
gzip /var/log/nginx/access.log.$DATE
gzip /var/log/nginx/error.log.$DATE

More details on setting up crontab etc. found here: Rotating Nginx log files via Cron

score 1 · Answer 4 · answered Sep 14 '09 at 17:03

1

I'm afraid I don't really understand your question: Since nginx doesn't support any builtin logrotation, you will have to go with something like

mv access.log access.log.$(date "+%Y-%m%d")
kill -USR1 $(cat master.nginx.pid)

somewhere in /etc/cron.daily (you need to qualifiy the filenames above with full pathnames, of course) or install the apache2 utilities to have access to rotatelogs.

answered Sep 14 '09 at 17:03

Stefan Förster

1,151
5
8

This is the same as I could do with logrotate. And I want it better. – Sep 14 '09 at 19:47

Proper way to rotate Nginx logs

4 Answers4