How do I rotate my Rails 5 logs?

Question

I'm running Rails 5 on Ubuntu 14.04. Is there a way to rotate my logs without relying on the Linux logrotate system? I have this set up ...

myuser@myapp:~$ cat /etc/logrotate.d/myapp
/home/rails/myapp/log/*.log {
  daily
  missingok
  rotate 2
  compress
  delaycompress
  notifempty
  copytruncate
}

but yet my logs never rotate. Behold how bloated they are ...

myuser@myapp:~$ ls -al /home/rails/myapp/log/
total 3958356
drwxr-xr-x  2 rails rails       4096 Jul  3 22:31 .
drwxr-xr-x 15 rails rails       4096 Sep 21 17:21 ..
-rw-rw-r--  1 rails rails          0 Jun 22 10:22 development.log
-rw-rw-r--  1 rails rails      14960 Jun  1 22:39 development.log.1
-rw-rw-r--  1 rails rails          0 Oct 22  2016 .keep
-rw-r--r--  1 rails rails  198362787 Oct 31 16:28 production.log
-rw-r--r--  1 rails rails    8615654 Jul  3 22:31 production.log.1
-rw-r--r--  1 rails rails  640621243 Jun 29 13:16 production.log.2.gz
-rw-rw-r--  1 rails rails 2856792698 Oct 31 17:12 sidekiq.log
-rw-rw-r--  1 rails rails  348853619 Jul  3 22:31 sidekiq.log.1
-rw-rw-r--  1 rails rails          0 Jul  3 22:31 test.log
-rw-rw-r--  1 rails rails      54246 Jul  3 22:31 test.log.1

Is there another way to get the logs rotated or is there a way to fix the configuration I have included?

Edit: Here's the cron script that's set

myuser@myapp:~$ cat /etc/cron.daily/logrotate
#!/bin/sh

# Clean non existent log file entries from status file
cd /var/lib/logrotate
test -e status || touch status
head -1 status > status.clean
sed 's/"//g' status | while read logfile date
do
    [ -e "$logfile" ] && echo "\"$logfile\" $date"
done >> status.clean
mv status.clean status

test -x /usr/sbin/logrotate || exit 0
/usr/sbin/logrotate /etc/logrotate.conf

Edit: Per the comment I tried adding this to my config/environment/production.rb file ...

config.logger = ActiveSupport::Logger.new(config.paths['log'].first, 1, 50 * 1024 * 1024)

but the logs get ever bigger without being rotated.

Answer 1

By following the 12-factor methodologies (Treat logs as event streams) you could "should" delegate this task to the supervisor.

For example, by using immortal it will do all the rotation process for you not depending on the operating system.

A basic configuration file (run.yml) may look like:

cmd: bundle exec unicorn -c unicorn.rb
cwd: /arena/app-1
env:
    DEBUG: 1
    ENVIRONMENT: production
log:
    file: /var/log/app-1.log
    age: 86400 # seconds
    num: 7     # int
    size: 1    # MegaBytes
    timestamp: true # will add timesamp to log

In case you would like to split the logs stderr & stdout this could be used:

cmd: bundle exec unicorn -c unicorn.rb
cwd: /arena/app-1
env:
    DEBUG: 1
    ENVIRONMENT: production
log:
  file: /var/log/app.log
  age: 86400 # seconds
  num: 7     # int
  size: 1    # MegaBytes
stderr:
  file: /var/log/app-error.log
  age: 86400 # seconds
  num: 7     # int
  size: 1    # MegaBytes
  timestamp: true # will add timesamp to log

As a side note, from the 12-factor site:

Twelve-factor app processes should never daemonize or write PID files. Instead, rely on the operating system’s process manager to manage output streams, respond to crashed processes, and handle user-initiated restarts and shutdowns.

[Disclaimer: I'm the author of immortal], and one of the ideas behind was indeed to cover the automation of applications at scale without need to worry about filling up the disk when logs where not properly rotated besides being available to deploy (start/restart) in the easiest possible way by just modifying a run.yml file without requiring root privileges.

Answer 2

You can use ruby Logger class and shift_age method to select frequency of rotation: daily, weekly or monthly.

shift_age: Number of old log files to keep, or frequency of rotation (daily, weekly or monthly). Default value is 0, which disables log file rotation.

https://ruby-doc.org/stdlib-2.4.0/libdoc/logger/rdoc/Logger.html#method-c-new

To use it in Rails you can insert a line like this in your config/application.rb:

config.logger = ActiveSupport::Logger.new("log/#{Rails.env}.log", shift_age = 'daily')

Note: today's log will not have any date, only tomorrow (if daily rotation) the old log will be 'rotated' and date appended to filename.

Answer 3

Recommending using STDOUT for app logging is a terrible idea. The Twelve-factor app is highly focused for container-based deployment when UNIX daemons are not welcomed, you should not buy this just as-is.

Standard output has no structure, it's just a plain-text interface. While it makes much sense for UNIX multi-process communication, it's just awful for logging. The only proper logging interface has always been either syslog or system journal. Our Rails app takes advantage of both and you can configure it either with STDOUT (for dev environments), syslog or journald for structured logging.

This way you can embed things like request, session or correlation id along with all messages. Logs can be sent over the wire to logging servers easily. Log file rotation is no problem at all if you use this approach. And finally, you can integrate with central logging solutions like ElasticSearch (or ELK stack) for further data analysis.

Indeed it's a little bit of work and unfortunately Rails developers are very opinionated about this - they use super-simple and not flexible Ruby logger, they also send input parameters to INFO level log and refuse to make it a configurable option (some APIs have huge inputs in our app which floods log). But it is what it is - you can override this in your app and give Rails a proper logging stack. Our solution:

• https://github.com/TwP/logging
• https://github.com/lzap/logging-journald (plugin for logging gem)