13

I am running a scala app on elastic beanstalk and the server gets overfilled with container logs which are not rotated. The default scheme of beanstalk rotates docker logs in the /var/log/eb-docker/containers/eb-current-app/* folder, but additional logs are accumulating in /var/lib/docker/containers/<container-id>/* and since they are not rotating the disk becomes full very quickly.

I've tried dealing with this problem by adding another logrotation claues to /etc/logrotate.elasticbeanstalk.hourly/logrotate.elasticbeanstalk.applogs.conf manually, and this solved the problem.

However, when I try to implement the change in every instance beanstalk launches using an .ebextension config file, the altered log rotation file does not appear on the server.

Here is my .config file - 

files:
  "/etc/logrotate.elasticbeanstalk.hourly/logrotate.elasticbeanstalk.applogs.conf":
    mode: "000644"
    content: |
      /var/log/eb-docker/containers/eb-current-app/* {
      size 10M
      rotate 5
      missingok
      compress
      notifempty
      copytruncate
      dateext
      dateformat %s
      olddir /var/log/eb-docker/containers/eb-current-app/rotated
      }

      /var/lib/docker/containers/*/*.log {
      size 10M
      rotate 5
      missingok
      compress
      notifempty
      copytruncate
      dateext
      dateformat %s
      olddir /var/log/eb-docker/containers/eb-current-app/rotated
      }

I've also tried to use the simplest form of config file I've found in this reddit post - https://www.reddit.com/r/aws/comments/2u3afj/elastic_beanstalk_issues_with_ebextensions/, but it also failed to have any effect on my launched instances.

Any advice as to why my configuration fails to affect the environment would be greatly appreciated.

Yaron Idan
  • 6,207
  • 5
  • 44
  • 66
  • Did you try and debug your logrotate config? http://superuser.com/a/255970/141 or http://serverfault.com/a/58403/783 – VonC May 10 '16 at 06:29
  • My logrotate config is working properly wheb i add to the server manually, the problem i have is with preconfiguring the instances using .ebextensions. – Yaron Idan May 10 '16 at 06:39
  • And could you try in your `.ebextensions` the path `/etc/logrotate.d/logrotate.elasticbeanstalk.applogs.conf` instead? As in https://sandro-keil.de/blog/2015/03/11/logrotate-for-docker-container/ – VonC May 10 '16 at 06:41
  • The problem is not within my logrotation configuration, when I set them manually they work just fine. The question is how to make the config fiiles in .ebextension work. You can look at the reddit link I've attached that even the simplest action didn't yield results. – Yaron Idan May 11 '16 at 16:32

1 Answers1

7

Sadly, the problem ended up being a typo. I created a folder called .ebextentsion. When I fixed it everything started to work nicely.

Yaron Idan
  • 6,207
  • 5
  • 44
  • 66