Elastic Beanstalk Docker with Amazon CloudWatch

Question

I have an Elastic Beanstalk application deployed with a Docker container. The application itself is a Java Application.

My goal is to get the logs to Cloudwatch. In particular I would like to get the stdouterr.log file to Cloudwatch. The file can be found under /var/log/eb-docker/containers/eb-current-app/*

I followed the official AWS documentation here. Based on the example configuration files I managed to get the nginx Webrequest to Cloudwatch.

For the EB docker stdouterr log I adapted the cwl-log-setup.config file to the following:

Mappings:
  CWLogs:
    ApplicationLogGroup:
      LogFile: "/var/log/eb-docker/containers/eb-current-app/*"
      TimestampFormat: "%d/%b/%Y:%H:%M:%S %z"

Outputs:
  ApplicationLogGroup:
    Description: "The name of the Cloudwatch Logs Log Group created for this environments web server access logs. You can specify this by setting the value for the environment variable: WebRequestCWLogGroup. Please note: if you update this value, then you will need to go and clear out the old cloudwatch logs group and delete it through Cloudwatch Logs."
    Value: { "Ref" : "AWSEBCloudWatchLogs8832c8d3f1a54c238a40e36f31ef55a0WebRequestLogGroup"}


Resources :
  AWSEBCloudWatchLogs8832c8d3f1a54c238a40e36f31ef55a0WebRequestLogGroup:    ## Must have prefix:  AWSEBCloudWatchLogs8832c8d3f1a54c238a40e36f31ef55a0
    Type: "AWS::Logs::LogGroup"
    DependsOn: AWSEBBeanstalkMetadata
    DeletionPolicy: Retain     ## this is required
    Properties:
      LogGroupName:
        "Fn::GetOptionSetting":
          Namespace: "aws:elasticbeanstalk:application:environment"
          OptionName: ApplicationLogGroup
          DefaultValue: {"Fn::Join":["-", [{ "Ref":"AWSEBEnvironmentName" }, "webrequests"]]}
      RetentionInDays: 14

The cloudwatch log group is created but no logs arrive. What steps am I missing or what is wrong in my configuration file?

Bacon · Answer 1 · 2016-12-08T09:27:32.493

5

I've just encountered the same issue - I managed to get the log files by changing the LogFile configuration to

Mappings:
  CWLogs:
    WebRequestLogGroup:
      LogFile: "/var/log/eb-docker/containers/eb-current-app/*.log"
      TimestampFormat: "%d/%b/%Y:%H:%M:%S %z"

Note this only works if there is a single log file, if you re-deploy a container or apply a configuration change which results in multiple logs in this directory then only the events from the log file with the most current modified time will be processed by the awslogs agent

Also by default the agent will compare the first line of the log file to determine if it is a different file, if the first line is the same it will ignore it. You can specify the lines the agent uses to fingerprint the file by adding file_fingerprint_lines configuration,

for example to use lines 1 - 20 to identify the file:

AWSEBAutoScalingGroup:
    Metadata:
      "AWS::CloudFormation::Init":
        CWLogsAgentConfigSetup:
          files:
            ## any .conf file put into /tmp/cwlogs/conf.d will be added to the cwlogs config (see cwl-agent.config)
            "/tmp/cwlogs/conf.d/apache-access.conf":
              content : |
            [    apache-access_log]
                file = `{"Fn::FindInMap":["CWLogs", "WebRequestLogGroup", "LogFile"]}`
                log_group_name = `{ "Ref" : "AWSEBCloudWatchLogs8832c8d3f1a54c238a40e36f31ef55a0WebRequestLogGroup" }`
                file_fingerprint_lines = 1-20
                log_stream_name = {instance_id}
                datetime_format = `{"Fn::FindInMap":["CWLogs", "WebRequestLogGroup", "TimestampFormat"]}`
              mode  : "000400"
              owner : root
              group : root

edited Dec 08 '16 at 09:27

answered May 31 '16 at 08:46

Bacon

1,229
2
14
26

Do you happen know if there's a workaround for this? – Ian Sep 22 '16 at 14:47
I've since found that this is incorrect, the awslogs agent which pushes the log file events to CloudWatch will use the file with the most current last modified time, I'll edit the answer – Bacon Oct 20 '16 at 14:27
Do you mean that logging will now continue after a deployment? – Ian Nov 15 '16 at 19:14
1

Yes, it will continue – Bacon Nov 15 '16 at 22:00
2

I'm finding it's not the case for me. I get logs from my initial deployment, but not from subsequent updates. Any advice? – andrew.w.lane Dec 06 '16 at 21:06
I've found the agent uses the first line of the log file to determine if it is a new log file and to send it to cloudwatch. The first line of all of my logs was a log4j error so it was ignoring them. You can specify the number of lines the agent uses to fingerprint the file – Bacon Dec 08 '16 at 09:13
CWLogsAgentConfigSetup: files: "/tmp/cwlogs/conf.d/apache-access.conf": content : | [apache-access_log] file = `{"Fn::FindInMap":["CWLogs", "WebRequestLogGroup", "LogFile"]}` log_group_name = `{ "Ref" : "AWSEBCloudWatchLogs8832c8d3f1a54c238a40e36f31ef55a0WebRequestLogGroup" }` file_fingerprint_lines = 1-20 log_stream_name = {instance_id} datetime_format = `{"Fn::FindInMap":["CWLogs", – Bacon Dec 08 '16 at 09:15
the **file_fingerprint_lines = 1-20** specifies to use lines 1 to 20 to identify the file – Bacon Dec 08 '16 at 09:18
See http://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/AgentReference.html for more – Bacon Dec 08 '16 at 09:18
Are you sure that's all you did to get it working? I did what you did with no luck. – nzhenry May 13 '17 at 21:41

score 3 · Answer 2 · answered Jun 05 '18 at 22:31

3

I create Dockerrun file for each ElasticBeanstalk application. https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/create_deploy_docker_image.html

There you can specify a logging configuration:

           "logConfiguration": {
                "logDriver": "awslogs",
                "options": {
                    "awslogs-group": "${you-log-group-name}",
                    "awslogs-region": "${region}",
                    "awslogs-stream-prefix": "${serviceName}"
                }
            }

This sets up Docker's logging driver configuration more details: https://docs.docker.com/config/containers/logging/configure/

answered Jun 05 '18 at 22:31

Nune Isabekyan

531
2
4

1

This configuration is only available for multiple docker container at the moment – Baboo Jun 26 '19 at 08:50
This is now possible in Amazon Linux 2's docker platform using docker-compose.yaml: ``` version: '3.8' services: your-service: logging: driver: awslogs options: awslogs-group: "${your-log-group-name}" awslogs-region: "us-west-2" ``` – Mustafa Alammar Sep 07 '21 at 17:57

score 0 · Answer 3 · answered Sep 07 '21 at 17:58

This is now possible in Amazon Linux 2's docker platform using docker-compose.yaml:

version: '3.8'
services:
  your-service:
    logging:
      driver: awslogs
      options:
        awslogs-group: "${your-log-group-name}"
        awslogs-region: "us-west-2"

See here for more options: https://docs.docker.com/config/containers/logging/awslogs/

Elastic Beanstalk Docker with Amazon CloudWatch

3 Answers3