Elastic Beanstalk output custom logs to

Question

I have an Elastic Beanstalk app running a .Net app through IIS in a Windows environment. I want to get my custom logs to show up in AWS CloudWatch.

The app uses a Serilog logger in one of its static classes.
The logger outputs a message when I go to an endpoint path (ex. "/api/log-a-message").
Logs are written to a file "C:\LogsFolder\LogFile.log".

Following some online searching and reading through other questions and the AWS Documents. I ended up writing a .ebextensions/log_configuration.conf with the following content:

### BEGIN .ebextensions/CloudWatch.config
files:
  "C:/Program Files/Amazon/ElasticBeanstalk/config/taillogs.d/":
    content: |
      [ZeW logs]
      log_group_name = `{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "Serilog"]]}`
      log_stream_name = {instance_id}
      file = C:/LogsFolder/LogFile.log
### END .ebextensions/CloudWatch.config

But the logs are still not appearing in CloudWatch.

Did you setup permissions in IAM for Elastic Beanstalk to talk to Cloudwatch? I have the `CloudWatchAgentServerPolicy` and `AmazonEC2RoleforSSM` policies added to the aws-elasticbeanstalk-ec2-role, though it looks like the latter is deprecated in favor of `AmazonSSMManagedInstanceCore`. — littleforest, Oct 04 '19 at 23:53
My EC2 role has the following policy, which should allow for the needed privileges ... "logs:CreateLogGroup", "logs:CreateLogStream", "logs:GetLogEvents", "logs:PutLogEvents", "logs:DescribeLogGroups", "logs:DescribeLogStreams", "logs:PutRetentionPolicy" — ZeW, Oct 07 '19 at 08:01
I also set up two commands to run using the commands directive in that same file: `chkconfig awslogs on`, and then `service awslogs restart`. — littleforest, Oct 07 '19 at 23:24
Those are linux commands. The Windows version (I think) needs a conf file when restarting, but I don't know where my `.ebextensions/CloudWatch.config` file gets flung to, otherwise I could just use the `appendConf` option — ZeW, Oct 09 '19 at 11:27

ZeW · Accepted Answer · 2019-10-15T10:16:09.030

I've managed to almost do it. ... So it turns out that for AWS CloudWatch you need to append a configuration in JSON format.

Below is my .ebextensions/custom_logs.config file that Elastic Beanstalk uses for extensions. This just creates a custom_logs.json file for CloudWatch to use.

files:
  "C:/Users/Administrator/Desktop/custom_logs.json":
    content: |
      {
        "agent": {
          "metrics_collection_interval": 5
        },
        "logs": {
          "logs_collected": {
            "files": {
              "collect_list": [{
                "file_path": "C:\\MyCustomLogsFolder\\MyCustomLogFile.log",
                "log_group_name": "/aws/elasticbeanstalk/UsuallyThisIsTheEnvironmentName/MyCustomLogGroup-Log",
                "timezone": "UTC",
                "timestamp_format": "%Y-%m-%d %H:%M:%S",
                "multi_line_start_pattern": "{timestamp_format}"
              }]
            }
          }
        }
      }

After that file is created on my Desktop by Elastic Beanstalk, I can connect to the instance and run the following command (including the & at the start):

& C:\\'Program Files'\\Amazon\\AmazonCloudWatchAgent\\amazon-cloudwatch-agent-ctl.ps1 -a append-config -m ec2 -c file:C:\\Users\\Administrator\\Desktop\\cu
stom_logs.json -s

The only thing I need to figure out now is how to do this automatically once the instance starts.

Managed to do it with the following code (in addition to the files: statement above):

services:
  windows:
    AmazonCloudWatchAgent:
      enabled: 'true'
      ensureRunning: 'true'
      files:
        - "C:/MyCustomLogsFolder/MyCustomLogFile.log"
container_commands:
  01_cloudwatch_append:
    command: powershell.exe -ExecutionPolicy Bypass -Command "$cwa_ctl='C:\\Program Files\\Amazon\\AmazonCloudWatchAgent\\amazon-cloudwatch-agent-ctl.ps1'; $custom_logs_config='C:\\Users\\Administrator\\Desktop\\custom_logs.json'; & $cwa_ctl -a append-config -m ec2 -c file:$custom_logs_config -s;"
    ignoreErrors: true
    waitAfterCompletion: 10

This question helped a lot ... https://stackoverflow.com/questions/28035945/how-can-i-enable-cloudwatchlogs-in-an-elasticbeanstalk-environment-running-windo — ZeW, Oct 15 '19 at 14:58

Elastic Beanstalk output custom logs to

1 Answers1