How to configure AWS cloudWatchAgent to collect logs recursively from a folder on an EC2 instance?

Question

I have a logs directory on a EC2 instance and cloud watch agent running over there. In the CloudWatch agent configuration file I have given log file details as below

"logs": {
    "logs_collected": {
      "files": {
        "collect_list": [
          {
            "file_path": "/home/ec2-user/logs/**/*",
            "log_group_name": "test0",
            "log_stream_name": "{instance_id}"
          }
        ]
      }
    }
  }

but this is not collecting logs recursively. I want cloud agent to send all the logs present under one directory(having nested directories as well) to cloud watch logs. How is that possible?

Answer 1

The first ** wildcard is all you need to pick up all logs in all sub-directories:

"file_path": "/home/ec2-user/logs/**",

Alternatively you can pick up a specific log file in all sub-directories with:

"file_path": "/home/ec2-user/logs/**/mylogfile.log",

It looks like the additional * wildcard on the end of your file_path is causing the issue.