0

I have an issue with the logs on Cloud Watch. I keep having the logs on different entries instead of one single entry per log. Here's an example.

Output in the console:

[2020-12-21T11:51:21.966] [ERROR] default - [handlerProcess] [
  ValidationError {
    target: OnBroadcastStart {
      broadcast_id: 'bro_...',
      session_id: null,
      timestamp: '2020-12-21T11:51:21.920Z'
    },
    value: null,
    property: 'session_id',
    children: [],
    constraints: {
      isNotEmpty: 'session_id should not be empty',
      isString: 'session_id must be a string'
    }
  }
]

This is the output from a single error log.

Output in cloud watch:

[2020-12-21T11:51:21.966] [ERROR] default - [handlerProcess] [
[2020-12-21T11:51:21.966]   ValidationError {
[2020-12-21T11:51:21.966]     target: OnBroadcastStart {
[2020-12-21T11:51:21.966]       broadcast_id: 'bro_...',
[2020-12-21T11:51:21.966]       session_id: null,
[2020-12-21T11:51:21.966]       timestamp: '2020-12-21T11:51:21.920Z'
[2020-12-21T11:51:21.966]     },
[2020-12-21T11:51:21.966]     value: null,
[2020-12-21T11:51:21.966]     property: 'session_id',
[2020-12-21T11:51:21.966]     children: [],
[2020-12-21T11:51:21.966]     constraints: {
[2020-12-21T11:51:21.966]       isNotEmpty: 'session_id should not be empty',
[2020-12-21T11:51:21.966]       isString: 'session_id must be a string'
[2020-12-21T11:51:21.966]     }
[2020-12-21T11:51:21.966]   }
[2020-12-21T11:51:21.966] ]

Every single line is an entry here.

Is there any way to fix this? As logger for the project I'm using Log4js.

Mido
  • 89
  • 1
  • 10

1 Answers1

2

So if you are using the Cloudwatch Agent, you need to tune the Timestamp_format setting. If you don't specify a format, CloudWatch will ping each and every line of the log with it's epoch-specific timestamp and date.

To tune the agent settings, you can find information here: https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-Agent-Configuration-File-Details.html

Eric E
  • 199
  • 6
  • What if I didn't set up an agent? For example, those logs are set up automatically from an ECS container using Fargate. I log on console from my app and it logs onto cloud watch. I didn't configure anything. – Mido Dec 22 '20 at 09:50
  • Okay, so this is cloudwatch picking up your Log4js logs and streaming them to CW (as I understand). – Eric E Dec 22 '20 at 20:04
  • Alright - what you have is Log4JS not outputting logs in a format recognized by Cloudwatch - so it is appending each line with a received by DTG. check out the log4js cloudwatch appender (https://www.npmjs.com/package/log4js-cloudwatch-appender) for your project (I'm guessing it's nodeJS) If, instead you are talking about log4j (for Apache), please let me know. – Eric E Dec 22 '20 at 20:11
  • Thanks a lot Eric! I'll definitely check it out! – Mido Dec 22 '20 at 21:58
  • @Mido - did this end up resolving your problem? If not, did you find another solution? – mojave Oct 19 '21 at 21:21
  • 1
    @mojave I ended up outputting on cloudwatch as a string. The result is that the logs will parse the string into object when selecting the "view as text" checkbox on the console. The reason why I ended up doing this is also because this way you can use the find feature, and it's going to find the whole section, instead of just the word separate. I hope this helps – Mido Oct 20 '21 at 08:11