AWS Cloudwatch Logs timestamp parsing with milliseconds

Asked Sep 21 '17 at 04:31

Active May 30 '19 at 20:18

Viewed 7,628 times

I have a hive-server2 log that's formatting its output with timestamps like:

2017-09-19T18:04:39,717 ERROR [words.

But the datetime_format spec doesn't list any support for milliseconds. There is only zero padded microseconds %f -> 000000 ... 999999.

So I tried to specify most of the string as follows:

datetime_format = %Y-%m-$dT%H:%M:%S,

But I'm getting parse warnings for this in the /var/log/awslogs.log. It would appear that it's trying to use the format to match the whole first white-space separated field, or maybe using it to find where to start splitting on whitespace and then expecting the 717 part to be a log level.

Does anyone know which it is and also whether there's some way for me to write a regex equivalent of .*, \d* or \S* for after the comma?

edited May 30 '19 at 20:18

Samuel Bolduc

18,163
7
34
55

asked Sep 21 '17 at 04:31

dlamblin

43,965
20
101
140

I've also opened this with AWS support with similar silence. – dlamblin Sep 28 '17 at 07:43
I might have a similar problem. Using `datetime_format = '%Y-%m-%dT%H:%M:%S.%f%z'` and parsing the message `{"@timestamp":"2018-03-13T10:15:06.522+00:00", "message":"Hello World"}` I'll get an error `timestamp could not be parsed from message`. awslogs is version 1.1.4 – VolkerK Mar 13 '18 at 10:36
I would love an answer to this that I could accept. – dlamblin Aug 16 '18 at 06:44
Thanks 2018q3 to 2019q2 – dlamblin Jun 05 '19 at 10:20
https://forums.aws.amazon.com/thread.jspa?threadID=166865 – ab77 Sep 24 '19 at 16:21

AWS Cloudwatch Logs timestamp parsing with milliseconds

0 Answers0