4

I am trying to push some old logs (-10 days) using boto3 put_log_events to cloud watch. if I set the current time in milliseconds for timestamp field, I am able to run cloudwatch queries in log streams. if it is set actual log DateTime, cloud watch responds with

"No data found for this time range"

Sample Log:

{"Namespace": "AWS/ECS", "Metric": "CPUUtilization", "Dimensions": {"Average": 0.08905141301220283, "Sum": 0.08905141301220283, "Unit": "Percent"}, "Timestamp": "2021-01-16T22:19:00+00:00"}

s = datetime.datetime(2021, 1, 15, 3, 17, tzinfo=tzutc()
time_data=int(round(s.timestamp() * 1000)) ## cloudwatch search does not work if this is set.

time_data = int(round(time.time() * 1000)) ## cloudwatch search works if this timestamp is set.

        msg=[ { "timestamp": time_data, 
                "message": message}
            ]

Cludwatch query:

filter @logStream = 'daily-2021-01-27'
 | fields @timestamp, @message
 | filter Namespace = 'AWS/ECS'

Form AWS DOC:

-- timestamp -> (long)
The time the event occurred, expressed as the number of milliseconds after Jan 1, 1970 00:00:00 UTC.

-- None of the log events in the batch can be older than 14 days or older than the retention period of the log group.

Not sure what I am missing.

(Data that is timestamped 24 hours or more in the past may take in excess of 48 hours to become available from submission time using GetMetricStatistics.

) is this related to this behaviour?

Ragav
  • 942
  • 4
  • 19
  • 37
  • Using nodejs sdk putLogEvents with logs from the previous day is "putting" the logs into the log stream, but they are not queryable in cloudwatch insights. I have not been able to find any documentation or tickets filed around this, but it is definitely a problem. Bump – Vontei Jul 27 '21 at 15:17

0 Answers0