Questions tagged [aws-cloudwatch-log-insights]

301 questions
4
votes
0 answers

CloudWatch Logs Insights: converting strings to timestamp or datetime values

I have log events in CloudWatch whcih have certain fields like @timestamp and @message. @timestamp looks like this: 2022-08-09T08:49:51.971+02:00 and @message like this: 2022-08-09 06:49:49.376 INFO [xyz, 123abc456] xxxxxxx I can extract the time…
4
votes
1 answer

What query language is used by AWS Cloudwatch Logs Insights?

The AWS guides for insight queries aren't very specific but I wondered what query language it may use or at least, most resemble? This is the example they give when you navigate to Logs Insights: fields @timestamp, @message | sort @timestamp desc |…
Mike
  • 91
  • 6
4
votes
0 answers

Disable [INFO] log in cloudwatch to reduce IMR cost of AWS account

How do I filter out [INFO] log in cloudwatch in aws so that I only keep the [ERROR] logs to reduce IMR as it is occupying majority of our IMR cost? We are getting many request logs in an interval of 1 sec. How can I disable the INFO log from my log…
4
votes
0 answers

Aws cloud watch log insight vs Aws Athena

I have implemented some queries and dashboards where I used cloudwatch logInsight. I have used sample queries where I used parse, fields, aggregate functions etc. I am curious if we need complex queries like joins, windows functions etc, can we use…
4
votes
2 answers

Format the way I see AWS Cloudwatch timestamps in Log Insights

Is there a way to show the timestamps in cloudwatch in a friendlier format? If we take a super simple query: fields @timestamp, @message | sort @timestamp desc | limit 200 I get the standard @timestamp column to…
4
votes
1 answer

Is there any method in aws sdk for javascript to get the aws log insights query method results synchronously?

I am trying to get the log insights from aws using sdk for javascript v3, I can see that we can only schedule a query using StartQuery and later get results using getQueryResults methods respectively. I came across filterLogEvents method which…
4
votes
1 answer

Container Insights on Amazon EKS AccessDeniedException

I'm trying to add a Container Insight to my EKS cluster but running into a bit of an issue when deploying. According to my logs, I'm getting the following: [error] [output:cloudwatch_logs:cloudwatch_logs.2] CreateLogGroup API responded with…
4
votes
3 answers

How to search for multiple strings in logs using aws cloudwatch log insights query?

For my aws loggroups, I want to write a cloudwatch log insgights query to search for multiple strings in the logs. I tried something like this : fields @timestamp, @message, @logStream | filter @message like /(?i)\$\{jndi/ | filter @message like…
4
votes
0 answers

How to get stats from combined aggregated bin data in AWS Cloudwatch Logs Insights

I have some AWS CloudWatch logs which output values every 5 seconds. I'd like to get the max over a rolling 10 minute interval and then get the average value per day based on that. Using the CloudWatch Logs Insights QuerySyntax I cannot seem to get…
4
votes
0 answers

Cloud watch Log insights query, combining two querires

I currently use two different cloud watch log insights queries one to get total request count and the other to get total error count. Below are the queries: Total count: fields @timestamp, @message | filter @message like /reply.*MyAPI/ |parse…
4
votes
3 answers

How to enable container insights on already created ECS Fargate using AWS Console / GUI

How to enable container insights on already created ECS Fargate? I am running multiple ECS clusters for which container insight was not enabled while creating those. Now I have enabled that option in Account Settings. But it is for only the new…
4
votes
2 answers

CloudWatch Insights query: Format a DateTime string for grouping

I have json-formatted CloudWatch logs whose entries resemble this: { "message": "resource_liked", "context": { "date": { "date": "2021-05-07 16:52:11.000000", "timezone_type": 3, "timezone": "UTC" …
4
votes
0 answers

Cloudwatch query does not work if @timestamp is to previous date

I am trying to push some old logs (-10 days) using boto3 put_log_events to cloud watch. if I set the current time in milliseconds for timestamp field, I am able to run cloudwatch queries in log streams. if it is set actual log DateTime, cloud watch…
4
votes
0 answers

Escape Character for aws CloudWatch Log Query Insights

I'm working with an api that excecutes an aws Insights query. I check some business layers and then add filters to the query. Those filters are from a list of errors that I can't change. The trouble is that I cant make an standard function to add…
4
votes
0 answers

Cloudwatch Dashboard - Hide x-axis label from a bar graph

I am currently using cloudwatch insights to plot a bar graph on dashboard by querying cloudwatch logs. The log data (which now will become label) is a bit sensitive to be shown up on the dashboard, however while plotting the bar graph continues to…