Amazon CloudWatch Insights Query

Question

I have logs like this:

I, [2020-06-17T09:32:48.100103 #9]  INFO -- : [54b35e04-9c19-443d-adff-b2c3192b5590] Completed 500 Internal Server Error in 7ms (ActiveRecord: 2.3ms | Allocations: 1705)

I, [2020-06-17T10:37:27.169909 #9]  INFO -- : [c800e9ce-fba3-4e1a-a19f-526f32746925] Completed 401 Unauthorized in 0ms (ActiveRecord: 0.0ms | Allocations: 115)

So as you can see in message it is always fallowing the pattern: Completed [THE ERROR CODE] [ERROR MESSAGE] ...

I'm using this query to retrieve logs with certain error codes:

fields @timestamp, @message
| filter @message like /401/
| sort @timestamp desc
| limit 20

But how can I parse the message to get separate fields for error code and message?

Answer 1

You can use parse function with the regex syntax: https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_QuerySyntax.html

Like this for example:

parse @message /Completed (?<errorCode>\d+) (?<errorMessage>.+) in (?<timeMilis>\d+)ms /
| filter isPresent(errorCode)

Result will be something like this

-------------------------------------------------
| errorCode |     errorMessage      | timeMilis |
|-----------|-----------------------|-----------|
| 500       | Internal Server Error | 7         |
| 401       | Unauthorized          | 0         |
-------------------------------------------------

This will just filter and extract the fields, you can do further processing from there.