Parsing logs in Cloudwatch insight

Question

I am trying to parse the following log using cloudwatch insights.

2021.10.25 19:56:20:459 UTC | Info       | HTTP | GOOGLE_cf95a06b-b5fa-4f70-bc18-28fc30dfd9cc

    Tx [http] 200.61.132.110:80 -> 200.61.132.89:31812 StatusCode=200

Insight query:

fields @timestamp, @message
| filter @message like "StatusCode="
| parse @message "* | * | * | *" as aa, bb, cc, dd
| display dd

I get this:

GOOGLE_cf95a66b-b5fa-4f70-bc10-28fc30fdd9cc Tx [http] 200.61.132.110:80 -> 200.61.132.89:31812 StatusCode=200

How can I parse individual items example "GOOGLE_cf95a66b-b5fa-4f70-bc10-28fc30fdd9cc" and "StatusCode=200"

Any pointers are much appreciated ?

score 6 · Accepted Answer · answered Dec 02 '21 at 18:59

It depends on the structure of your log events. If there is a fixed structured and some fields that does not change (specially the blank spaces as fields separator), it can help to use them in the query. For example, according to the log example that you have showed, you can use this query:

fields @timestamp, @message
| filter @message like "StatusCode="
| parse @message "* | * | * | * * [*] * -> * StatusCode=*" as f1, f2, f3, field_google, f4, protocol, ip1, ip2, status_code
| display field_google, protocol, ip1, ip2, status_code

Parsing logs in Cloudwatch insight

1 Answers1