How do I show CloudWatch stats per server?

Question

I have some metrics that are being published to CloudWatch logs. (It would probably be better as CloudWatch metrics)

Basic Query:

fields @timestamp, jvm.threads.live, process.cpu.usage, server , system.load.average.1m , jvm.memory.usage.after.gc
| sort by @timestamp desc

Sample data json:

{
    "@timestamp": "2022-xx-xxT11:03:00.027Z",
    "jvm.gc.overhead": "0",
    "jvm.memory.usage.after.gc": "0.036",
    "jvm.threads.live": "47",
    "process.cpu.usage": "1",
    "server": "MyServer.localdomain",
    "system.load.average.1m": "2.351"
}

I can draw a graph using the below, but it does not take the server into account and it just averages all servers.

The server names are random, so I don't have a list of them upfront.

fields @timestamp, jvm.threads.live, process.cpu.usage, server , system.load.average.1m , jvm.memory.usage.after.gc
| stats avg(jvm.threads.live) as threads, avg(process.cpu.usage * 100) as cpuPercentage, avg(system.load.average.1m) as loadAvg, avg(jvm.memory.usage.after.gc * 100) as memoryPercentage  by bin(1m)

How do I show each server's stats and not just an average of all servers as below?

`by server, bin(1m)` gives a query result, but I am unable to run a visualisation on it. — rjdkolb, Sep 14 '22 at 09:07

jaisheel_mistry · Accepted Answer · 2022-09-14T09:13:42.450

1

this may work

fields @timestamp, jvm.threads.live, process.cpu.usage, server , system.load.average.1m , jvm.memory.usage.after.gc
| stats avg(jvm.threads.live) as threads, avg(process.cpu.usage * 100) as cpuPercentage, avg(system.load.average.1m) as loadAvg, avg(jvm.memory.usage.after.gc * 100) as memoryPercentage  by server, bin(1m)

edited Sep 14 '22 at 09:13

answered Sep 14 '22 at 08:55

jaisheel_mistry

28
3

1

sorry did not see your new comment, i would say you may have to transorm your logs to metrics using a metric-filter, you can reference all your values in the logs using ${.fielname} convention. You may need to create multiple metrics per field and set the dimension to the server name – jaisheel_mistry Sep 14 '22 at 09:22
thanks, how would do that? I have not worked with a metrics filter before – rjdkolb Sep 14 '22 at 10:58
on aws console , go to ctoudwatch-> log groups -> select your log group -> actions -> create metric filter check https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/MonitoringLogData.html and https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/FilterAndPatternSyntax.html – jaisheel_mistry Sep 14 '22 at 11:29
The answer actually visualises as a bar chart, but not a line chart – rjdkolb Sep 14 '22 at 13:07

How do I show CloudWatch stats per server?

1 Answers1