9

I use the AWS elasticsearch service. There is no much data is stored. I use m3.large.elasticsearch instance type. When I check the Maximum memory utilization (SysMemoryUtilization) metric, it is always about 90%. I guess this is the memory usage ratio of elasticsearch instance. Does this mean I need to enlarge my instance memory? Actually, I wondered whether this metric is accurate. Does anyone meet the same situation?

Previously, my elasticsearch instance type is t2.small, the SysMemoryUtilization is almost 90%. I changed the instance type to m3.large.elasticsearch but the SysMemoryUtilization still keeps being almost 90%.

see the SysMemoryUtilization: https://i.stack.imgur.com/PMt2Z.png

Xiaowu Feng
  • 91
  • 1
  • 3

3 Answers3

7

yes, it is accurate.

actually, JVM will get all memory needed from OS and use it when if elasticsearch needed. JVM memory is called HEAP and you can check it via this command:

curl -XGET [ES-IP-MASTER]:9200/_cat/nodes?v

you should check HEAP percentage usage and it is dangerous if heap 90% utilized.

hamid bayat
  • 2,029
  • 11
  • 20
4

We have the exact same scenario. I noticed that the metric SysMemoryUtilization is indeed reflecting what the cluster is also reporting. However JVMMemoryPressure utilisation is within range and performs with a sawtooth pattern.

My question though is whether the SysMemoryUtilization should be considered as a valid metric to set up an alert. Amazon does not even list it as one of the recommended alerts as mentioned here: https://docs.aws.amazon.com/en_pv/elasticsearch-service/latest/developerguide/cloudwatch-alarms.html

remedix
  • 492
  • 6
  • 13
3

It is not a valid metric for instance scaling. If you refer to the documentation it says that

"System memory utilization that is above 90% doesn't indicate any heap usage issues or an overloaded OpenSearch Service cluster. A system memory utilization above 90% is considered normal, especially for nodes running OpenSearch Service. Therefore, you don't need to scale up the size of your cluster."

Here is a shortlist of the metrics you want to monitor for scaling OpenSearch Domain clusters;

  • JVM memory pressure
  • CPU utilization
  • Free storage space

https://aws.amazon.com/premiumsupport/knowledge-center/opensearch-high-sysmemoryutilization/

And here is a link to a larger list of recommended metrics for monitoring for multiple purposes.

https://docs.aws.amazon.com/opensearch-service/latest/developerguide/cloudwatch-alarms.html

Draziray
  • 31
  • 2