2

I'm trying to set up autoscaling group of instances on GCE. Autoscaling policy setup to add instance(s) based on percent of used memory.

On every instance I installed Stackdriver Monitoring Agent (collectd) like this: 

$ curl -O https://repo.stackdriver.com/stack-install.sh
$ sudo bash stack-install.sh --write-gcm

I can read metrics reported by the agent into Stackdriver using monitoring API:

GET https://monitoring.googleapis.com/v3/projects/test-cluster/timeSeries?filter=metric.type+%3D+%22agent.googleapis.com%2Fmemory%2Fpercent_used%22+AND+resource.label.instance_id+%3D+%224770937493855508384%22&interval.endTime=2017-01-10T01%3A10%3A00Z&interval.startTime=2017-01-10T01%3A05%3A00Z&key={API_KEY}

Instance group manager setup to autoscale using agent.googleapis.com/memory/percent_used metric

However, Instance Group manager reports:

"There was no data received for the custom metric that is configured for autoscaling"

Problem: clearly agent.googleapis.com/memory/percent_used metric reported to the Stackdriver but is not returned to the instance group manager for some reason.

vtrv101
  • 21
  • 2
  • To create an autoscaler that uses Stackdriver Monitoring metrics, you must provide the desired target utilization level, the custom metric name, and the utilization target type. You can visit [Enable autoscaling using monitoring metrics](https://cloud.google.com/compute/docs/autoscaler/scaling-stackdriver-monitoring-metrics#create) for more details. – Faizan Jan 25 '17 at 00:31
  • @vtrv101 I have similar problem. I want to use nginx current connections for autoscale. Did you figure out a way to do this – Rafał Malinowski Feb 24 '17 at 09:15
  • @RafałMalinowski nop, I finały decided to go kubernetes route – vtrv101 Mar 11 '17 at 19:36
  • I have created custom metric based on nginx stats. – Rafał Malinowski Mar 15 '17 at 07:45
  • Were you able to workaround this issue? If so can you provide the details? The community will certainly benefit from your findings. – Carlos Mar 28 '17 at 18:44

2 Answers2

1

Autoscaler used to supports v2 metrics and 'agent.googleapis.com/memory/percent_used metric' is a v3 metric. It looks like this metric can be used now.

On this link you can get the metrics that are available on Stackdriver Monitoring v2.

On this other link, you can find all the v3 metrics, where you will find the 'percent_used' metric.

Marilu
  • 296
  • 1
  • 7
0

I'm adding a new answer to this page to include the step by step instruction in order to autoscale based on percent memory used which has been made simpler with the evolution of the platform.

The OP may have received the message they did at the time of the post as a result of autoscaling not supporting the required metric at the time.

Memory can now be monitored by installing the Stackdriver agent on a machine, as opposed to the previous requirement for a custom metric needing to be created. It's therefore much easier to monitor memory now, as well as autoscale based on memory usage (which is now supported). It should be noted however that Premium tier Stackdriver account is required to use the monitoring agent.

You need a managed instance group to autoscale, and the machines within that managed instance group need the Stackdriver agent installed on them to monitor memory.

There are a couple of ways you can do this. You could add a start-up script to the instance template that the managed instance group uses and add the commands required to install the monitoring agent (see here), or you could create an image that already has the monitoring agent installed and use this image as the source image for the instances in the group (please see supported operating system for the agent here).

Here are some steps you could follow to install the monitoring agent via a startup script, and then autoscale based on percent memory used.

1) Design an instance template. Compute Engine > instance templates> Create Instance Template.

2) In the 'Startup script' section of the create instance template page add the following:

#!/bin/bash
curl -sSO https://dl.google.com/cloudagents/install-monitoring-agent.sh
sudo bash install-monitoring-agent.sh

3) Create a managed instance group: Compute Engine > Instance Groups > Create instance group

4) Ensure you select the following options (in addition to your others personalised options) for your instance group in the relevant fields:

'Group type' = 'Managed instance groups'

'Instance Template' = Select the instance template you created in step 1/2.

'Autoscale based on' = 'Stackdriver monitoring metric'

'Metric identifier' = 'agent.googleapis.com/memory/percent_used'

'Target' = The percent of memory you would like to initiate the autoscaler, for example 60

You now have a managed instance group set to autoscale based on percent memory used.

neilH
  • 977
  • 1
  • 6
  • 16
  • You need to set percent_used to the target that you want, say 60% multiplied by the max number of instances, say 10. Then target should be: percent_used=600 – Thiago Melo Apr 29 '20 at 17:59