1

I started adding alarms to CloudWatch (ECS/Fargate) but they all show "Insufficient data"

My first theories were that there was no CloudWatch agent running on my instances or that the container role or security groups doesn't allow access to CloudWatch but there are some conflicting observations:

a) Even metrics in the AWS/ApplicationELB namespace (+TargetResponseTime) show Insufficient data. I was under the assumption that the ELB published "automatically" to CloudWatch(?)

b) In the CloudWatch console under Insights -> Container Insights I can see "Avg CPU" and "Avg memory %" for the services and tasks which indicate that there is some agent running on the instances and they can publish to CloudWatch(?)

c) under Log groups there is a /aws/ecs/containerinsights//performance category with FargateTelemetry-nnnn and ServiceTelemetry- logs which would also lead to the same conclusions as in b)(?)

Is it possible that a configuration of the alarm parameters could lead to inconsistencies, resulting in the Insufficient data?

I created the alarm in CloudFormation

  ServerEndpointCPUAlarm:
    Type: AWS::CloudWatch::Alarm
    Properties:
      AlarmDescription: Server endpoint CPU high
      AlarmName: ServerEndpointCPUHigh
      AlarmActions:
        - !Ref AlertTopic
      ComparisonOperator: GreaterThanOrEqualToThreshold
      Namespace: AWS/ECS
      MetricName: CPUUtilization
      Statistic: Maximum
      DatapointsToAlarm: 3
      EvaluationPeriods: 5
      Period: 60
      Threshold: 80
      Unit: Percent
      Dimensions:
        - Name: Cluster
          Value: !Ref Cluster
        - Name: Service
          Value: !Ref ServerEndpointService

Thankful for pointers or clarifications,

  • Nik
Nicklas Karlsson
  • 317
  • 1
  • 4
  • 12
  • You have misconfigured the alarms. How are you creating the alarms exactly? Through the web console, or the CLI, or CloudFormation or Terraform or something? – Mark B Sep 27 '22 at 12:18
  • Added CloudFormation snippet – Nicklas Karlsson Sep 27 '22 at 19:37
  • Usually when I have a problem with an alarm I've created through CloudFormation or Terraform showing Insufficient Data, I go into the web UI and create the same alarm there, and then compare the two to see what I missed. – Mark B Sep 27 '22 at 19:39
  • Hmm, works in the web console. I'm pretty sure I've misunderstood something with the DataportsToAlarm/EvaluationPeriods/Period/Threshold combo. Back to the docs... – Nicklas Karlsson Sep 28 '22 at 05:46
  • 1
    Or more likely the Dimensions references – Nicklas Karlsson Sep 28 '22 at 06:04
  • The working alarm references the Target Group as "targetgroup//" but my !Ref MyTargetGroup apparently uses the ARN. Looking for some GetAtt that would be useable as reference – Nicklas Karlsson Sep 28 '22 at 06:45
  • 1
    I believe that "hexhash" is called the ARN Suffix. That's a value you can reference in Terraform, that is specifically for creating CloudWatch Metrics. It looks like in CloudFormation you need to use the `TargetGroupFullName` attribute of the Target group, which gives you the complete `"targetgroup//"` string. – Mark B Sep 28 '22 at 12:03
  • 1
    Yep, came to the same conclusion when comparing the UI-generated alarm with the CG one, many resources are referenced by arn but for some reason these have a more loose coupling. Dimensions: - Name: LoadBalancer Value: !GetAtt PublicLoadBalancer.LoadBalancerFullName - Name: TargetGroup Value: !GetAtt ServerEndpointTargetGroup.TargetGroupFullName works – Nicklas Karlsson Sep 28 '22 at 18:06

0 Answers0