30

I'm running a wordpress off of AWS, and I can't figure out how to monitor if the volume is running out of capacity. There are many options for monitoring other things, but I just want to know when we run low on space.

diogeneshamilton
  • 568
  • 1
  • 5
  • 12

4 Answers4

32

There are no built in metrics for disk space - Because Amazon is looking at this 'from the outside', they don't know what you've done with an EBS volume: it could be part of a raid set, formatted in some exotic format etc.

Amazon has a system called CloudWatch that can be used to monitor AWS systems and resources. Luckily, CloudWatch supports custom metrics: you can use the CloudWatch APIs to add any data you want (and then create alerts off them).

Amazon provides some sample scripts that will publish (among other things) disk space utilisation to Cloudwatch. Do be aware that there is a small charge associated with using custom metrics - $0.50 per metric per month.

Frederick Cheung
  • 83,189
  • 8
  • 152
  • 174
  • 2
    But you get 10 free ones a month. – Martin Sep 11 '12 at 08:58
  • Keep in mind that when you use these scripts you may want to provide `--aws-credential-file=/home/ubuntu/monitoring-scripts/awscreds.template` file filled with your AWS key and secret. That's something which could have been emphasized better in the documentation. – Dimitry K Dec 06 '14 at 15:19
  • 2
    @DimitryK, even better than providing a credentials file like you suggest, is to use an IAM Role. If you launch your instance associated with an Instance Profile which is associated with an IAM Role, those scripts (and the AWS CLI and the AWS SDKs and many other tools) are able to automatically obtain temporary credentials -- no need to deploy and manage (rotate, revoke, etc) credentials files manually anymore. – Bruno Reis Dec 29 '14 at 18:04
  • The sample script link is now being redirected to 'What Are Amazon CloudWatch, Amazon CloudWatch Events, and Amazon CloudWatch Logs?' – dezso Jun 14 '16 at 09:23
  • For details on how to setup metric on Windows Server 2016 see: http://stackoverflow.com/questions/37441225/how-to-monitor-free-disk-space-at-aws-ec2-with-cloud-watch-in-windows/43583258#43583258 – Simon Hutchison May 04 '17 at 05:17
  • Is it just me or is $0.50 for a single piece of text data absolutely and utterly absurd? This is basically the definition of rent seeking. I'd write a bash script to check `df` metrics and send an email or ping some endpoint when they get above a certain %, or use something like Prometheus. – Peter Kionga-Kamau Jan 30 '23 at 21:42
8

Selected answer will no longer work because:

The CloudWatch monitoring scripts are deprecated. We recommend that you use the CloudWatch agent to collect metrics and logs. - https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/mon-scripts.html

I found this guide helpful at setting up custom cloudwatch metric such as disk monitoring: https://marbot.io/blog/monitoring-ec2-disk-usage.html

The only thing that the above is missing is setting up IAM role and attaching it to the EC2, guide for this is here (do this first, then follow the blog post): https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/create-iam-roles-for-cloudwatch-agent.html

Per recommendation, summary steps below:

  1. Set up IAM role per link here: https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/create-iam-roles-for-cloudwatch-agent.html
  2. Attach it to EC2 you want to monitor
  3. ssh into your EC2
  4. cd /tmp
  5. wget https://s3.amazonaws.com/amazoncloudwatch-agent/amazon_linux/amd64/latest/amazon-cloudwatch-agent.rpm (to download the agent rpm)
  6. sudo rpm -U amazon-cloudwatch-agent.rpm (to install)
  7. sudo vi /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json (to edit config)
  8. paste the config (in the link, no changes necessary)
  9. sudo systemctl restart amazon-cloudwatch-agent (restart agent)
  10. tail -f /opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log (see if its working)
  11. Navigate to cloudwatch console and set up alarm based on custom metric that will show up as CWAgent (thats the name specified in the config, more detail in the link on exact way to set up alarm)
eMTy
  • 328
  • 1
  • 8
  • 20
2

I agree with Frederick Cheung's answer; however, I found this utility for Windows instances: http://www.eleven41.com/labs/cloudwatch-monitor-for-windows/. It runs as a service and will send the disk space and memory usage metrics to CloudWatch for you. From there, you'd just need to set up the alarm in CloudWatch.

Christopher Hinkle
  • 21
  • 2
1

I agree with Christopher Hinkle's agreement with Frederick Cheung's answer :)

Instead of using a utility where you still have to setup all the alerts yourself, try using Blue Matador. It will set up all the alerts for you, on all the mount points, on all the servers.

That said, you should also watch a lot more metrics than just the disk space. There's a guide called How to Monitor Amazon EBS with CloudWatch that goes over all the metrics and how you should approach monitoring them (if you don't use Blue Matador to automate them).

mbarlocker
  • 1,310
  • 10
  • 16