0

I am running an Apache web server on AWS EC2 (c4.xlarge). My site's peak hour is during 8pm - 10pm. For some reason, during the peak hour, the system load just goes off to the roof, while CPU and Memory usage is kinda OK. Also, the number of Apache workers are only hitting 40 (Configured max clients is 80).

The instance is running on AWS EBS (Magnetic drive), but I don't see much IO from the screenshot below.

Any chance the system load is caused by the network bandwidth hitting the limit?

Last 24 Hours New Relic Server Monitoring

Attached is the Apache (HTTPD) process instances count, CPU and memory usage from NewRelic:

Apache (HTTPD) process instances count

CloudWatch for the EBS attached to the Instance (our local time is UTC+10 hours):

CloudWatch for the EBS attached to the Instance

This is a screenshot of htop while high load: Htop screenshot while high load

starchx
  • 533
  • 10
  • 24
  • Network I/O limit for c4.xlarge is 750Mb/s, you are using about 45Mb/s at peak. 40 concurrent page renders is likely too much if each is rather CPU intensive. – jordanm Jul 10 '17 at 06:12
  • Since system load is very high, I thought Apache would spawn more and more child processes (max 80 as configured) and push CPU to 100%. However, that's not the case in here. – starchx Jul 10 '17 at 06:33
  • 2
    Load average is a calculation of processes currently "running" (waiting for or using CPU time). A high load average without maxing CPU usage just means that it's a concurrency issue, not throughput. Increase your number of cores or reduce your number of concurrent running processes. – jordanm Jul 10 '17 at 07:18
  • 2
    You can use CloudWatch to look at a lot of useful statistics around [EBS](http://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/ebs-metricscollected.html). VolumeQueueLength and BurstBalance are the key ones to look at. Any reason you're using magnetic over SSD? – Tim Jul 10 '17 at 07:48
  • Just edit the question to add EBS cloudwatch and it doesn't seem to be EBS related. It was old instance and never get chance to migrate to SSD EBS. – starchx Jul 10 '17 at 23:38
  • @jordanm The MaxClients value is calculated based on system available memory and average apache process memory. I have just added htop screenshot while load is high. Any thoughts? – starchx Jul 11 '17 at 11:31
  • Yes, you have far too many apache workers in "D" and "R" states. – jordanm Jul 11 '17 at 14:33

0 Answers0