1

I'm running a API to a database written in Python (Flask Restless) and served using apache2 and mod_wsgi as per the Flask Documentation.

The app is hosting on AWS EC2 instances inside an Auto-scaling group. We're currently hosting on m3.medium instances which have:

  • 1 vCPU
  • 3.75 Gb RAM

A regular problem that we have is memory errors. Apache uses up all available memory in the instances and causes [critical memory allocation] failure. This issue with apache is well documented.

The docs and other S.O. questions explain that I can prevent excess memory usage by find the average size of an apache process and then limiting MaxRequestWorkers (a.k.a MaxClients) to ensure that I can't start more workers than I have memory available however most docs don't show how to reduce the RAM that apache uses in the first place.

I'm using the following command to give the average process size:

sudo ps -ylC apache2 | awk '{x += $8;y += 1} END {print "Apache Memory Usage (MB): "x/1024; print "Average Process Size (MB): "x/((y-1)*1024)}'

Which is currently ~130Mb, is this an unreasonably large number

sudo free -m shows that I have ~3000Mb memory free so I can have 3000/130 = ~20 Workers before I use too much memory.

I can successfully limit the number of workers to prevent excess memory usage however when I do this the result is simply that I drop ~100 requests/min, returning 504 errors. This is unacceptable in my scenario.

Clearly I have 2 options, increase the amount of RAM in my server or reduce the amount of memory that each Apache Process consumes.

I have already removed the unnecessary modules from apache but am no longer sure what else I can do to reduce the memory footprint of each apache2 process. I want to run 50-100 Workers which would require 13GB RAM which seems excessive?

The WebApp is used to manage a system of about 30,000 internet connected products. Most requests are to log a connection to the system or return a json array of data to the user.

My Apache2.conf file is as follows:

Timeout 120
KeepAlive on
MaxKeepAliveRequests 100
KeepAliveTimeout 5
ThreadsPerChild 20
MaxRequestWorkers 100
ServerLimit 100
User ${APACHE_RUN_USER}
Group ${APACHE_RUN_GROUP}
HostnameLookups Off
LogLevel warn
IncludeOptional mods-enabled/*.load
IncludeOptional mods-enabled/*.conf
Include ports.conf
AccessFileName .htaccess
IncludeOptional conf-enabled/*.conf
IncludeOptional sites-enabled/*.conf

What steps can I take now, given that I don't think my App ought to require servers with 13+Gb of RAM and I don't want to reduce the number of MaxRequestWorkers because doing so causes the system to reject about 100 reqs/min

David McLean
  • 191
  • 1
  • 12
  • Could you please update if you were able to figure this out? – Chakradhar Vyza Jul 19 '21 at 06:28
  • My Flask-Restless app was using joined load as the default loading strategy which meant that every query tried to load much more data than required. At the time I thought the high memory usage might be something to do with Apache, it hindsight it should have been clear that threads need a large amount of memory was more likely to be a problem in the application than in the server! I'd recommend taking a close look at your application to see whether you're doing anything particular memory-inefficient – David McLean Aug 22 '21 at 18:58

0 Answers0