1

I have about 12 httpd processes running for an RHEL aws box that no one is hitting in the browser (shared team dev box). These processes together are consuming over 1.8 GB on the dev box, and I have seen it up to 6 GB on production. Each one of those processes consumes about 800 MB as per ps aux. I did an strace on one of them and left it for a while to find:

<venv>/django/contrib/flatpages/templatetags/analytical  0x7fff37ef41a0) = -1 ENOENT (No such file or directory)
<venv>/django/contrib/flatpages/templatetags/analytical.py, O_RDONLY) = -1 ENOENT (No such file or directory)
<venv>/pagination/templatetags/expert_tags.py, O_RDONLY) = -1 ENOENT (No such file or directory)
<venv>/django/templatetags/raven.py, O_RDONLY) = -1 ENOENT (No such file or directory)
<vevn>/django_extensions/templatetags/raven.py O_RDONLY) = -1 ENOENT (No such file or directory)

There are; with out a joke, thousands of these ENOENT messages over a span of just 5 minutes. All types of different files.

An strace of 3 other processes shows something also interesting

read(4, 0x7fff37efa00f, 1)              = -1 EAGAIN (Resource temporarily unavailable)

Any way to find out which Resource is being referenced here?

I'm not an OS programming expert, but I presume this is not normal? Any idea how I can I find out what's causing this? How to prevent it?

Strangely this raven.py is of course truly a missing file, but we have a third party library in the virtual env called 

raven (3.5.1)
Sam Hammamy
  • 189
  • 5
  • 17
  • Somehow the strace output got corrupted. The `` part in the start of each line is wrong, something else was supposed to be there. It is relevant to know what was there. You should update the question with the correct output from the strace command. – kasperd Jan 24 '15 at 17:03
  • The `read(4, 0x7fff37efa00f, 1) = -1 EAGAIN (Resource temporarily unavailable)` part doesn't indicate any problem, if it only happens a few times. If it repeats endlessly with only the same output, then it certainly indicates a bug. You can look in `/proc/%d/fd/4` to find out what `4` is. Most likely it is a socket. – kasperd Jan 24 '15 at 17:06

1 Answers1

0

While still not truly sure why so many ENOENT, there was a miss-understanding of the apache config. I made an assumption that mod_wsgi was running as a daemon, when it was running in embedded mode. The apache worker.c section set the number of processes to 8 with an expansion of 25 and that's why there were so many stand-by processes.

Each process was reserving about 800MB of VM space, and about 120MB of RAM. Once mod_wsgi was changed to a daemon, those numbers came down to about 200MB for VM space, and 8MB for RAM! Overall memory consumption by apache went down from over 1 GB down to 64MB!

Sam Hammamy
  • 189
  • 5
  • 17