3

After moving our web servers from Debian to Ubuntu we have noticed a large increase in the amount of memory (RSS) used by uWSGI master and worker processes. The version of uWSGI (1.9.13) remains the same.

I've used pmap to examine the memory usage for each process. Under Debian a master process looked like this:

Address             RSS     Dirty   Mode    Mapping 
----------------    ------  ------  ------          
kB                  15976   11316           
0000000001a880000   7308    7308    rw---   [anon
400000              1608    0       r-x--   uwsgi   
00007ff16c5a70000   772     772     rw---   [anon
00007ff176c3a0000   668     0       r-x--   libcrypto.so.0.9.8  
00007ff175f010000   548     0       r-x--   libc-2.11.3.so  
00007ff177fc70000   512     512     rw---   [anon
00007ff17807b0000   512     512     rw---   [anon
00007ff1759e50000   356     0       r-x--   libstdc++.so.6.0.13 
80                  268     184     rw---   uwsgi

I'm only showing the most relevant memory blocks and the numbers under RSS are in kilobytes.

While now under Ubuntu uWSGI has allocated itself much more memory. 

Address             RSS     Dirty   Mode    Mapping     
----------------    ------  ------  ------              
kB                  24216   19196               
2401000             15508   15508   rw---   [   anon    ]
00007f3094e86000    1648    0       r-x--   libpython2.7.so.1.0     
00007f308c6ce000    772     772     rw---   [   anon    ]
00007f3094abe000    680     0       r-x--   libc-2.17.so        
00007f309595d000    612     0       r-x--   libcrypto.so.1.0.0      
00007f3096de4000    512     512     rw---   [   anon    ]
00007f3096e67000    512     512     rw---   [   anon    ]
400000              480     0       r-x--   uwsgi       
00007f30945b3000    352     0       r-x--   libstdc++.so.6.0.17     
00007f309532e000    340     248     rw---   libpython2.7.so.1.0

The configuration and type of application loaded into these master procesess is otherwise identical. Kernel versions are 2.6.32-5-xen-amd64 for Debian and 3.8.0-19-generic for Ubuntu.

Does anyone have any idea as to why each master would be allocating itself another 8megs of memory?

aychedee
  • 153
  • 1
  • 7
  • uwsgi usually links with libpython but on the top it doesn't. That seems highly suspicious. Are you sure you aren't running PyPy or that there are indeed otherwise identical. – user Jul 12 '13 at 17:11
  • I generated this output with `pmap -x `. `pmap -x | grep libpython` gives output on Ubuntu but nothing on Debian. They are definitely both running Python2.7. And there are no other differences that I can discern... but maybe this is a clue? – aychedee Jul 12 '13 at 17:23
  • My thinking is that it is something to do with the big leap in kernel versions between 2.6.32 and 3.8.0. Something major might have change d in regards to memory management. – aychedee Jul 12 '13 at 17:28
  • @aychedee Kernel changes could very well be the difference. Try it on Wheezy which has the 3.x kernel to see if there's a difference. – Nathan C Jul 12 '13 at 18:03

1 Answers1

1

Discovered the problem, the system version of Python 2.7 that Ubuntu comes with causes uWSGI processes to use more memory. Compiling a new version of Python and using that to launch the uWSGI emperor solved the problem.

Compiling Python and installing uWSGI can be achieved with the below commands on Ubuntu 13.04.

wget http://python.org/ftp/python/2.7.5/Python-2.7.5.tar.bz2
tar jxf Python-2.7.5.tar.bz2
cd Python-2.7.5/
./configure --prefix=/opt/python2.7.5 && make && make install
cd /opt/python2.7.5/
curl -O http://python-distribute.org/distribute_setup.py
bin/python2.7 distribute_setup.py
bin/easy_install-2.7 pip
bin/pip-2.7 install uwsgi
aychedee
  • 153
  • 1
  • 7