I'm renting a virtual CentOS based server where I have about 5 active sites running. Several months ago I noticed a significant performance degradation in how my sites are running.
First symptom was too many (~300) apache processes running after that HTTP server stops responding. The problem disappeared after reboot but appeared again sporadically in a day or in a week and so on. I played with httpd.config options but couldn't get rid of that problem and the only solution I could found is to schedule server reboot every 1 hour. The reboot was quite fast so temporary it solved the problem.
However a couple months ago I started observing another problem with server performance - periodically sites just respond too slowly.
I need some advice or help to figure out the source of problem as I'm not sure I understand it for the moment:
It looks like it's enough memory, here is the output from top: Mem: 524288k total, 299252k used, 225036k free, 0k buffers
It's enough disk space (from "vm -h"):
Filesystem Size Used Avail Use% Mounted on
vzfs 40G 28G 13G 70% /
none 3.9G 4.0K 3.9G 1% /dev
And here is runtime information about resource usage from "vmstat -S M 2 100":
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy id wa st
0 0 0 220 0 0 0 0 8 111 0 1 6 4 90 0 0
0 0 0 220 0 0 0 0 0 0 0 272 0 0 100 0 0
0 0 0 220 0 0 0 0 0 0 0 176 0 0 100 0 0
0 0 0 220 0 0 0 0 0 0 0 132 0 0 100 0 0
0 0 0 220 0 0 0 0 0 1950 0 330 0 0 100 0 0
2 0 0 220 0 0 0 0 0 4 0 1854 21 19 60 0 0
2 0 0 210 0 0 0 0 0 4 0 1958 73 23 4 0 0
2 0 0 199 0 0 0 0 0 4 0 829 41 40 19 0 0
1 0 0 207 0 0 0 0 0 26 0 1212 47 17 36 0 0
1 0 0 206 0 0 0 0 0 4 0 1405 50 4 45 0 0
0 0 0 209 0 0 0 0 4 140 0 1251 34 5 60 0 0
0 0 0 208 0 0 0 0 0 0 0 213 1 0 99 0 0
0 0 0 208 0 0 0 0 0 0 0 353 0 0 100 0 0
0 0 0 208 0 0 0 0 0 6 0 317 0 0 100 0 0
0 0 0 206 0 0 0 0 0 0 0 299 0 0 100 0 0
0 0 0 214 0 0 0 0 12 114 0 336 0 1 99 0 0
By the way, the lines with high CS (context switching values) correspond to moments when I refreshed a couple site pages.
Any advice is very appreciated.