3

I'm looking for ways to optimize our current web server hosted in-house. I'm trying to supply as much relevant information below. Please let me know if you would require additional information in order to assist.

Server is running 1 single website, which is an online pizza ordering platform built on Zend Framework (ver1). On traffic stats from the last month aprox 6.000 pageloads per day, concentrated mainly around dinnertime. Around 1500 loads/hour peaks in that period.

We recently upgraded from a 2/2mbit aDSL-line to 100/100mbit fiber, and we still have performance issues at dinner time. We assumed the 2mbit was the issue. Website is pretty snappy in low-load periods.

Hardware

CPU: Intel(R) Xeon(R) CPU            5160  @ 3.00GHz (3000.13-MHz K8-class CPU)

Mem: 328M Active, 4427M Inact, 891M Wired, 244M Cache, 623M Buf, 33M Free
Swap: 16G Total, 468K Used, 16G Free
(6GB physical, 16GB swap)

Filesystem   Type     Size    Used   Avail Capacity  Mounted on
/dev/ad7s1a  ufs      4.8G    768M    3.7G    17%    /
devfs        devfs    1.0K    1.0K      0B   100%    /dev
/dev/ad7s1g  ufs      176G    5.2G    157G     3%    /home
/dev/ad7s1e  ufs      4.8G    2.8M    4.5G     0%    /tmp
/dev/ad7s1f  ufs       19G    3.5G     14G    19%    /usr
/dev/ad7s1d  ufs      4.8G    550M    3.9G    12%    /var

Server OS

FreeBSD 8.2-RELEASE

Software

apache-2.2.17
php5-5.3.8
mysql-server-5.5

Apache footprint (example, taken from # top)

31140 www              1  45    0   377M 41588K lockf   2   0:00  0.00% httpd
31122 www              1  44    0   375M 35416K lockf   2   0:00  0.00% httpd
31109 www              1  44    0   375M 38188K lockf   2   0:00  0.00% httpd
31113 www              1  44    0   375M 35188K lockf   2   0:00  0.00% httpd

Apache is using the prefork MPM, APC (Alternative PHP Cache). SSL module is loaded, but not utilized (as in don't really work, thus not used). There is a file containing settings for MPM modules, but as i see it's not included in the httpd.conf file, the include line is commented out. Thus i would guess that the prefork MPM is working of default values too.

Here are some other Apache conf values that i found - which are included in https.conf

Timeout 300
KeepAlive On
MaxKeepAliveRequests 100
KeepAliveTimeout 5

UseCanonicalName Off
HostnameLookups Off

Load when experiencing performance issues

enter image description here

Note: API requests to backend

As for our online platform, it communicates with our backend system through an API (GET/POST). For price-quotes on baskets (cached until new item added), products (cached), prices on products (cached). And the backend-server is actually showing real load when seeing performance issues on the web apear. So i'm more striding to that it's the API requests that is taking forever to complete, thus holding up clients at the web server. So my question is, is the stats given above (traffic/load), and the current config of the software. Compared to the specs of the internetconnection and the hardware, is that signs of a stressed server? And would i be able to increase max number of clients on the web server, based on the available resources and the actual footprint of Apache?

Phliplip
  • 541
  • 8
  • 22
  • 1
    Have you monitored the server load and ram usage (using 'top' command) during both heavily loaded times and normal times? is the database (i assume there is one) hosted on the same server? – user16081-JoeT Oct 15 '12 at 22:57
  • 3
    what performance issues are you experiencing? – Alastair McCormack Oct 15 '12 at 23:09
  • What firewall are you using, if any? – suitablyawesome Oct 15 '12 at 23:19
  • @Toquonce What do you think a firewall would be doing to cause this?? – Chris S Oct 16 '12 at 03:23
  • @Chris S : I don't know what performance issues the user is experiencing, but I know pf has default limits that could potentially be reached under very heavy load, such as the state table being exhausted, which would cause connections to drop. Since FreeBSD was mentioned, pf could be used and possibly interfering. Unlikely, perhaps, but nothing should discounted. – suitablyawesome Oct 16 '12 at 03:48
  • @user16081: I have added load on the web server during performance issues. The database is a MySQL hosted on the same server. – Phliplip Oct 16 '12 at 08:17
  • @Fuzzyfelt : Really slow pageloads, people calling in to our callcenter complaining of on the general speed of the website. Haven't seen it myself or heard of any of the "too many connections" error messages though. Maybe in sync with my theory about our backend API i mention in the note in the post. – Phliplip Oct 16 '12 at 08:20
  • @Toquonce : We're using a hosted firewall out of our MPLS network, it's a Fortigate. On the web server the ipfw service is running firewall. – Phliplip Oct 16 '12 at 08:25
  • @ChrisS : I understand your edit, and agree that FreeBSD is NOT linux ;) But for the case of optimizing Apache i think that would justify the Linux tag, as it would be the same on both OS. And there is a-heap-a-lot more people following the linux tag :) – Phliplip Oct 16 '12 at 08:31
  • based on the load you posted, your server is chilling. the cpu is not even 1/2 utilized and your memory is not swapping. I'd say the problem is something external to your server hardware. Especially if you're not able to recreate the problem. Could be a peering issue with your isp, I'd run some traceroutes to the customers having issues. – user16081-JoeT Oct 17 '12 at 03:15
  • update: just saw your note about the api, yes it sounds like that's the bottleneck – user16081-JoeT Oct 17 '12 at 03:26
  • @user16081 : On the backendserver, this morning I have added extra ram to total of 12GB, and added an extra CPU so it now has 2 physical 3Ghz Xeon Processors. Now the bottleneck and the current performance issues set aside, is there any recommendations for the webserver? Is the footprint ok? or a bit on the high side? I have been stripping down modules both on Apache and PHP. Would you recommend the worker MPM compared to the preform MPM? – Phliplip Oct 17 '12 at 10:30
  • @Phliplip The tags are supposed to classify the question, not attract experts in other systems. I understand your thought process, but Apache runs on Solaris, Windows, and other systems too. The problem is almost certainly in the code's design and throwing more hardware at it will not fix the fundamental problems. You'll have to profile the code and figure out what specific bottlenecks there are. All the technology components you've mentioned are capable of handling very high load, there's fundamental problems in the way you're using them... ie. code problems. – Chris S Oct 20 '12 at 13:56
  • @Toquonce *When you hear hoof beats, think horses not zebras*. PF's limits are by default higher than Apache's defaults, so you'd see problems in Apache first. Also, the load he's talking about is several orders of magnitude less than those limits. – Chris S Oct 20 '12 at 13:58

4 Answers4

1

Web site performance is a big subject. From what I see the problem is most likely not Apache.

There are a few things that can help you to improve your performance though.

  • Caching! I do not know how much you cache in your PHP code. But as you're not using SSL and I guess you have a lot of read-only data, try Varnish

  • Enable mod_expires

  • Profile your website/backend API with JMeter

Olav Grønås Gjerde
  • 392
  • 2
  • 8
0

No, the above are not signs of a stressed server.

While you might be experiencing network I/O saturation (since you provide no actual data on it), this is extremely unlikely with so few actual requests occuring during peak times.

You need to run some tests on the HTTP backend to see if that is running at max capacity. If it is, then you need to fix THAT.

For future reference, you should configure and use the server-status handler to monitor apache, as documented here: http://httpd.apache.org/docs/2.2/mod/mod_status.html

adaptr
  • 16,576
  • 23
  • 34
0

Apache performance can be tweaked with Worker MPM and Prefork settings, altering the keepalive and max spare threads could help in increase of performance and consumption of less memory footprint, You can find the optimal setting for Prefork and worker from here

Sharma
  • 1
-2

You may be running out of processes (defined by MaxSpareServers and StartServers). The default is only 10, so if your response times are high, that may not be enough for one request every 2 seconds (1500 / hour).

Turn on the server-status module to monitor the processes in use. Make sure you set access rules so that this is only accessible internally.

AngerClown
  • 320
  • 1
  • 3