php-fpm using 100% CPU and nginx gives 404

Question

I am playing around with Magento CE on Nginx and its working fine.

Problem starts when I do a 500 user load test. The php-fpm process hogs up all the CPUs to 100% and I start getting 404 as the response.

Here's my nginx config

fastcgi_cache_path /var/run/nginx-cache levels=1:2 keys_zone=magento:100m inactive=60m;
fastcgi_cache_key "$scheme$request_method$host$request_uri";
fastcgi_cache_use_stale error timeout invalid_header http_500;
fastcgi_ignore_headers Cache-Control Expires Set-Cookie;

server {

...
location ~ \.php$ {
    try_files $uri =404;
    expires off;

    fastcgi_cache magento;
    fastcgi_cache_valid 200 60m;
    fastcgi_cache_methods GET HEAD;
    add_header X-Fastcgi-Cache $upstream_cache_status;
    fastcgi_cache_bypass $no_cache;
    fastcgi_no_cache $no_cache;

    fastcgi_read_timeout 900s;
    fastcgi_index index.php;
    fastcgi_pass unix:/var/run/php5-fpm.sock;
    fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
    include /etc/nginx/fastcgi_params;
}

I am a bit new to nginx and fastcgi so not really sure what configs I should be focusing on.

I have enabled zend opcache so I believe it php-fpm should not be under stress. Is this assumption correct?

A friend of mine says its expected behaviour and it would go 100% on a 2 core machine. But I find it hard to believe. Nginx is performing blazing fast if I do a static file load test. Its just PHP which is slowing it down.

Thought?

You are not demonstrating "the problem". Start from the basis that if you can't show a server-side error message there is missing information - _read your log files_. How are you performing a 500 user load test, and when it triggers an error - exactly what error (response headers and body) do you get? You also need to show your php-fpm config (e.g. the contents of `/etc/php5/fpm/pool.d/www.conf` - without the comments) but a 500 user test (do you mean 500 concurrent users?) can quite easily saturate any low powered machine running a typical OTS php project. — AD7six, Jan 23 '15 at 09:04
You're pasting the nginx config, yet you state that php-fpm is using all the system resources. Where do you think you need to do the tuning? — pauska, Jan 23 '15 at 09:21
you're trying to squeeze 500 simultaneous users on 2 cores? and you wonder what's wrong? this is some kind of joke? — ADM, Jan 23 '15 at 11:07
@ADM Apologies for wrong expectation. But I have seen better performance with core PHP or even CI for that matter. This is the first time I am exploring Magento and its performance. The only problem I had was Nginx and MySql are performing great and php is consuming all the CPU. This made me wonder if I can do something to improve php performance. But after AD7six's explanation, its pretty clear to me what I should expect. — Салман, Jan 23 '15 at 12:09

score 3 · Accepted Answer · answered Jan 23 '15 at 09:56

3

Magento is a heavy piece of software. What makes you think your server would NOT scream in agony during your 500 users performance test? Of course serving static content is fast, but parsing all that black magic PHP that Magento contains... well, it's a heavy duty job for your server.

The PHP op-code cache does not mean that your PHP would be as fast as static content. Without some kind of front-end cache (such as Varnish or Squid) it still has to do lots of stuff, including some heavy SQL queries, so the overall performance will be anything from dozens of page loads per second to perhaps 100 page loads per sec, not more. (Unless Magento has somehow optimized its code A LOT since I last tried it)

answered Jan 23 '15 at 09:56

Janne Pikkarainen

31,852
4
58
81

All true. The `fastcgi_cache` settings suggest the expectation for only the first request to actually hit php (if urls are not unique per user) and thereafter it'd be treated the same as a static file. Receiving a 404 suggests either poor debugging (it's actually a 5xx, and there's nothing more really to be said) OR it really is a 404 - which would be a bit weird IMO. and warrants some further investigation (at least for the sake of curiosity). – AD7six Jan 23 '15 at 10:09
@AD7six Sorry my bad, initially I didn't have `fastcgi_cache` and was expecting opcache to work super fast. But After @Janne's explanation, I realized that I kept wrong expectations. Anyway, after implementing `fastcgi_cache` the performance is of course blazing (40-70ms). But naturally, the user is served with old copy of the page. – Салман Jan 23 '15 at 10:28
I just need one advice, what kind of performance should I expect from php opcache? To exaggerate, I was under impression that opcache means it would behave as good as compiled, ;) :P – Салман Jan 23 '15 at 10:31
@Салман "better". Op cache (which you should pretty much always have enabled btw) only stops php needing to parse php files - it does absolutely nothing to speed up e.g. the time spent establishing a connection to the db, db queries, disk i/o, time spent by the application performing calculations etc. etc. If a page loads in 2s without opcache, turning it on might bring that down to 1.95s - it significantly reduces the time parsing php files - which typically make up very little of "sizable" php scripts. – AD7six Jan 23 '15 at 10:35
@AD7six Thanks a lot. You cleared my concept related to this. So basically, apart from `opcache`, smartly configured `fastcgi_cache`, proper `nginx` configs and optimum mysql setup, there's not much you can do to improve the performance of a heavily database driven and dynamic/e-commerce application like Magento. Increasing hardware is the only solution it seams. – Салман Jan 23 '15 at 10:43

php-fpm using 100% CPU and nginx gives 404

1 Answers1