php process LD_LIBRARY_PATH big cpu usage

Question

I'm finding the causes of PHP process using so much CPU , server is Apache mpm worker + suphp below is some output of 'strace php', as you can see, the process is looking for dso in 'incorrect' path, these are just compiled PHP extensions, is there a way to specify the search path to be '/lib64:/usr/lib64' ? cuz i found it eventually loaded the dso either in lib64 or /usr/lib64 after many negative path tries, which is system CPU consuming, you know, so much system calls in vain. Thanks in advance.

open("/opt/xslt/lib/libselinux.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/opt/tidy/lib/libselinux.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/opt/mhash/lib/libselinux.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/opt/libmcrypt/lib/libselinux.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/opt/pcre/lib/libselinux.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/opt/curlssl/lib/libselinux.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/lib64/mysql/libselinux.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/opt/xml2/lib/libselinux.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/opt/pcre/lib64/libselinux.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/opt/curlssl//lib64/libselinux.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/opt/php_with_imap_client//lib64/libselinux.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/opt/libmcrypt//lib64/libselinux.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/opt/mhash//lib64/libselinux.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/opt/tidy//lib64/libselinux.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/opt/xslt//lib64/libselinux.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/lib64/tls/libselinux.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/lib64/libselinux.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/lib64/libselinux.so.1", O_RDONLY) = 3

open("/opt/xslt/lib/libsepol.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/opt/tidy/lib/libsepol.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/opt/mhash/lib/libsepol.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/opt/libmcrypt/lib/libsepol.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/opt/pcre/lib/libsepol.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/opt/curlssl/lib/libsepol.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/lib64/mysql/libsepol.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/opt/xml2/lib/libsepol.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/opt/pcre/lib64/libsepol.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/opt/curlssl//lib64/libsepol.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/opt/php_with_imap_client//lib64/libsepol.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/opt/libmcrypt//lib64/libsepol.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/opt/mhash//lib64/libsepol.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/opt/tidy//lib64/libsepol.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/opt/xslt//lib64/libsepol.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/lib64/tls/libsepol.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/lib64/libsepol.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/lib64/libsepol.so.1", O_RDONLY)  = 3
open("/opt/xslt/lib/libnss_files.so.2", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/opt/tidy/lib/libnss_files.so.2", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/opt/mhash/lib/libnss_files.so.2", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/opt/libmcrypt/lib/libnss_files.so.2", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/opt/pcre/lib/libnss_files.so.2", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/opt/curlssl/lib/libnss_files.so.2", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/lib64/mysql/libnss_files.so.2", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/opt/xml2/lib/libnss_files.so.2", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/opt/pcre/lib64/libnss_files.so.2", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/opt/curlssl//lib64/libnss_files.so.2", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/opt/php_with_imap_client//lib64/libnss_files.so.2", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/opt/libmcrypt//lib64/libnss_files.so.2", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/opt/mhash//lib64/libnss_files.so.2", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/opt/tidy//lib64/libnss_files.so.2", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/opt/xslt//lib64/libnss_files.so.2", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/lib64/tls/libnss_files.so.2", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/lib64/libnss_files.so.2", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/lib64/libnss_files.so.2", O_RDONLY) = 3

Is my opinion correct? Or is the below causing the big CPU usage?

lstat("/home", {st_mode=S_IFDIR|0711, st_size=4096, ...}) = 0
lstat("/home/socialth", {st_mode=S_IFDIR|0711, st_size=4096, ...}) = 0
lstat("/home/socialth/public_html", {st_mode=S_IFDIR|0750, st_size=4096, ...}) = 0
lstat("/home/socialth/public_html/components", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat("/home/socialth/public_html/components/com_content", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat("/home/socialth/public_html/components/com_content/router.php", {st_mode=S_IFREG|0644, st_size=8157, ...}) = 0
stat("/home/socialth/public_html/components/com_content/router.php", {st_mode=S_IFREG|0644, st_size=8157, ...}) = 0
stat("/home/socialth/public_html/index.php", {st_mode=S_IFREG|0644, st_size=1971, ...}) = 0
access("/home/socialth/public_html/components/com_content/router.php", F_OK) = 0
lstat("/home", {st_mode=S_IFDIR|0711, st_size=4096, ...}) = 0
lstat("/home/socialth", {st_mode=S_IFDIR|0711, st_size=4096, ...}) = 0
lstat("/home/socialth/public_html", {st_mode=S_IFDIR|0750, st_size=4096, ...}) = 0
lstat("/home/socialth/public_html/components", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat("/home/socialth/public_html/components/com_content", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat("/home/socialth/public_html/components/com_content/router.php", {st_mode=S_IFREG|0644, st_size=8157, ...}) = 0
lstat("/home", {st_mode=S_IFDIR|0711, st_size=4096, ...}) = 0
lstat("/home/socialth", {st_mode=S_IFDIR|0711, st_size=4096, ...}) = 0
lstat("/home/socialth/public_html", {st_mode=S_IFDIR|0750, st_size=4096, ...}) = 0
lstat("/home/socialth/public_html/components", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat("/home/socialth/public_html/components/com_content", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat("/home/socialth/public_html/components/com_content/router.php", {st_mode=S_IFREG|0644, st_size=8157, ...}) = 0
stat("/home/socialth/public_html/components/com_content/router.php", {st_mode=S_IFREG|0644, st_size=8157, ...}) = 0
stat("/home/socialth/public_html/index.php", {st_mode=S_IFREG|0644, st_size=1971, ...}) = 0
access("/home/socialth/public_html/components/com_content/router.php", F_OK) = 0
lstat("/home", {st_mode=S_IFDIR|0711, st_size=4096, ...}) = 0
lstat("/home/socialth", {st_mode=S_IFDIR|0711, st_size=4096, ...}) = 0
lstat("/home/socialth/public_html", {st_mode=S_IFDIR|0750, st_size=4096, ...}) = 0
lstat("/home/socialth/public_html/components", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat("/home/socialth/public_html/components/com_content", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat("/home/socialth/public_html/components/com_content/router.php", {st_mode=S_IFREG|0644, st_size=8157, ...}) = 0
lstat("/home", {st_mode=S_IFDIR|0711, st_size=4096, ...}) = 0
lstat("/home/socialth", {st_mode=S_IFDIR|0711, st_size=4096, ...}) = 0
lstat("/home/socialth/public_html", {st_mode=S_IFDIR|0750, st_size=4096, ...}) = 0
lstat("/home/socialth/public_html/components", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat("/home/socialth/public_html/components/com_content", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat("/home/socialth/public_html/components/com_content/router.php", {st_mode=S_IFREG|0644, st_size=8157, ...}) = 0
stat("/home/socialth/public_html/components/com_content/router.php", {st_mode=S_IFREG|0644, st_size=8157, ...}) = 0
stat("/home/socialth/public_html/index.php", {st_mode=S_IFREG|0644, st_size=1971, ...}) = 0
access("/home/socialth/public_html/components/com_content/router.php", F_OK) = 0
lstat("/home", {st_mode=S_IFDIR|0711, st_size=4096, ...}) = 0
lstat("/home/socialth", {st_mode=S_IFDIR|0711, st_size=4096, ...}) = 0
lstat("/home/socialth/public_html", {st_mode=S_IFDIR|0750, st_size=4096, ...}) = 0
lstat("/home/socialth/public_html/components", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat("/home/socialth/public_html/components/com_content", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat("/home/socialth/public_html/components/com_content/router.php", {st_mode=S_IFREG|0644, st_size=8157, ...}) = 0
lstat("/home", {st_mode=S_IFDIR|0711, st_size=4096, ...}) = 0
lstat("/home/socialth", {st_mode=S_IFDIR|0711, st_size=4096, ...}) = 0
lstat("/home/socialth/public_html", {st_mode=S_IFDIR|0750, st_size=4096, ...}) = 0
lstat("/home/socialth/public_html/components", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat("/home/socialth/public_html/components/com_content", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat("/home/socialth/public_html/components/com_content/router.php", {st_mode=S_IFREG|0644, st_size=8157, ...}) = 0
stat("/home/socialth/public_html/components/com_content/router.php", {st_mode=S_IFREG|0644, st_size=8157, ...}) = 0
stat("/home/socialth/public_html/index.php", {st_mode=S_IFREG|0644, st_size=1971, ...}) = 0
access("/home/socialth/public_html/components/com_content/router.php", F_OK) = 0
lstat("/home", {st_mode=S_IFDIR|0711, st_size=4096, ...}) = 0
lstat("/home/socialth", {st_mode=S_IFDIR|0711, st_size=4096, ...}) = 0
lstat("/home/socialth/public_html", {st_mode=S_IFDIR|0750, st_size=4096, ...}) = 0
lstat("/home/socialth/public_html/components", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat("/home/socialth/public_html/components/com_content", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat("/home/socialth/public_html/components/com_content/router.php", {st_mode=S_IFREG|0644, st_size=8157, ...}) = 0
lstat("/home", {st_mode=S_IFDIR|0711, st_size=4096, ...}) = 0
lstat("/home/socialth", {st_mode=S_IFDIR|0711, st_size=4096, ...}) = 0
lstat("/home/socialth/public_html", {st_mode=S_IFDIR|0750, st_size=4096, ...}) = 0
lstat("/home/socialth/public_html/components", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat("/home/socialth/public_html/components/com_content", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat("/home/socialth/public_html/components/com_content/router.php", {st_mode=S_IFREG|0644, st_size=8157, ...}) = 0
stat("/home/socialth/public_html/components/com_content/router.php", {st_mode=S_IFREG|0644, st_size=8157, ...}) = 0    
stat("/home/socialth/public_html/index.php", {st_mode=S_IFREG|0644, st_size=1971, ...}) = 0
access("/home/socialth/public_html/components/com_content/router.php", F_OK) = 0
lstat("/home", {st_mode=S_IFDIR|0711, st_size=4096, ...}) = 0
lstat("/home/socialth", {st_mode=S_IFDIR|0711, st_size=4096, ...}) = 0
lstat("/home/socialth/public_html", {st_mode=S_IFDIR|0750, st_size=4096, ...}) = 0
lstat("/home/socialth/public_html/components", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat("/home/socialth/public_html/components/com_content", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat("/home/socialth/public_html/components/com_content/router.php", {st_mode=S_IFREG|0644, st_size=8157, ...}) = 0
lstat("/home", {st_mode=S_IFDIR|0711, st_size=4096, ...}) = 0
lstat("/home/socialth", {st_mode=S_IFDIR|0711, st_size=4096, ...}) = 0
lstat("/home/socialth/public_html", {st_mode=S_IFDIR|0750, st_size=4096, ...}) = 0
lstat("/home/socialth/public_html/components", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat("/home/socialth/public_html/components/com_content", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat("/home/socialth/public_html/components/com_content/router.php", {st_mode=S_IFREG|0644, st_size=8157, ...}) = 0
stat("/home/socialth/public_html/components/com_content/router.php", {st_mode=S_IFREG|0644, st_size=8157, ...}) = 0
stat("/home/socialth/public_html/index.php", {st_mode=S_IFREG|0644, st_size=1971, ...}) = 0
access("/home/socialth/public_html/components/com_content/router.php", F_OK) = 0
lstat("/home", {st_mode=S_IFDIR|0711, st_size=4096, ...}) = 0
lstat("/home/socialth", {st_mode=S_IFDIR|0711, st_size=4096, ...}) = 0
lstat("/home/socialth/public_html", {st_mode=S_IFDIR|0750, st_size=4096, ...}) = 0
lstat("/home/socialth/public_html/components", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat("/home/socialth/public_html/components/com_content", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat("/home/socialth/public_html/components/com_content/router.php", {st_mode=S_IFREG|0644, st_size=8157, ...}) = 0

What are you trying to do here? All I see is a wall of system calls. — Chris Laplante, Feb 27 '13 at 16:02
i was doubting the php process costs much cpu cuz of these calls — user2115959, Mar 01 '13 at 15:00

score 0 · Answer 1 · answered Feb 27 '13 at 15:54

Not much you can do about it, especially since all those packages have installed themselves off in their own little directory tree. while /opt is a fine concept for isolating entire packages from each other, it does mean that any such package which offers shared libs to the rest of the system MUST be put into the library search path, and you end up with long pointless open/stat calls as ANY shared lib is searched for. The only optimizations you could do are perhaps rearranging the path so that less frequently used libraries are at the tail end of the search path, so that hits come "sooner" and don't waste time checking areas that are rarely used.

Ditto for your PHP autoloader. The long stat chains leading down to any particular file is also necessary, as the system checks for access rights at each stage.

Incidentally, this is the exact reason why using .htaccess files is discouraged. As soon as .htaccess is enabled, Apache has to check EVERY SINGLE directory leading up to a requested file for the presence of a .htaccess file. If the file's present , it must be loaded/parsed/etc... All this wastes a ludicrous amount of cpu cycles and disk bandwidth.

.htaccess is fine while developing/testing, but once you've got a setup that works, the .htaccess directives SHOULD be transferred to the main server .conf files, so they're only loaded/parsed ONCE at server startup.

php process LD_LIBRARY_PATH big cpu usage

1 Answers1