Why is there a delay in script execution when calling many php scripts simultaneoulsy using multi_curl?

Question

I have a script that calls around 40 individual php scripts (located on the same server) simultaneously. I notice that there is a delay in the start of each individual php script of up to 5 seconds. The server load CPU and memory seem fine, no swapping, see top command down:

top - 22:32:36 up  1:06,  1 user,  load average: 0.18, 0.05, 0.01
Tasks: 181 total,   1 running, 179 sleeping,   0 stopped,   1 zombie
Cpu(s):  0.0%us,  0.5%sy,  0.0%ni, 99.5%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   1048576k total,  1024144k used,    24432k free,    20160k buffers
Swap:  2097144k total,        0k used,  2097144k free,   352520k cached

The last scripts in the loop that binds all the requests for the multi_curl generally have the largest delay. I measured the delay at the very start of each script.

What can I do to prevent this delay?

Below is my httpd.conf

#
# This is the main Apache HTTP server configuration file.  It contains the
# configuration directives that give the server its instructions.
# See <URL:http://httpd.apache.org/docs/2.2> for detailed information.
# In particular, see 
# <URL:http://httpd.apache.org/docs/2.2/mod/directives.html>
# for a discussion of each configuration directive.
#
# Do NOT simply read the instructions in here without understanding
# what they do.  They're here only as hints or reminders.  If you are unsure
# consult the online docs. You have been warned.  

ServerRoot "/etc/httpd"
Listen 80

#LoadModule dummy_module /usr/lib/apache/mod_dummy.so
#LoadModule php5_module        /usr/lib/apache/libphp5.so
Include /etc/httpd/conf/extra/httpd-phpmodules.conf

User apache
Group apache

KeepAlive On
KeepAliveTimeout 30

ServerAdmin admin@localhost
DocumentRoot "/var/www/html"

MaxClients 800
MaxRequestsPerChild 800

<Directory /home/*>
    AllowOverride All
    Options -MultiViews -Indexes FollowSymlinks IncludesNoExec +Includes
<Limit GET POST OPTIONS PROPFIND>
    Order allow,deny
    Allow from all
</Limit>
<LimitExcept GET POST OPTIONS PROPFIND>
    Order deny,allow
    Deny from all
</LimitExcept>
</Directory>

<Directory />
    Options All
    AllowOverride All
</Directory>

<Directory "/var/www/html">
    Options Indexes FollowSymLinks
    AllowOverride All
    Order allow,deny
    Allow from all
   <IfModule mod_suphp.c>
        suPHP_Engine On
        suPHP_UserGroup webapps webapps
    SetEnv PHP_INI_SCAN_DIR
   </IfModule>
</Directory>

<IfModule mod_dir.c>
    DirectoryIndex index.html index.htm index.shtml index.php index.php5 index.php4 index.php3 index.phtml index.cgi
</IfModule>

<FilesMatch "^\.ht">
    Order allow,deny
    Deny from all
    Satisfy All
</FilesMatch>

ErrorLog /var/log/httpd/error_log
LogLevel warn

#replace %b with %O for more accurate logging
<IfModule mod_logio.c>
    LogFormat "%h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined
    LogFormat "%h %l %u %t \"%r\" %>s %O" common
    LogFormat "%O %I" bytes
    LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %I %O" combinedio
</IfModule>

    CustomLog /var/log/httpd/access_log common

<IfModule mod_alias.c>
    # Include some DirectAdmin alias
    Include conf/extra/httpd-alias.conf
</IfModule>

<Directory "/var/www/cgi-bin">
    AllowOverride None
    Options None
    Order allow,deny
    Allow from all
</Directory>

DefaultType text/plain

<IfModule mod_mime.c>
    TypesConfig conf/mime.types
    AddType application/x-gzip .tgz
    AddEncoding x-compress .Z
    AddEncoding x-gzip .gz .tgz
    AddType application/x-compress .Z
    AddType application/x-gzip .gz .tgz
    AddHandler cgi-script .cgi
    AddHandler type-map var
    AddType text/html .shtml
    AddOutputFilter INCLUDES .shtml
    AddType video/x-ms-asf .avi
    AddType video/mpeg .mpg
    AddType video/mpeg .mpeg
    AddType video/quicktime .mov
    AddType video/x-ms-wmv .wmv
</IfModule>

#EnableMMAP off
#EnableSendfile off

#######################################################################################
# Do not change anything in included files, because they are rewritten by DirectAdmin #
#######################################################################################

# This is needed for PHP
Include conf/extra/httpd-php-handlers.conf

# Server-pool management (MPM specific)
Include conf/extra/httpd-mpm.conf

# Multi-language error messages
Include conf/extra/httpd-multilang-errordoc.conf

# Fancy directory listings
Include conf/extra/httpd-autoindex.conf

# Language settings
Include conf/extra/httpd-languages.conf

# User home directories
#Include conf/extra/httpd-userdir.conf

# Real-time info on requests and configuration
Include conf/extra/httpd-info.conf

# Virtual hosts
Include conf/extra/httpd-vhosts.conf

# Local access to the Apache HTTP Server Manual
#Include conf/extra/httpd-manual.conf

# Distributed authoring and versioning (WebDAV)
Include conf/extra/httpd-dav.conf

# Various default settings
Include conf/extra/httpd-default.conf

# Secure (SSL/TLS) connections
Include conf/extra/httpd-ssl.conf

# Deflate module settings
Include conf/extra/httpd-deflate.conf

# All the DirectAdmin vhosts
Include conf/extra/directadmin-vhosts.conf

# All suPHP directives
Include conf/extra/httpd-suphp.conf

# For user configurations not maintained by DirectAdmin. Empty by default.
Include conf/extra/httpd-includes.conf

#######################################################################################
# End of included files that are rewritten by DirectAdmin                             #
#######################################################################################

<IfModule mod_ssl.c>
SSLRandomSeed startup builtin
SSLRandomSeed connect builtin
</IfModule>

score 0 · Answer 1 · answered Jul 05 '12 at 21:01

0

The most common reason for this is, that something is blocking while a script was called.

This could be php's build in session-handler, that is blocking concurrent requests for the same session-id (keyword: race-conditions). If this is your problem and you need a solution for this, you can implement your own session-handler, which is handling session data in a way that fits you.

Another reason could be a flock'd file or stressed out system IO (slow disks for example).

answered Jul 05 '12 at 21:01

Ron

1,336
12
20

Could the session-handler be blocking requests when they each request is called with http://www.domain.com/phpscript.php? Wouldn't that seem like an external request? – BastiaanWW Jul 05 '12 at 21:46
Read this article: http://konrness.com/php5/how-to-prevent-blocking-php-requests/ – Ron Jul 05 '12 at 21:54

Why is there a delay in script execution when calling many php scripts simultaneoulsy using multi_curl?

1 Answers1