php is locking a deleted session file

Question

Our php app, running under apache, is occaisionally trying to lock a deleted session file, this hangs the apache process and eventually we run out of processes.

The evidence :

strace : flock(89, LOCK_EX                   -----stuck in flock for File Descriptor 89

and then :

lsof, for the same process :

httpd 22161 apache 89u ... /var/lib/php/session/sess_mf7svvirg7rbu9l0m5999pm6h4 (deleted)

Any ideas as to why this is occurring? And what we could do to prevent it?

a) This question is probably better directed towards http://webmasters.stackexchange.com/ b) Who deleted the session file? You or PHP? How long ago was it deleted? Are you sure it has been deleted, and is not just locked by another process? — DaveRandom, Feb 20 '12 at 17:06
are you using php's session management or a custom session handler? — dqhendricks, Feb 20 '12 at 20:36
Anything come through the error log which may tie into why this is happening? — Mike Purcell, Feb 21 '12 at 06:22
php's session handler, nothing in the error log. The file is deleted. webmaster seems to be mainly concerned with Google analytics. The file wasn't deleted by hand, but I can look for a rogue cron. — derbexuk, Feb 21 '12 at 09:11
I would not know _why_ this is happening, but the standard Debian way to prevent has been for ages to not let Apache / a request do the garbage collection, but set the chance the 0 and instead run a cronscript that just checks the timestamps on files. For more control you could define a custom dir for those session files so the timeout can once again be set and not hindered by other apps running. As a guess to why: is the process / request locking the file also the one trying to reopen it, in other words: does it happen often that a request with the very same session-id cleans up its old one? — Wrikken, Feb 23 '12 at 23:51

Quinn Comendant · Answer 1 · 2015-11-12T21:10:48.253

I've experienced a problem with identical symptoms to yours, except whether the session file was deleted or not was irrelevant (and may be for you too—do you have evidence the deletion of the session file is related to the flock?).

In my case, it was a race condition for access to the same session file between two scripts:

/page1.php executes fine when loaded. It includes javascript that does a XMLHttpRequest to /background.php.
When /background.php is accessed it runs readfile(http://remote.url), which is a non-existent URL.
When navigating to /page2.php the script is stalled. A strace shows flock(89, LOCK_EX, and lsof indicates it is waiting for read/write access on a session file.

In this scenario, both /page2.php and /background.php are both waiting on the same session file, but the latter was unable to do so because it was held up waiting for the readfile() to timeout. I got this in a php_error_log:

PHP Warning:  readfile(http://remote.url): failed to open stream: Connection timed out in […]/background.php on line […]

So the problem was with an entirely different script than the perceived culprit.

You can check for this problem quickly by grepping all opened files for the php session file to see which httpd process is using it:

$ sudo lsof | grep sess_it9q6kkvf83gcj72vu05p9fcad7
httpd     31325    apache   74u      REG                8,5       410   11634061 /var/lib/php/session/sess_it9q6kkvf83gcj72vu05pfa543
httpd     31632    apache   74uW     REG                8,5       410   11634061 /var/lib/php/session/sess_it9q6kkvf83gcj72vu05pfa543

If two httpd processes are accessing the same session file, probably this is your problem. Now you can check what those processes are doing with strace, or in my case it was enough to use apachectl fullstatus:

$ sudo apachectl fullstatus | grep 31632
5-7  31632 0/1/1169  _ 0.05 6     30692 0.0  0.02  3.45 111.222.333.444 www.example.com.com     /background.php

score 1 · Answer 2 · edited May 23 '17 at 12:13

Is there any tremendous(client) need to delete session file? You shouldn't do that. PHP itself implements a garbage collection mechanism to delete defunct session files. It's going to be much more efficient than anything else you could write yourself using PHP.

You can use:

session_destroy — Destroys all data registered to a session
session_unset - Free all session variables

And do session_start() again.

See more here.

Update:

PHP is a very clean scripting engine, which leaves no garbage on your system! The session files are automatically deleted after the session-timeout is reached. session-timeout may be case with your system I think so . So if the time-out is set to 20 minutes, the files will be deleted 20 minutes after the last access. Same for the cookie. Every time, an page is requested, the cookie-ttl is set to now + 20 minutes.

check out some of the configuration directives. Odds are that you have set gc_maxlifetime to a very high value, and they simply haven't reached the "expiry" limit to be automatically collected. You should check your applications to be sure that they aren't setting strange values for the session settings (using ini_set()) during runtime.

PHP has an internal algorithm that runs on session_start() that decides whether it should do a garbage cleaning run and remove old session files. For performance reasons, I believe the default chance to do a run each time is 1%, so on average a garbage-collection routine will be started once in every 100 session_start() calls. If you find this isn't enough, you can raise the gc_divisor setting to something higher than 1.

I don't think he's doing it deliberately... the question is about why PHP is trying to delete the session file. — Lightness Races in Orbit, Mar 03 '12 at 18:17
I read `is occaisionally trying to lock a deleted session file`. So that's I given this answer. Then main concern is which code is trying to delete session file. — Somnath Muluk, Mar 03 '12 at 18:28
If the OP already knew what it was that is trying to lock a deleted session file, then there would be no question. — Lightness Races in Orbit, Mar 03 '12 at 18:29

score 0 · Answer 3 · edited May 23 '17 at 12:24

No idea as to the why for your case.

This question have gotten some good suggestions...

Have you tried reinstalling the environment, or moving to a different server? Will it work if you overrule the default session handler and use fopen($file, 'w') instead of fopen($file, 'x')?

Also the workaround of implementing a session handler based on a database, there are a million or so guides to setup a "php and mysql session handler" for instance.

score 0 · Answer 4 · answered Mar 05 '12 at 14:32

0

This is probably something more of a workaround for your case, but why don't you migrate sessions altogether from the file system to the database?

You could get more information here http://php.net/manual/en/function.session-set-save-handler.php

answered Mar 05 '12 at 14:32

mobius

5,104
2
28
41

score 0 · Answer 5 · answered Mar 05 '12 at 17:37

0

Did you tried reading those 2 open bugs?

https://bugs.php.net/bug.php?id=47640 https://bugs.php.net/bug.php?id=32092

Try to call session_write_close() (ugly work-around) as your last line or to upgrade PHP.

answered Mar 05 '12 at 17:37

dAm2K

9,923
5
44
47

php is locking a deleted session file

5 Answers5