3

I am currently debugging a script that constantly runs into OutOfMemory exceptions. It is run as a cronjob and usually runs fine, but when the cronjob wasn't run for a while (for whatever reason) the script has to handle to many elements that queued up and will run into a OutOfMemory exception.

From examining the code I was not able to spot the problem. I believe one of the iterative function calls might leak memory, but I am not sure which one and where. Is there an option to get PHP to dump the heap, when an OutOfMemory exception occurs? I might be able to spot the problem from there (most likely).

trincot
  • 317,000
  • 35
  • 244
  • 286
Daniel Baulig
  • 10,739
  • 6
  • 44
  • 43

6 Answers6

5

While I was not able to find a "dump heap on Exception" option, I did find get_defined_vars() which is basically a heap dump if called from a global scope. Using this I was able to see that there where hundreds (actually thousands) of still referenced database rows hanging around in my memory. This was due to a not freed mysql result resource somewhere in the infamous function that caused the leak. I found it and fixed it. It runs well now.

Daniel Baulig
  • 10,739
  • 6
  • 44
  • 43
  • How did you use get_defined_vars() to actually pinpoint the problem? Did you just run the script for a while and then have it dump, or is there a way to trigger calling that function when the script runs into the memory leak? – Jordan Warbelow-Feldstein Dec 29 '10 at 23:58
  • 1
    I implemented an iteration counter that would reveal me the approximate number of iterations before it crashed, in my case that was somewhere between 20000 and 22000 iterations. I then had the script run and used get_defined_vars on the 18000th iteration to dump the heap. In my case it then was painfully obvious what cluttered the memory. – Daniel Baulig Jan 08 '11 at 11:33
  • To help make things a little clearer, I dumped `get_defined_vars()` as json to a file for each iteration. Then ran those files through `jq .` to get them similarly formatted. Once you have that you can diff the two files and probably see something added. With that info I went back to the last dump file and found the added items. – Mat Schaffer Feb 10 '16 at 06:55
1

Well, easiest approach would be to use a try-catch block around that part of your script where the error possibly occurs and you will have to dump the stack in the catch part. The problem might be that the machine won't be able to react cause the memory is full and it terminates. I do not know if it helps to discard some variables to free up some memory to output some data.

EDIT: For this purpose use the php function debug-backtrace. This will give you a stack trace. So finding the error will be much likely in case the machine is still up.

danronmoon
  • 3,814
  • 5
  • 34
  • 56
Thariama
  • 50,002
  • 13
  • 138
  • 166
  • I think his question is about *how* to dump the heap. – Pekka Oct 25 '10 at 12:01
  • Pardon me, but how do I "dump the heap" in the catch part? I am of course not talking about SplHeap, but the PHP scripts memory heap. – Daniel Baulig Oct 25 '10 at 12:03
  • Searching for Memory (http://php.net/manual-lookup.php?pattern=memory&src=) or Heap (http://php.net/manual-lookup.php?pattern=heap&src=) do not yield any promising results. (OT: how do I embed links in comments, or can't I?) – Daniel Baulig Oct 25 '10 at 12:16
  • I also already know in which part of the script it dies. It's not always the exact line of code, but always within a certain loop. I suspect a memory leak anywhere within that loop, however, the stack of that loop is pretty deep (so lots of function calls, within function calls, within function calls) and the point where the script crashes must not necessary be the place where the script leaks. In fact it crashes at various points all the time (propably due to non-deterministic gc). – Daniel Baulig Oct 25 '10 at 12:22
  • Hard to grasp because of nested function calls - roughly 1000 lines, maybe a bit more. – Daniel Baulig Oct 25 '10 at 12:32
  • did you try to find out the number of loops at which the error occurs? – Thariama Oct 25 '10 at 12:41
  • No, I did not. How could that help me? – Daniel Baulig Oct 25 '10 at 14:25
  • it would help if you would know that the error occured after the 435th loop - it would make debbuging easier - in that case you could check for the 435th loop and print out debugmessages (then you can find the error in O (log n) time when n is the amount of lines of code to check). Make sure to restart your server everytime you check for the number of loop in order to empty the memory used by your server. – Thariama Oct 25 '10 at 15:10
  • I dont think knowing that the error occurs on the 435th iteration will help me, since what the loop is doing is processing a bunch of table rows and updating them back again. The table rows are basicly the same (of course different values, but nothing fancy or of variant length like blobs). Each iteration is basicly the same, except that one row gets value A and another row gets value B. Of course this is simplyfied, but in end it boils down to exactly that. Determining if a row gets value A or B of course is complex and involves lots of stuff. – Daniel Baulig Oct 25 '10 at 19:14
  • But again, nothing like "Oh, that produces an overflow on my array_size variable if $i is 435 and my array suddenly is 4 billion elements long", that could cause a sudden OOM exception on the 435th iteration. I dont want to rule that kind of stuff out completely, after all I might miss something, but a memory leak is the most likely cause for the problem (maybe a recursive reference somewhere ) and I will not start hunting for witches until I can rule out a memory leak. Anyway, thank you for your precious time and your willingness to help, +1 for that. – Daniel Baulig Oct 25 '10 at 19:19
  • thanks, to count the iterations of the loop is fast (takes 2 lines of code). You can never have enough information regarding a bug or other problem :) – Thariama Oct 26 '10 at 07:10
0

This is as good of a 'heap dump' as I'm able to quickly write in PHP. I take the defined variables and functions, then sort by their serialized length. Serialized length isn't a 100% reliable method for getting a variable's size, but it's pretty good, and generally useful for determining which objects are your memory hogs:

$memmap = array_map(function($var) { return strlen(serialize($var)); }, array_merge(get_defined_functions(), get_defined_vars())); arsort($memmap); var_dump($memmap);

You may want to tweak the callback function a bit if you'd like your results to be more verbose, or to recurse through the defined variables.

Josh from Qaribou
  • 6,776
  • 2
  • 23
  • 21
0

I've never seen PHP provide a native facility for this but a few other things might exist:

Try: https://github.com/mcfunley/php-heap/blob/master/php-heap.py

It could also be possible to write an extension to achieve the same.

jgmjgm
  • 4,240
  • 1
  • 25
  • 18
0

Just do not load all objects together to memory, but read-as-you-process-them?

joni
  • 5,402
  • 1
  • 27
  • 40
  • That is what the script seems to do, from visually inspecting the code. As I said, I suspect a memory leak. Maybe due to circular references (this is PHP 5.0), but debugging thousands of lines of code by reading them line for line to maybe spot the problem is just a waste of time. – Daniel Baulig Oct 25 '10 at 12:15
  • You need to be creative in PHP to create a memory leak, except you just take an array and put in data (possibly using some strange associative keys) without removing it again... – joni Oct 25 '10 at 12:37
  • You mean as creative as creating recursive associations? They, for example, will create memory leaks in pre 5.3 because the gc cannot detect them. Since the code I am debugging is not mine, I do not know what sick stuff might be going on. – Daniel Baulig Oct 25 '10 at 14:24
0

I've had lots of problems with simpleXML and memory leaks. They are a pain in the are to track down... took me days to figure out that simpleXML was causing then and then fix them. As far as i know you cand programatically set a handled for OOM:)

Also, PHP's functions for displaying memory info fails to detect the memory leaks, i had scripts eating up ~1Gb of ram, but PHP's functions reported only 100Mb used:)

Quamis
  • 10,924
  • 12
  • 50
  • 66