How to "release" memory in loop?

Question

I have a script that is running on a shared hosting environment where I can't change the available amount of PHP memory. The script is consuming a web service via soap. I can't get all my data at once or else it runs out of memory so I have had some success with caching the data locally in a mysql database so that subsequent queries are faster.

Basically instead of querying the web service for 5 months of data I am querying it 1 month at a time and storing that in the mysql table and retrieving the next month etc. This usually works but I sometimes still run out of memory.

my basic code logic is like this:

connect to web service using soap;
connect to mysql database
query web service and store result in variable $results;
dump $results into mysql table
repeat steps 3 and 4 for each month of data

the same variables are used in each iteration so I would assume that each batch of results from the web service would overwrite the previous in memory? I tried using unset($results) in between iterations but that didn't do anything. I am outputting the memory used with memory_get_usage(true) each time and with every iteration the memory used is increased.

Any ideas how I can fix this memory leak? If I wasn't clear enough leave a comment and I can provide more details. Thanks!

***EDIT

Here is some code (I am using nusoap not the php5 native soap client if that makes a difference):

$startingDate = strtotime("3/1/2011");
$endingDate = strtotime("7/31/2011");
// connect to database
mysql_connect("dbhost.com", "dbusername" "dbpassword");
mysql_select_db("dbname");
// configure nusoap
$serverpath ='http://path.to/wsdl';
$client = new nusoap_client($serverpath);
// cache soap results locally
while($startingDate<=$endingDate) {
    $sql = "SELECT * FROM table WHERE date >= ".date('Y-m-d', $startingDate)." AND date <= ".date('Y-m-d', strtotime($startingDate.' +1 month'));
    $soapResult = $client->call('SelectData', $sql);
    foreach($soapResult['SelectDateResult']['Result']['Row'] as $row) {
        foreach($row as &$data) {
            $data = mysql_real_escape_string($data);
        }
        $sql = "INSERT INTO table VALUES('".$row['dataOne']."', '".$row['dataTwo']."', '".$row['dataThree'].")";
        $mysqlResults = mysql_query($sql);
    }
    $startingDate = strtotime($startingDate." +1 month");
    echo memory_get_usage(true); // MEMORY INCREASES EACH ITERATION
}

No ideas so far. Try to paste some code, maybe we see something odd. — Jose Armesto, Sep 09 '11 at 21:03

score 7 · Accepted Answer · answered Sep 10 '11 at 07:59

7

Solved it. At least partially. There is a memory leak using nusoap. Nusoap writes a debug log to a $GLOBALS variable. Altering this line in nusoap.php freed up a lot of memory.

change

$GLOBALS['_transient']['static']['nusoap_base']->globalDebugLevel = 9;

to

$GLOBALS['_transient']['static']['nusoap_base']->globalDebugLevel = 0;

I'd prefer to just use php5's native soap client but I'm getting strange results that I believe are specific to the webservice I am trying to consume. If anyone is familiar with using php5's soap client with www.mindbodyonline.com 's SOAP API let me know.

answered Sep 10 '11 at 07:59

Devin Crossman

7,454
11
64
102

You could use xhprof for these kind of problems in the future. – chelmertz Sep 10 '11 at 08:01
Huh. Very interesting. Nice find! I hope people cross-reference this answer, since I couldn't find anything like this. I should have known to check the globals though. – Michael B Sep 10 '11 at 09:24
Thank you SO much for this. It was my savior after spending nearly 3 days trying to determine why one of my php scripts was using upwards of 500MB of memory. This was the exact issue. I also has problems with the native SOAP client in PHP5, as it kept giving me errors saying I wasn't providing fields that I was clearly providing, based on the WSDL. – Ricky Aug 03 '13 at 02:47
Nice job. memory usage 200MB turned into 40MB – ReZa Jun 12 '16 at 08:44

score -1 · Answer 2 · answered Sep 10 '11 at 06:09

-1

Have you tried unset() on $startingDate and mysql_free_result() for $mysqlResults?

Also SELECT * is frowned upon even if that's not the problem here.

EDIT: Also free the SOAP result too, perhaps. Some simple stuff to begin with to see if that helps.

answered Sep 10 '11 at 06:09

Michael B

1,743
4
21
35

mysql_free_result() doesn't work on INSERT queries and I have tried to unset every variable $startingDate, $sql, $mysqlResults, $row, $data, $soapResult and the memory usage doesn't change – Devin Crossman Sep 10 '11 at 06:12
Huh. Looks like others may have your problem too. http://www.php.net/manual/en/ref.soap.php#52758 Seems like a real memory hog. I've never consumed SOAP that much, mostly RESTful services serving JSON. There's no alternative like that, is there? – Michael B Sep 10 '11 at 06:16
But I'd try not SELECT *'ing from the SOAP server first. That probably isn't helping things here. – Michael B Sep 10 '11 at 06:19
Additionally: http://stackoverflow.com/questions/322510/moving-from-nusoap-to-php5-soap/556419#556419 Shows people with problems with SOAP in general. It's a stupid way to transfer data anyway; people need to get on the JSON bandwagon already. :p – Michael B Sep 10 '11 at 06:23
I agree I much prefer working with JSON but in this case I don't have any alternatives. I actually do not SELECT * I just wrote that when I wrote the example code – Devin Crossman Sep 10 '11 at 06:40
Saying that select * is a potential problem, doesn't make sense unless you're familiar with the database itself. The table being selected could have 2 columns, and "select *" would be no different from "select col1,col2". And suggesting the usage of JSON isn't helpful when SOAP web services are extremely common, and are often the only way to write an application into 3rd party services. – Ricky Aug 03 '13 at 17:30
The problem is for SOAP not SQL – ReZa Jun 12 '16 at 08:45

How to "release" memory in loop?

2 Answers2