I've run into a problem where I exceed my memory limit in php 7.0.5, memory limit is set to 3072M
PHP Fatal error: Allowed memory size of xxx bytes exhausted PDO
IIS Server running 4 G Ram total. I am connecting to a remote dedicated MySQL server.
class DB{
private $dbh;
private $stmt;
public function __construct($db) {
$host = HOST;
$this->db = $db;
try {
$this->dbh = new PDO(
"mysql:host=$host;dbname=$this->db",
USER,
PASS
);
}catch (PDOException $e) {
print "Error!: " . $e->getMessage() . "<br/>";
die();
}
}
public function query($query) {
$this->stmt = $this->dbh->prepare($query);
return $this;
}
public function bind($pos, $value, $type = null) {
if( is_null($type) ) {
switch( true ) {
case is_int($value):
$type = PDO::PARAM_INT;
break;
case is_bool($value):
$type = PDO::PARAM_BOOL;
break;
case is_null($value):
$type = PDO::PARAM_NULL;
break;
default:
$type = PDO::PARAM_STR;
}
}
$this->stmt->bindValue($pos, $value, $type);
return $this;
}
public function execute() {
return $this->stmt->execute();
}
public function resultset() {
$this->execute();
return $this->stmt->fetchAll();
}
public function fetchArray() {
$this->execute();
return $this->stmt->fetchAll();
}
public function numRows() {
$this->execute();
return $this->stmt->rowCount();
}
public function single() {
$this->execute();
return $this->stmt->fetch();
}
public function insertID()
{
return $this->dbh->lastInsertId();
}
}
function updateBooleanInRecord()
{
$query = "select inRecordID, recordStatus from `ems-reporting`.inrecord";
$db = new DB("reporting");
$db->query($query);
if($db->numRows() > 0){
$out = $db->fetchArray();
foreach($out as $output){
if($output['recordStatus'] == "False"){
updateFalseRecord($output['inRecordID']);
}
if($output['recordStatus'] == "True"){
updateTrueRecord($output['inRecordID']);
}
}
}
}
function updateFalseRecord($inRecordID){
$query = "UPDATE `reporting`.inrecord set recordStatus = '0' where inRecordID = ?";
$dba = new DB("ems-reporting");
$dba->query($query);
$dba->bind(1, $inRecordID);
$dba->execute();
unset($dba);
$dba = null;
}
function updateTrueRecord($inRecordID){
$query = "UPDATE `reporting`.inrecord set recordStatus = '1' where inRecordID = ?";
$dbb = new DB("ems-reporting");
$dbb->query($query);
$dbb->bind(1, $inRecordID);
$dbb->execute();
unset($dbb);
$dbb = null;
}
updateBooleanInRecord();
The dataset I am pulling from contains over 36 million rows of data and the error I am receiving appears to relate to:
public function execute() {
return $this->stmt->execute();
}
The research I have done so far suggests that I should add the following just as after:
$this->dbh = new PDO(
"mysql:host=$host;dbname=$this->db",
USER,
PASS
);
Buffered Query Attribute Addition:
$this->dbh->setAttribute(PDO::MYSQL_ATTR_USE_BUFFERED_QUERY, FALSE);
I have tried this, but the request runs for a few seconds, maybe 10 and then simply stops with no error being displayed in the browser or appearing in the logs. So I have zero indication of what halted the script. The memory usage does not appear to grow at all when this addition is made.
Before I added the above setAttribute
I opened the task manager to watch the process consume memory until it reached its limit, then it the process fails with the following msg in the browser:
Fatal error: Allowed memory size of 3221225472 bytes exhausted (tried to allocate 1166262464 bytes) in D:\path\to\folder\index.php on line 54
Hopefully someone can point out what I may be doing wrong here, or some tips to try and locate the issue, basically any help would be greatly appreciated.
EDIT: The following SQL statements resolved my issue at the source, I wasn't thinking clearly. Thank you for the responses though, I see fetchAll() is obviously not the way to do this and leads to exactly the issue I encountered...time to sleep...
UPDATE `reporting`.inrecord set recordStatus=1 where recordStatus="True"
followed by
UPDATE `reporting`.inrecord set recordStatus=0 where recordStatus="False"