Caching large Array causes memory exhaustion

Question

So I'm trying to cache an array in a file and use it somewhere else.

import.php

// Above code is to get each line in CSV and put in it in an array 
// (1 line is 1 multidimensional array) - $csv 

$export = var_export($csv, true);
$content = "<?php \$data=" . $export . ";?>";
$target_path1 = "/var/www/html/Samples/test";

file_put_contents($target_path1 . "recordset.php", $content);

somewhere.php

ini_set('memory_limit','-1');
include_once("/var/www/html/Samples/test/recordset.php");
print_r($data);

Now, I've included recordset.php in somewhere.php to use the array stored in it. It works fine when the uploaded CSV file has 5000 lines, now if i try to upload csv with 50000 lines for example, i'm getting a fatal error:

Fatal error: Allowed memory size of 67108864 bytes exhausted (tried to allocate 79691776 bytes)

How can I fix it or is there a possible way to achieve what i want in a more convenient way? Speaking about the performance... Should i consider the CPU of the server? I've override the memory limit and set it to -1 in somewhere.php

first of all try `phpinfo()` to check if `memory_limit` has been overridden. Tell me if you are using Linux or Windows? — Apul Gupta, Oct 17 '14 at 07:13

score 0 · Answer 1 · edited May 23 '17 at 12:06

There are 2 ways to fix this:

You need to increase memory(RAM) on the server as memory_limit can only use memory which is available on server. And it seems that you have very low RAM available for PHP.

To Check the total RAM on Linux server:

<?php
  $fh = fopen('/proc/meminfo','r');
  $mem = 0;
  while ($line = fgets($fh)) {
    $pieces = array();
    if (preg_match('/^MemTotal:\s+(\d+)\skB$/', $line, $pieces)) {
      $mem = $pieces[1];
      break;
    }
  }
  fclose($fh);

  echo "$mem kB RAM found"; ?>

Source: get server ram with php

You should parse your CSV file in chunks & every time release occupied memory using unset function.

Caching large Array causes memory exhaustion

1 Answers1