1

I've got a 1.5gig big file where I want to do some replacements.

But my apache runs out of memory. Is there a possibility to this job line by line?

<?php

$myfile = fopen("./m.sql", "r") or die("Unable to open file!");

$dateiinhalt = file_get_contents("./m.sql");


$search = array("ä",
    "Ä",
    "ö",
    "Ö",
    "ü",
    "Ü",
    "€",
    "§",
    "ß",
    "latin1_general_ci",
    "CHARSET=latin1",
    "‚",
    "„",
    "‘",
    "’",
    "“",
    "â€",
    "©",
    "®");

$replace = array("ä",
    "Ä",
    "ö",
    "Ö",
    "ü",
    "Ü",
    "€",
    "§",
    "ß",
    "utf8_general_ci",
    "CHARSET=utf8",
    "‚",
    "„",
    "‘",
    "’",
    "“",
    "”",
    "©",
    "®");


$content = str_replace($search, $replace, $dateiinhalt);
file_put_contents("./m.sql", $content);

echo "done";

thanks at all

Updated Code:

$reading = fopen('m.sql', 'r');
$writing = fopen('m.tmp', 'w');

$replaced = false;



$search = array("ä",
    "Ä",
    "ö",
    "Ö",
    "ü",
    "Ü",
    "€",
    "§",
    "ß",
    "latin1_general_ci",
    "CHARSET=latin1",
    "‚",
    "„",
    "‘",
    "’",
    "“",
    "â€",
    "©",
    "®");

$replace = array("ä",
    "Ä",
    "ö",
    "Ö",
    "ü",
    "Ü",
    "€",
    "§",
    "ß",
    "utf8_general_ci",
    "CHARSET=utf8",
    "‚",
    "„",
    "‘",
    "’",
    "“",
    "”",
    "©",
    "®");

if ($reading) {
    // read one line or 4096 bytes, whichever comes first
    while (($dateiinhalt = fgets($reading, 4096)) !== false) {
        // replace in that and write to output file
        fwrite($writing, str_replace($search, $replace, $dateiinhalt));
    }
    if (!feof($reading)) { // fgets() failed, but not at end of file
        echo "Error: unexpected fgets() fail\n";
    }
    fclose($reading);
    fclose($writing);
}

echo "done";

Felix
  • 5,452
  • 12
  • 68
  • 163
  • also http://stackoverflow.com/questions/2603807/how-to-save-memory-when-reading-a-file-in-php/2603923#2603923 and many others – Gordon Jun 03 '16 at 11:53
  • also, consider doing these replacements with `sed` on the commandline instead *or* [mysqldump](http://dev.mysql.com/doc/refman/5.7/en/mysqldump.html) the sql file with the correct encoding to start with. – Gordon Jun 03 '16 at 11:56

3 Answers3

5

Instead of file_get_contents (which reads the entire file into memory at once), use fopen and fgets to stream the file line-by-line.

ceejayoz
  • 176,543
  • 40
  • 303
  • 368
2

Yes. Its not hard. You seem to have already started - you should not open a file before running file_get_contents() on it (unless you explicitly want to lock it).

$search=Array(...
$replace=Array(...
$myfile = fopen("./m.sql", "r");
$output = fopen("./output.sql", "w");
while ($line=fgets($myfile)) {
   fputs($output, str_replace($search, $replace, $line));
}
symcbean
  • 47,736
  • 6
  • 59
  • 94
1

Indeed. To adapt the example in the manual ( http://php.net/manual/en/function.fgets.php ):

<?php
$handleR = fopen("./m.sql", "r") or die('cannot open input file!');
$handleW = fopen("./m.out.sql", "w") or die('cannot open output file!');
if ($handleR) {
    // read one line or 4096 bytes, whichever comes first
    while (($dateiinhalt = fgets($handleR, 4096)) !== false) {
        // replace in that and write to output file
        fwrite($handleW, str_replace($search, $replace, $dateiinhalt));
    }
    if (!feof($handleR)) { // fgets() failed, but not at end of file
        echo "Error: unexpected fgets() fail\n";
    }
    fclose($handleR);
    fclose($handleW);
}
Piskvor left the building
  • 91,498
  • 46
  • 177
  • 222