Reduce looping execution time

Question

i got 35 second to execution this code. how to reduce the execution time? what should i change in this source code.

    $file_handle = fopen("WMLG2_2017_07_11.log", "r");
while (!feof($file_handle)) {
   $line = fgets($file_handle);
   if (strpos($line, 'root@CLA-0 [WMLG2]  >') !== false) {
        $namafileA = explode('> ', $line);
        $namafile = str_replace(' ', '_', $namafileA[1]);
        $filenameExtension = $namafile.".txt";
        $file = preg_replace('/[^A-Za-z0-9\-_.]/', '', $filenameExtension); // hapus special character kecuali "." dan "_"
    } else {
        $newfile = fopen("show_command_file_Tes2/$file", "a");
        fwrite($newfile, $line);
    }
}
fclose($file_handle);

Fiber · Accepted Answer · 2017-08-26T09:31:16.103

I found some mistakes you did with the original code that could impact your performance, but I am not sure how much.

If I understand correctly, you are opening a log file and sorting the messages out to separate files.

You have not pasted an example from the log file, but I assume you have duplicate file targets, not every line of the log file has individual file targets.

Your code opens, but never closes the handles and it stays open during the script run. The file handles are not closing on outer-scope by garbage collector, you have to do it manually to release the resources.

Based on that you should store the file pointers (or at least close them) and re-use that handle what is already open. You are opening at least X line of handle during the execution and not closing it / reusing it where X is the line count in the file.

Other thing I noticed, your lines may be long ones, an that is a rare case where php's strpos() function could be slower than a regex matching the correct position of the string. Without the log file, I can't say for sure because preg_match() is pretty expensive function on simple / short strings (strpos() is way faster.)

If its a log file, most likely starts with that "root@CLA"... string, you should try to match that if you can specify the string position with ^ (begining of the string) or $ (end of string).

<?php

$file_handle = fopen("WMLG2_2017_07_11.log", "r");

//you 'll store your handles here
$targetHandles = [];

while (!feof($file_handle))
{
    $line = fgets($file_handle);
    if (strpos($line, 'root@CLA-0 [WMLG2]  >') !== false)
    {
        $namafileA = explode('> ', $line);
        $namafile = str_replace(' ', '_', $namafileA[1]);
        $filenameExtension = $namafile . ".txt";
        $file = preg_replace('/[^A-Za-z0-9\-_.]/', '', $filenameExtension); // hapus special character kecuali "." dan "_"
    }
    else
    {
        //no $file defined, most likely nothing to write yet
        if (empty($file))
        {
            continue;
        }

        //if its not open, we'll make them open
        if (empty($targetHandles[$file]))
        {
            $targetHandles[$file] = fopen("show_command_file_Tes2/$file", "a");
        }
        //writing the line to target
        fwrite($targetHandles[$file], $line);
    }
}

//you should close your handles every time
foreach ($targetHandles as $handle)
{
    fclose($handle);
}

fclose($file_handle);

from your code, variable $file undefined here its the log files if you want to see it https://www.dropbox.com/s/z8tvmegokghrjok/WMLG2_2017_07_11.log?dl=0 @Fiber — nawfal Af, Aug 26 '17 at 09:25
Yes, you are correct, but I didn't know what were your intentions with that variable, you are defining in `$file = preg_replace('/[^A-Za-z0-9\-_.]/', '', $filenameExtension); // hapus special character kecuali "." dan "_"` )` but that could be undefined. Let me edit my post and correct that. — Fiber, Aug 26 '17 at 09:28
$file = preg_replace('/[^A-Za-z0-9\-.]/', '', $filenameExtension) im use it to remove all space, because if im not defined it the file name will be ex: show_alarm_active .txt there are space in the last of string before i put extension — nawfal Af, Aug 26 '17 at 09:30
Yes, but `$file` as a variable never defined before, you are defining with that expression. I made some changes to skip when `$file` is not yet defined. — Fiber, Aug 26 '17 at 09:33
wow its worked, im just defined $file = ""; and its run <1second thanks buddy — nawfal Af, Aug 26 '17 at 09:37
Make sure the result is correct, that `$file` could cause problems and data loss at the begining of the source file while `$file` is not defined. — Fiber, Aug 26 '17 at 09:39
now im confuse how to implement it with file storage laravel — nawfal Af, Aug 26 '17 at 09:53
I am not familiar with laravel, but looking at the docs I would pull up the lines into memory (if the log file is not too large) and flush them at the end of the script parsing. Alternatively you can make a solution where you are pulling up the lines to memory (collecting target files and content) and flushing the files after every X line. Try to implement and throw up a new question with it, maybe someone have a better solution with better konwledge of laravel. — Fiber, Aug 26 '17 at 09:59

Reduce looping execution time

1 Answers1