Why does this code cause high load average on the server?

Question

The code below sends a msg when a value, obtained from a not so stable API, is within a certain range.

This code causes the load average, but not the CPU usage, to go up, likely due to high I/O wait.

CentOS 6.5
Kernel 2.6.32-431.11.2.el6.x86_64
Apache 2.2.15
PHP 5.3.3 (mod_fcgid/2.3.7)

> cat /sys/block/sda/queue/scheduler
noop anticipatory deadline [cfq]

The code had the same problematic effect on both dedicated hardware and on the cloud (both cases as a VM on KVM/Virtio).

What could be done to keep this code from *causing the processor to wait on instruction completion before processing new instructions**? I understand that lowering the timeout doesn't really solve the problem, only diminishes its impact.

*this is my understanding why this code causes load average to go up.

<?php

$min = '1';
$last_min = '1';

if (!empty($_GET['min'])) {
    $min = $_GET['min'];
} else {
    $min = false;
}

$ctx=stream_context_create(array('http'=>
    array(
        'timeout' => 10 // seconds timeout
    )
));

$json = file_get_contents('https://www.domain.com/api/ticker/',false,$ctx);

if (!empty($json )) {
    echo($json);
    if (@file_get_contents('log.txt')) {
        if (quote_changed($json)) {
            file_put_contents('log.txt', $json, FILE_APPEND);
        }
    } else {
            file_put_contents('log.txt', $json);
        }

    $obj = json_decode($json,true);
    $last = $obj['ticker']['last'];
    if (is_numeric($last)) {
        $last = (int)$last;
        $last_min = @file_get_contents('last_min.txt');
        $notified = @file_get_contents('notified.txt');
        if ($notified === false) {
            $notified = 'false';
            #echo "no notify file\n";
        }
        if (($last_min === false) || (($min) && ($last_min <> $min))) {
            $last_min = 1;
            $notified = 'false';
            file_put_contents('last_min.txt', $min);
            #echo "no min file or diff min\n";
        }
        #echo ('notified='.$notified.'\n');

        if (($last >= $min) && ($notified=='false')) {
            #$url = ('http://otherdomain.com/nexmo/sendmsg.php' . '?name=blah' . $last);
            #file_get_contents($url);
            #switch to SMS when going abroad and plugin new number when available
            mail("8885551212@mail.net","Blah at".$last,"","From: gaia@domain.com\n");
            file_put_contents('notified.txt', 'true');
            #echo "msg sent\n";
            } elseif (($last < $min) && ($notified=='true')) {
                    file_put_contents('notified.txt', 'false');
                    #echo "not sent\n";
        }           
    }
}

function quote_changed($current) {
    $previous = tailCustom('log.txt');
    #echo ('previous='.$previous);
    if ($previous === (trim($current))) {
        return 0;
    } else {
        return 1; 
    }
}

function tailCustom($filepath, $lines = 1, $adaptive = true) {

        // Open file
        $f = @fopen($filepath, "rb");
        if ($f === false) return false;

        // Sets buffer size
        if (!$adaptive) $buffer = 4096;
        else $buffer = ($lines < 2 ? 64 : ($lines < 10 ? 512 : 4096));

        // Jump to last character
        fseek($f, -1, SEEK_END);

        // Read it and adjust line number if necessary
        // (Otherwise the result would be wrong if file doesn't end with a blank line)
        if (fread($f, 1) != "\n") $lines -= 1;

        // Start reading
        $output = '';
        $chunk = '';

        // While we would like more
        while (ftell($f) > 0 && $lines >= 0) {

            // Figure out how far back we should jump
            $seek = min(ftell($f), $buffer);

            // Do the jump (backwards, relative to where we are)
            fseek($f, -$seek, SEEK_CUR);

            // Read a chunk and prepend it to our output
            $output = ($chunk = fread($f, $seek)) . $output;

            // Jump back to where we started reading
            fseek($f, -mb_strlen($chunk, '8bit'), SEEK_CUR);

            // Decrease our line counter
            $lines -= substr_count($chunk, "\n");

        }

        // While we have too many lines
        // (Because of buffer size we might have read too many)
        while ($lines++ < 0) {

            // Find first newline and remove all text before that
            $output = substr($output, strpos($output, "\n") + 1);

        }

        // Close file and return
        fclose($f);
        return trim($output);

}

?>

have you considered spiting your code in multiple pieces to locate the problem by yourself instead of throwing here an entire file of code? — Rotari Radu, May 06 '14 at 14:28
It's in the while loop conditions. Where is lines coming from, a database? — a coder, May 06 '14 at 14:46
Puts some timestamp log on the begin of every loop, then put timestamp before some function to locate better the problem — gafreax, May 06 '14 at 14:47

score 1 · Answer 1 · answered May 06 '14 at 14:56

1

Move the data into a database or if that's not an option, cache the unstable file into your server locally so you don't have to hit the external provider every time.

If neither of those are an option, it seems to me that the people providing the API need to improve their performance, you can verify this by just benchmarking how many milliseconds each hit takes through Firebug.

answered May 06 '14 at 14:56

user3608720

11
1

Thanks. But I do need the up to date date every minute. It changes very fast. I imagine there's a way in PHP to deal with unreliable data sources - does it have to make every instruction wait when it doesn't receive data? – Gaia May 06 '14 at 19:33

Why does this code cause high load average on the server?

1 Answers1