1

I'm using a PHP script to minimize javascript. The problem is that the script frequently takes a second or so before the server responds, which almost completely defeats the purpose. Even if just sending a 304, it still sometimes takes a second or so to send the response, during which time the browser is hanging.

On load, the script functions like this:

  1. determine what file (or files) to load based on query string var (or use default if query string var is null)
  2. check if a cached copy of the file is available and up-to-date (this is a .txt file stored on the server)
  3. if so, do nothing. If not, the cache file is created or recreated (I'm using JShrink 0.5.1 for minification)
  4. finally, check if the etags and last-modified of the cache file match what the user has
  5. -- if so, just send a 304
  6. -- if not, do a readfile on the cache file

It does NOT do this consistently - on one refresh it may take a second, the next time around it'll take 100ms, with both returning a 304. Since it's not consistent my best guess is that the delay is in accessing the cache file on the hard drive (even when sending a 304 it still has to do filemtime() on the cache file and any file(s) in $fileList, and later an md5_file() on the cache file to generate the current etag).

Any thoughts on how to get around this? Here are some relevant blocks of code:

<script type="text/javascript" src="/loadJS.php"></script>

// steps 2 and 3 (check cache, update if necessary, fileList is an array of paths to .js files):

if ( file_exists( $cacheFile ) ) {
    $cacheLastModified = filemtime( $cacheFile );
    for( $i = 0; $i < sizeof( $fileList ); $i++ ) {
        /* check if any files in the list are newer than the cache's last modification date
        * if so, cache must be updated
        */
        if ( $cacheLastModified < filemtime( $fileList[$i] ) ) {
            $outdated = true;
        }
    }
}

if ( isset( $outdated ) || !file_exists($cacheFile) ) {
    // update the cache file now
    ob_start();
    foreach( $fileList as $file ) {
        readfile( $file );
    }
    $output = ob_get_clean();
    include( "includes/JShrink-0.5.1.class.php" );
    $output = Minifier::minify($output, array('flaggedComments' => false));

    if ( $cacheFileFP = fopen( $cacheFile, "w" ) ) {
        if ( fwrite( $cacheFileFP, $output ) ) {
            fclose( $cacheFileFP );
        } else {
            // handle fwrite errors
        }
    } else {
        // handle fopen errors
    }
}



// step 4 (function checks etags and modification date):

function fileNotModified ( $file ) {
    /* returns false if file NOT modified
    * -- or an array of headers if file has been modified
    * use two criteria to determine if file has been modified
    * a) if-modified-since header vs the file's last-modified time
    * b) etag match
    * if if-modified-since is the same as file mod time, or etags match
    * --- return array of headers to send
    * else return false
    *
    * see http://stackoverflow.com/questions/2000715/answering-http-if-modified-since-and-http-if-none-match-in-php/2015665#2015665
    */

    $gmtModifiedTime = gmdate('r', filemtime($file) );
    $eTag = md5_file( $file );
    $result = false;

    if( isset( $_SERVER['HTTP_IF_MODIFIED_SINCE'] ) || isset( $_SERVER['HTTP_IF_NONE_MATCH'] ) ) {        
        if ( $_SERVER['HTTP_IF_MODIFIED_SINCE'] == $gmtModifiedTime || str_replace('"', '', stripslashes($_SERVER['HTTP_IF_NONE_MATCH'])) == $eTag ) {
            $result["ETag"] = $eTag;
            $result["Last-Modified"] = $gmtModifiedTime;
        }
    }

    return $result;
}

// steps 5 and 6 (304 header or send readfile output):

if ( $headers = fileNotModified( $cacheFile ) ) {
    // send the headers and 304 not modified
    foreach( $headers as $name=>$value ) {
        header( $name . ": " . $value );
    }
    header('HTTP/1.1 304 Not Modified');
    /*exit();*/
} else {
    // read and output the cache
    header("Last-Modified: " . gmdate("r", filemtime( $cacheFile )) );
    header("Etag: " . md5_file( $cacheFile ) );
    readfile( $cacheFile );
}
Mike Willis
  • 1,493
  • 15
  • 30
  • 4
    you don't need to minify on every page load you should only minify the js source files once then upload the minified scripts to your web server. – Dave Jul 23 '13 at 14:22
  • I can reflect what @Dave said too, this isn't the job of PHP. Let your web server handle it. – Rudi Visser Jul 23 '13 at 14:23
  • @Dave This condition : `if ( isset( $outdated ) || !file_exists($cacheFile) )` says that it is not minified "on every page load". – Brewal Jul 23 '13 at 14:25
  • @brewal clearly thats why some times it takes 1 second + to load and other times ms, there's a logic error in the code but overall the primary logic error is WTF are you doing cache or minification like this its completely against the whole point. – Dave Jul 23 '13 at 14:26
  • Your `if ( $cacheFileFP = fopen( $cacheFile, "w" ) )` should be inside `if ( isset( $outdated ) || !file_exists($cacheFile) )` – Brewal Jul 23 '13 at 14:34
  • @Dave I would actually be interested in a pure unix server solution which dynamically generates minified files. – Brewal Jul 23 '13 at 14:42
  • @Brewal you're right, I apologize - your suggestion is exactly how I have it locally but I screwed it up during formatting when pasting the code into SO. I corrected the code in the original post. Since the code has been this way all along, I'm still unsure what the problem might be. – Mike Willis Jul 23 '13 at 14:54
  • @RudiVisser, Of course it can be the job of PHP. All that needs to be done is some caching. This minification is no different than any other dynamic resource. – Brad Jul 23 '13 at 14:58
  • @brewal check sourceforge there's hundreds of js and php minifiers on there. hell there's loads on the web in general that can be remote called via api. There's tidy for html+css too if you really want to goto extremes. either way he could simply run his current minification script on a cron job once a day/week/month/whatever his cache expiry is and then the web front end ony ever calls the cache file. if its missing it puts a require cache generation into a pending table then just have a clean up script to process the pending table once an hour or whatever. – Dave Jul 23 '13 at 14:59
  • Its not that minification doesn't need to be done its that it doesn't need to be done on the fly / on demand like this it can be done as a background cron script(s) and put all the load on the server / outside of apache. – Dave Jul 23 '13 at 15:00
  • @Dave, the minification only takes place if the source files have been modified. I agree it would be best to just upload a minified file, but I consider this a very fair trade-off in exchange for not having to think about minification ever again. Since the response is slow even when sending a 304 status code (which doesn't minify), I think this rules out minification as the cause of the problem (unless I still have a logic error in my code). – Mike Willis Jul 23 '13 at 15:01
  • @Brad I never said that it *couldn't* be the job of PHP, just that in my opinion it shouldn't be. If you wanted it to be half as efficient as a web server in this manner, you'd need opcode caching and memory caching for the actual minified file, otherwise you're always incurring a penalty for both invoking PHP and reading the file in the first place, every single time. – Rudi Visser Jul 23 '13 at 15:05
  • @RudiVisser (and Dave) also note that the method I've chosen eliminates the need for a cron, or table usage, etc. Plus if I change a JS file and upload it, I can reload any page on the site and immediately get the minified result, for me and all visitors to the site. There are others way to do it that work, like you've mentioned, but this is the way I prefer. But this debate is separate from the problem I'm having since (barring an error in my code) the minification itself is not causing the slowdown. – Mike Willis Jul 23 '13 at 15:05
  • @MikeWillis Depending on your server, you may not be wrong with your guess. I personally minify my scripts in console with [yui-compressor](http://yui.github.io/yuicompressor/). You have to minify them if you change any of your scripts, but in production purpose it is good enough. I would say, don't minify when in dev, minify once in prod. – Brewal Jul 23 '13 at 15:11
  • @MikeWillis I'm not referring to the minification, I'm talking about the caching principle in the first place - it's the job of the web server. When you're changing your site there should be some form of deployment strategy in place anyway no matter how small, so writing something to pre-minify and upload shouldn't be too much of a task. – Rudi Visser Jul 23 '13 at 15:18
  • @MikeWillis - I would recommend approaching this setup with a 404 handler in your css/js folder. If the minified file exists load it straight up....if not then go through the process of creating it via php. Also, make sure you clear you created new files when they are modified properly. Lastly, check out http://php.net/manual/en/function.stream-resolve-include-path.php – hendr1x Jul 23 '13 at 15:19
  • @RudiVisser I think you and Brewal are right, and you have a point about invoking PHP every time I want a .js file. Even if PHP is just checking etags and sending a 304, it can't be as fast as pointing straight at the .js file itself. So with speed as my goal the best solution must be to avoid PHP even if it's more convenient. Maybe I'll set up a cron to create these cache files (note that they also combine several scripts into one for me), and I can manually run the cron if I've made a change that I need to test right away. Less convenient for me, but probably best overall in terms of speed. – Mike Willis Jul 23 '13 at 15:26
  • @RudiVisser, In that case, I certainly do not disagree. – Brad Jul 23 '13 at 15:31
  • Out of curiosity I'm still wondering if disk access was the cause of the slow response time. Does everyone agree that this is the cause? If so that means doing the caching/minification with PHP has a nearly unsolvable problem (programmatically). I hate unsolvable problems. – Mike Willis Jul 23 '13 at 15:42
  • highly unlikely disk IO is an issue unless you're talking a site with hundreds of thousands of hits loading large files off disk every time. Stick APC on swap to memcached for your queries and pre-minify if you're really that fussed about speed (or do away with apache completely go with nginx,cherokee,lighttpd) personally I'd create a deploy folder I upload to there then have a 10 minute cron that checks that folder for changes. does it bit and moves to a live folder then empties the deploy folder again. problem solved and when you make a change it'll be no more than 10 mins before live – Dave Jul 23 '13 at 17:31
  • btw if you want to check if it is your minification code its simple pre-minify using one of the hundred or so minification scripts out there and upload them directly and strip out your php code and the 304 etc then see if its still slow or if its fast. – Dave Jul 23 '13 at 17:32

1 Answers1

0

I didn't test the following code, but this is what you should do:

if(!file_exists($js_path_cache) || filemtime($js_path_cache) < filemtime($js_path) {
  // MINIFY YOUR JS HERE
}
Chololoco
  • 3,140
  • 1
  • 21
  • 24