Access logging in PHP

Question

I want to log access to any files in the /files folder, so I can process it with PHP to generate some statistics.

I don't want to write a custom PHP handler called via RewriteRule because I don't want to have to deal with status codes, MIME-types and caching headers, and file locking issues.

I don't have access to the server configuration, so I can't use CustomLog (I do have access to .htacess).

I can't use X-Sendfile because it's not enabled.

I don't have access to the access.log.

---

Looking for an authorative answer.

Answer 1

That's quite a few restrictions you've placed there.

You can do this with a custom handler installed via a PHP include at the top of each applicable (or, with __FILE__ parsing, not applicable) script. You must have a script which runs when each file is hit, and you've excluded alterations to the server config (including, I believe, .htaccess when you said RewriteRule wasn't good enough), so that means that you will be doing this through a script-based gatekeeper. You cannot have a solution which meets your constraints and has users go to files without hitting PHP (or another server-side dynamic language) first. Caching can be preserved by redirecting the user to the actual files instead of running static content through PHP.

You can store the log information in a database, or a file in a location writable by the server (watch out for contention if you use files - append mode is tricky).

EDIT: quickshiftin points out two ways you can get PHP invoked without having to add include calls by hand.

Answer 2

Create an auto_prepend_file and define a function to log w/e you want. You'll need access to .htaccess in order to set these (and the webhost will need something like AllowOverride all in the vhost) or with PHP 5.3 you can use the per-directory INI feature.

.htaccess

php_value auto_prepend_file /path/to/file.php

per-directory php.ini (PHP 5.3 CGI/Fast CGI SAPI)

user_ini.auto_prepend_file = /path/to/file.php

Then for your file /path/to/file.php (something more elegant I'm sure ;))

file_put_contents(
    LOG_FILE,
    implode(PHP_EOL . PHP_EOL, array(
                'SERVER: ' . PHP_EOL . print_r($_SERVER, true),
                'REQUEST: ' . PHP_EOL . print_r($_REQUEST, true)
            )),
    FILE_APPEND
);

The beauty of this approach is you'll likely be able to get away with it and you'll only need to define / include the logging code in one place.

EDIT:

Upon retrospection I see you want this to work for arbitrary types of files... Yes that would be rather rough. Best bet I can think of is labeling these files as .php or defining custom mime types in .htaccess. The idea would be to run the files through the PHP interpreter, thereby executing the auto_prepend_file and since there are no PHP tags in the file the content is sent directly to the client. Maybe even a tiny bit of PHP atop each file of content setting the ContentType header. I'm not even sure that would work but it might.

Answer 3

That's pretty simple to do considering you don't need to restrict access.

build a page logger.php that takes in input the file requested like:

logger.php?file=abc.exe

In the logger.php you just have to log this access then rediret to the file:

file_put_contents('log', $_GET['file'] . ' requested',FILE_APPEND);
header('Location: files/'.$_GET['file']);

Just check the $_GET['file'] for malicious files

Of course you have to replace links in your site, from:

<a href="files/abc.exe">

to

<a href="logger.php?file=abc.exe">

Answer 4

It seems like the intention here is to circumvent all the systems that are inherently in place in Apache and PHP. If these restrictions are actually in place on your server instance you are far much better off asking for a change to your privileges than devising a workaround that your system admin may or may not be happy with you implementing.

Answer 5

Might not be exactly what you want but why don't you use a different solution altogether?

You could use Google Analytics VirtualPageviews to track the file downloads via Javascript.

See here for more information: http://support.google.com/googleanalytics/bin/answer.py?hl=en&answer=55529

You could even create your own JS to track the file downloads via the browser without having to bother with GA.

Update:

As I said you could easily create your own JS to track them without having to bother with GA. Here is a silly example in jQuery that would work (haven't tested it - just wrote it of the top of my head):

Code sample:

JS Side:

$(document).ready(function() {
  $("a").click(function() {
    if( $(this).attr('href').match(/\/files\/(.*)/) ) {
      $.ajax({
        url: '/tracking/the/file/downloads.php'
        data: {
          'ok': 'let\'s',
          'add': 'some information',
          'about': 'the user that initiated',
          'the': 'request',
          'file': $(this).attr('href')
        }
      });
    }

    return true;
  });
});

Answer 6

Works only in mod_php case. There is some performance hit -- apache_lookup_uri() does additional apache internal sub-request.

As others pointed you need .htaccess like

RewriteEngine On
RewriteRule ^/handler.php$ - [L]
RewriteRule ^/([a-zA-Z0-9\.]+)$ /handler.php?filename=$1 [L]

In handler.php file use virtual() function to perform apache subrequest. Example here: http://www.php.net/manual/en/function.virtual.php#88722

Updated and tested (but rather minimal) solution:

<?php
//add some request logging here
$file = $_GET["filename"];

$file_info = apache_lookup_uri($file);
header('content-type: ' . $file_info -> content_type);
// add other headers?
virtual($file);
exit(0);
?>

Answer 7

OK, here's an idea. Bear with me on this, it might at first seem unsuitable, but read the bit at the end. Hopefully it works with what you have in place. In the folder containing your files, you place a .htaccess which rewrites all requests to a PHP handler script in the same directory, something like this (untested):

RewriteEngine On
RewriteRule ^/handler.php$ - [L]
RewriteRule ^/([a-zA-Z0-9\.]+)$ /handler.php?filename=$1 [L]

In the PHP script, you do whatever logging is necessary using file_put_contents(). Then, you create handler.php with this code:

<?php
if (!file_exists) {
    header("Status: 404 Not Found");
    //if you have a 404 error page, you can use an include here to show it
    exit(0);
}

header("Content-disposition: attachment; filename={$_GET["filename"]}");
header("Content-type: ".get_mime_type($_GET["filename"]));
readfile($filename);

function get_mime_type($filename, $mimePath = '/etc') {
    $fileext = substr(strrchr($filename, '.'), 1);
    if (empty($fileext)) return (false);
    $regex = "/^([\w\+\-\.\/]+)\s+(\w+\s)*($fileext\s)/i";
    $lines = file("$mimePath/mime.types");
    foreach($lines as $line) {
        if (substr($line, 0, 1) == '#') continue; // skip comments
        $line = rtrim($line) . " ";
        if (!preg_match($regex, $line, $matches)) continue; // no match to the extension
        return ($matches[1]);
    }
    return (false); // no match at all
}
?>

Basically, you are creating a layer between the file request and the actual serving of the file. This PHP layer logs the file access, then serves the file. You said you didn't want to mess around with status codes and MIME types, but the beauty of this is that all that is taken care of. In case the file doesn't exist, it just generates a standard 404, and you can include a custom 404 error page. Yes, the status header is being changed here, but it's nothing complicated. As to MIME types, they are detected for you according to the same MIME type rules Apache uses. Point the get_mime_type function to the mime.types file on your server. If you don't know where it is, just download a copy from here. I'll admit, this solution is probably more technical than you were looking for, but with the restrictions you have it's a good solution. The best part is, it's completely transparent to the end user, as well as those who upload stuff.

Answer 8

The only unobtrusive monitoring you could do without filtering stuff through PHP would be checking up on all files and noting down their file access times each time any PHP file is requested (you just add a function to your php files or use a rewrite). It'll incur a little overhead but it's the only unobtrusive statistic you can get.

Obviously, this way you can't get exact numbers of accesses but more like frequencies so it's some kind of (viable) statistic too. To get something like hit numbers (this was opened 1000k times on march 25th at 2am) you need to have access to logs or pipe it all through a PHP or cgi script -- something just has to do the manual counting.

Answer 9

Assuming you're using PHP as a compiled Apache module then the virtual() function could make this happen. See: http://www.php.net/manual/en/function.virtual.php

<?php

$fn = $_GET['fn'];

log_file_access($fn); // You define how you want this to happen    
virtual($fn);

You then reference the files via:

http://example.com/file.php?fn=files/lolcat.jpg

Answer 10

I've tried a great many things and there seems to be no easy solution.

My solution uses the Location header trick proposed by @yes123 but I've tweaked it to match my preferences.

The links to the files are kept intact, so it's still: /files/path/to/my/file.abc I have a RewriteRule:

RewriteRule ^files/(.*) path/to/tracker.php?path=/$1

Then in the file I issue a Location header by adding ?track=no to the URL and an exception to the earlier RewriteRule:

RewriteCond %{QUERY_STRING} !(&|^)track=no(&|$)

I've added one more optimization. I've enabled E-Tags so if the client send an E-Tag header, see if it matches the file and return a 304 Not Modified instead of a Location.

$fs = stat($document_root . $path);
$apache_etag = calculate_apache_etag($fs);
if ((isset($_SERVER["HTTP_IF_MATCH"]) && etag_within_range($_SERVER["HTTP_IF_MATCH"], $apache_etag))
    || (isset($_SERVER["HTTP_IF_NONE_MATCH"]) && etag_within_range($_SERVER["HTTP_IF_NONE_MATCH"], $apache_etag))
) {
    header("ETag: " . $apache_etag, true, 304);
    exit;
}

function etag_within_range($etag1, $etag2) {
    list($size1, $mtime1) = explode("-", $etag1);
    list($size2, $mtime2) = explode("-", $etag2);
    $mtime1 = floor(hexdec($mtime1) / 1000000);
    $mtime2 = floor(hexdec($mtime2) / 1000000);
    return $mtime1 === $mtime2 && $size1 === $size2;
}

And implementation for calculate_apache_etag can be found here: How do you make an etag that matches Apache?

etag_withing_range solves the issue of comparing against a higher precision mtime in Apache.

---

Notes on solutions that didn't work

virtual

Test script:

var_dump(apache_response_headers());
virtual("/path/to/image.jpg");
var_dump(apache_response_headers());

Outputs:

array(1) { ["X-Powered-By"]=> string(10) "PHP/5.2.11" }
[[binary junk]]
array(5) { ["X-Powered-By"]=> string(10) "PHP/5.2.11" ["Keep-Alive"]=> string(18) "timeout=5, max=100" ["Connection"]=> string(10) "Keep-Alive" ["Transfer-Encoding"]=> string(7) "chunked" ["Content-Type"]=> string(9) "text/html" }

Content-Type: text/html reaaaaalllly? :(

Perhaps PHP5.3's header_remove function can solve this? I haven't tried.