Unbuffered IO in perl

Question

I have a Perl application which writes logs to a file using open and print calls.

open (FH, "d:\\temp.txt");
print FH "Some log";
close (FH);

However during an abrupt shutdown of the machine, the logs are not persisted to the file. So after searching at several places, two options were suggested for doing unbuffered IO (i.e writing the text to the disk instead of maintaining it in cache and then flushing it):

1. sysopen, syswrite
2. $| = 1;

I have tried both these options and it just doesn't work. Any write that I do seconds before the abnormal shutdown gets lost.

Is there any way that I can almost deterministically accomplish unbuffered IO in Perl? I am running Windows 7 64-bit with Perl 5.8.3.

EDIT: I searched for how to have windows perform unbuffered IO and this is how it can be done! Call

1. CreateFile with FILE_FLAG_NO_BUFFERING for the dwFlagsAndAttributes parameter. However, this has memory alignment issues to consider (i.e File Access buffers should be sector aligned; An application determine the sector size by calling GetDiskFreeSpace)
2. Use WriteFile to write data to the file. This write would be unbuffered and instead of going to the cache, it straightaway goes to the disk.
3. Finally, call FlushFileBuffers to flush the metadata associated with the files.

Could someone please assist with the Win32 APIs from Perl for these 3 calls.

Answer 1

use IO::Handle;
open(FH, "d:\\temp.txt");
FH->autoflush(1);
print FH "Some log";
close(FH);

That will get it out to the OS ASAP, but the OS might take a while to commit it to disk. Still I'm sure you'll find this will suit your needs.

If you were on unix, I'd refer to you sync for more info on having the OS commit data to disk.

Answer 2

How about this?

use strict;
use warnings;

use IO::Handle     qw( );  # For autoflush.
use Symbol         qw( gensym );
use Win32API::File qw( CloseHandle CreateFile GetOsFHandle OsFHandleOpen GENERIC_WRITE OPEN_ALWAYS FILE_FLAG_WRITE_THROUGH );
use Win32::API     qw( );

use constant WIN32API_FILE_NULL => [];

sub open_log_handle {
    my ($qfn) = @_;

    my $handle;
    if (!($handle = CreateFile(
        $qfn,
        GENERIC_WRITE,
        0,                        # Exclusive lock.
        WIN32API_FILE_NULL,       # No security descriptor.
        OPEN_ALWAYS,              # Create if doesn't exist.
        FILE_FLAG_WRITE_THROUGH,  # Flush writes immediately.
        WIN32API_FILE_NULL,       # No prototype.
    ))) {
        return undef;
    }

    my $fh = gensym();
    if (!OsFHandleOpen($fh, $handle, 'wa')) {
        my $e = $^E;
        CloseHandle($handle);
        $^E = $e;
        return undef;
    }

    $fh->autoflush(1);

    return $fh;
}

sub close_log_handle {
    my ($fh) = @_;

    my $handle = GetOsFHandle($fh)
        or return undef;

    if (!FlushFileBuffers($handle)) {
        my $e = $^E;
        close($fh);
        $^E = $e;
        return undef;
    }

    return close($fh);
}

my $FlushFileBuffers = Win32::API->new('kernel32.dll', 'FlushFileBuffers', 'N', 'N')
    or die $^E;

sub FlushFileBuffers {
    my ($handle) = @_;
    return $FlushFileBuffers->Call($handle);
}

{
    my $fh = open_log_handle('log.txt')
        or die $^E;

    print($fh "log!\n")
        or die $^E;

    close_log_handle($fh)
        or die $^E;
}

Answer 3

The best you can do is sysopen with the O_SYNC fcntl flag, or fsync() from File::Sync; the options you were given insure that data aren't buffered inside your program but do nothing about whether the kernel is buffering writes (which it does because constantly flushing the same block to disk slows down all other I/O). And even then you might lose, because some hard drives will lie to the OS and claim that data has been committed to media when it's actually still in an on-drive memory buffer.