mysqldump output to log4perl

Question

I have a Perl script that uses log4perl to log everything to file. The config for log4perl is stored in an external file.

This script runs the mysqldump command using qx. The output of the mysqldump command is printed to screen (i assume STDERR), but not to log file.

I would like to direct the output of mysqldump (and any other stmt that prints to screen) to log4perl log file.

I am not sure if this involves using something other than qx, or changing the log4perl config.

Note: i have seen some online examples of this, but they all use log4perl easy-config.

Here is the log4perl config file -

log4perl.rootLogger              = DEBUG, SCREEN, file
log4perl.appender.SCREEN         = Log::Log4perl::Appender::Screen
log4perl.appender.SCREEN.stderr  = 0
log4perl.appender.SCREEN.layout  = Log::Log4perl::Layout::PatternLayout
log4perl.appender.SCREEN.layout.ConversionPattern = %d %r %p %F{1} %M{1}.%L - %m%n
log4perl.appender.file          = Log::Log4perl::Appender::File
log4perl.appender.file.filename = log/mbackup.log
log4perl.appender.file.mode     = append
log4perl.appender.file.layout   = Log::Log4perl::Layout::PatternLayout
log4perl.appender.file.layout.ConversionPattern = %d %r %p{5} %F{1} %M{1}.%L - %m%n

Here is the perl script -

use strict;
use warnings 'FATAL' => 'all';
use Sys::Hostname;
use Log::Log4perl;

my $log_conf = "log4perl.conf";
Log::Log4perl::init($log_conf);
my $logp        = Log::Log4perl->get_logger();
my $dirTemp     = "temp";
my $fileTemp    = "temp_backup.gz";
my $dbSchema = 'test';
my $dbToken01 = 'root';
my $dbToken02 = 'haha';

sub create_backup {
    $logp->info("creating temp mysqldump...");
    my $backupCmd = "mysqldump -R --triggers -u $dbToken01 -p$dbToken02 -v --quick --single-transaction $dbSchema | gzip > $dirTemp/$fileTemp";
    qx($backupCmd);
}

#main
$logp->info("start backup script");
eval {
    create_backup();
};
if ($@) {
    $logp->error( 'error: ', $@ );
}
$logp->info("end backup script");

Here is the script output -

[root@localhost perl]# perl test.pl
2013/04/24 16:35:33 3 INFO test.pl main::.22 - start backup script
2013/04/24 16:35:33 4 INFO test.pl create_backup.16 - creating temp mysqldump...
-- Connecting to localhost...
-- Disconnecting from localhost...
2013/04/24 16:35:33 22 INFO test.pl main::.29 - end backup script

Clarification - When i said output, i meant the stderr output of the mysqldump command. I want all the text output of the mysqldump command, that is written on the screen, to be sent to the log4perl log file.

What output do you expect to see? You're piping all output from the `mysqldump` to `gzip > somefile`. — chrsblck, Apr 24 '13 at 21:59
For clarification, that's what my solution does. Captures all output and logs it. — chrsblck, Apr 25 '13 at 04:53

chrsblck · Accepted Answer · 2013-04-25T20:26:16.147

1

If you want to do both: Log the mysqldump and gzip you can do something like this:

By modifying sub create_backup

sub create_backup {
    my $sqlDumpFile = "$dirTemp/$fileTemp";
    $logp->info("creating temp mysqldump...");

    my $backupCmd = "mysqldump -R --triggers -u $dbToken01 -p$dbToken02 -v --quick --single-transaction $dbSchema 2>&1 | tee $sqlDumpFile";
    my $dumpOutput = `$backupCmd`;
    $logp->info($dumpOutput);
    `gzip $sqlDumpFile`; # note you will have renamed this file to "filename".gz
}

Update

If you want to only log Error then you can do something like this.

IPC::Run can make your life easier here.

sub create_backup {
    require IPC::Run qw(run); # or load this at the top of you script...
    my ($in, $out, $err);
    my $backupCmd = "mysqldump";
    my $sqlDumpFile = "$dirTemp/$fileTemp";

    $logp->info("creating temp mysqldump...");

    run([$backupCmd, '-R', '--triggers', '-u', $dbToken01, '-p', $dbToken02, 
      '-v', '--quick', '--single-transaction', $dbSchema, '>', $sqlDumpFile],
      \$in, \$out, \$err);

    if( $err ) {
        $logp->info($err);
        # do other stuff?
        # Eg. `unlink($sqlDumpFile);            
    } else {
        `gzip $sqlDumpFile`; # note you will have renamed this file to "filename".gz
    }
}

edited Apr 25 '13 at 20:26

answered Apr 24 '13 at 22:25

chrsblck

3,948
2
17
20

1

I think he only want's stderr to go to the log file. – jordanm Apr 25 '13 at 02:13
@jordanm The OP mentioned wanting all output. "I would like to direct the output pf mysqldump (and any other stmt that prints to screen) to log4perl log file." I can see wanting STDERR but not STDOUT. – chrsblck Apr 25 '13 at 02:41
apologize for the confusion. I want the stderr/stdout/any other stds from the mysqldump command to go to the log4perl log file. – Quest Monger Apr 25 '13 at 04:17
@QuestMonger Right, that's what the above answer does. – chrsblck Apr 25 '13 at 05:05
you're right. but i guess this creates new problems. mysqldump dumps all the DB info to stdout, which is then gzipped. by doing above, i am redirecting stderr to that file. this will cause the backup file to be corrupted with stmts like 'Connecting to localhost'. this renders the backup file useless (unless someone deletes those stderr lines from the backup file and then uses it for restoring DB back to MySQL). So i guess i need a solution that print stderr to screen and log, and not to the actual backup file which is gzipped. – Quest Monger Apr 25 '13 at 19:08
1

@QuestMonger Ok, Updated Answer to handle your new case. – chrsblck Apr 25 '13 at 20:26
That worked. Had to use [IPC:Run3](http://search.cpan.org/~rjbs/IPC-Run3-0.045/lib/IPC/Run3.pm) though, since Run is not [compatible](http://code.activestate.com/ppm/IPC-Run/) with new versions of Perl. Danke. – Quest Monger Apr 25 '13 at 22:45

mysqldump output to log4perl

1 Answers1