Sending multiple loglines (mysql-slow.log, php5-fpm.log.slow) to syslog

Question

I'm trying to aggregate several sources of error messages into one place, to easier put things into context. I'm currently looking at rsyslog, but I can change this - if it's preventing me from doing what I want to do.

My problem is, how can I best deal with the formatting of mysql-slow.log and php5-fpm.log.slow?

An example entry from php5-fpm.log.slow would be:

Oct 16 17:16:39.820707 [pool www] pid 13060
script_filename = /var/www/somesite/index.php
[0x0000000002f846c8] _path() /var/www/somesite/lib/Zend/Cache/Backend/File.php:895
[0x0000000002f844b0] _path() /var/www/somesite/lib/Zend/Cache/Backend/File.php:579
[0x0000000002f84210] _metadatasFile() /var/www/somesite/lib/Zend/Cache/Backend/File.php:545
[0x0000000002f83fb8] _loadMetadatas() /var/www/somesite/lib/Zend/Cache/Backend/File.php:479
[0x0000000002f830d0] _getMetadatas() /var/www/somesite/lib/Zend/Cache/Backend/File.php:750
[... Stack trace can be long, usually less than 30 lines  ...]

And an example entry from mysql-slow.log would be:

# Time: 111017  7:20:17
# User@Host: someusername[someusername] @  [127.0.0.1]
# Thread_id: 183761313  Schema: somesitedb  Last_errno: 0  Killed: 0
# Query_time: 13.705725  Lock_time: 0.000146  Rows_sent: 0  Rows_examined: 8315267  Rows_affected: 0  Rows_read: 18446744073709551491
# Bytes_sent: 1019  Tmp_tables: 0  Tmp_disk_tables: 0  Tmp_table_sizes: 0
# InnoDB_trx_id: B4BB5B67
use somesitedb;
SET timestamp=1318828817;
SELECT * FROM table;

I want to send one syslog message for each of these entries - and keep the timestamp intact, so it's easier to contextualize what is going on. How may I best achieve this?

Look into http://blog.gerhards.net/2013/09/imfile-multi-line-messages.html for parsing multi-line messages with rsyslog. — Gaia, Aug 12 '14 at 21:38

score 1 · Accepted Answer · answered Oct 24 '11 at 18:25

I researched this subject myself, and stumbled upon a really cool tool named Logstash. It can be easily configured to parse multiline entries - and it does it in an incredibly clean way.

input {
  file {
    type => 'mysql-slow'
    path => '/var/log/mysql/mysql-slow.log'    
  }
  file {
    type => 'php5-fpm-slow'
    path => '/var/log/php5-fpm.log.slow'
  }
}

filter {
  multiline {
    type => 'mysql-slow'
    pattern => '^# Time: '
    negate => true
    what => 'previous'
  }
  multiline {
    type => 'php5-fpm-slow'
    pattern => '^$'
    negate => true
    what => 'previous'
  }
}

I use Loggly and Papertrail. Is the syntax above specific to Logstash config files? — Gaia, Aug 12 '14 at 21:10

Sending multiple loglines (mysql-slow.log, php5-fpm.log.slow) to syslog

1 Answers1