0

I am using syslog-ng to parse some logs that I am receiving via a csv-parser. However, I want to achieve insert operations that are a bit more complex than the conventional insert using the "destination" option in syslog-ng. Currently, my destination into MYSQL from my syslog-ng conf file looks like this: 

destination d_sql_test
{
sql(
type(mysql)
host('<host>') 
username('<user>')
password('<pass>')
database('<db_name>')
table('test')
columns('col1')
values('${val1}')
);
};

However, this simply just inserts the contents of val1 into the column col1. I want to be able to specify my insert "logic" as shown in the example in this question.

I am unsure as to where to actually do this, and if it is even supported by syslog-ng

Community
  • 1
  • 1
bawse
  • 201
  • 3
  • 13
  • This question seems to be more about how to configure `syslog-ng` than programming. – Barmar Dec 10 '15 at 21:06
  • @Barmar I don't think there would be a syslog-ng tag on SO if these kinds of questions weren't allowed. – bawse Dec 10 '15 at 21:12
  • There are tags for lots of off-topic things. This is a legacy from before SE was spread into so many different sites. – Barmar Dec 10 '15 at 21:12
  • Then a suggestion of which site to post my question on would have been more helpful than your initial comment. – bawse Dec 10 '15 at 21:20
  • I voted to close it using the reason that will recommend ServerFault.com. – Barmar Dec 10 '15 at 21:21
  • I just had a look at serverfault.com, there are hardly any syslog-ng questions related to mysql. I think my question is in the correct place. – bawse Dec 10 '15 at 21:26
  • I think this is a perfectly okay question to ask on SO. – bawse Dec 10 '15 at 21:30
  • According to [the documentation](https://www.balabit.com/sites/default/files/documents/syslog-ng-ose-latest-guides/en/syslog-ng-ose-guide-admin/html/sql-interactions-syslog-ng.html) the only thing `syslog-ng` can do automatically is create, alter, and insert into the table. It sounds like you need to pipe to a script that implements your logic. – Barmar Dec 10 '15 at 21:31
  • So to make this on-topic for SO, you should try to write that script. If you can't get it working, post what you tried and we'll help you. – Barmar Dec 10 '15 at 21:32
  • Would this not be achievable with mysql triggers? – bawse Dec 10 '15 at 21:55
  • You've not said what your logic is. You've not given any indication of the options you've considered nor why they do not meet your expectations. – symcbean Dec 11 '15 at 13:12

1 Answers1

1

I think you can do this if you can somehow make the decision within syslog-ng.

  • You could try to use an in-list() filter to check if the username is already listed in a file. If it is not then, you can send the log into the mysql destination, and also to another destination (possibly a program() destination) that updates the file containing the list of users, and reloads the syslog-ng to update the inlist filter.
  • You can write a syslog-ng template-function in Python that implements the logic somehow, and for example sets a macro to 1 in the message if it should be sent to the database. Then you can use a filter for this macro in your log path with the mysql destination.
  • Or if you can write a separate destination that does the work in Python: Writing syslog-ng destinations in Python

Also, you might want to post this question on the syslog-ng mailing list, where the developers notice it more easily.

Robert Fekete
  • 557
  • 3
  • 5