1

I have a large amount of existing data stored in files on a NAS (stored in hierarchical date-based directories, not that I think that matters). These directories are still being written to.

I have created a new syslog-ng file() source to process this data. However, when I start up the syslog-ng service, it starts processing the data from the beginning (going back several years). It will take far to long to process all that data, and it's useless anyway.

I want syslog-ng to only process new data that is added to the directory on the NAS. To set the pointer stored in syslog-ng.persist directly to the end of the files, as it were. Is this possible?

Marc Harris
  • 11
  • 2
  • What version of syslog-ng are you running? – mghocke Jun 14 '12 at 17:04
  • 4.0.1a (premium) – Marc Harris Jun 14 '12 at 20:41
  • Ah, their closed source product. AFAIK it is based on their open sourced project and I assume the file format of syslog-ng.persist doesn't change too often. Just to check, could you run the 'strings' command on that file and tell us the magic string at the beginning of the file? It should say something like "SLP3" (pre 3.2 OSE) or "SLP4" (3.2 and above, OSE). – mghocke Jun 14 '12 at 21:10
  • The first line was "SLP4a" without the quotes – Marc Harris Jun 15 '12 at 16:06

1 Answers1

1

There are no official tools out there that allow you to edit the file location pointer within the syslog-ng.persist file. A quick look at the source code of syslog-ng shows, though, that it shouldn't be too hard to write your own tool to get this done. It takes a bit more effort if this is syslog-ng 3.1 or lower because most code has to be copied and pasted from the source distribution. With 3.2 and above the important code comes in a shared library and corresponding header file that can be used by the tool.

mghocke
  • 796
  • 4
  • 5