2

I have a script where I'm fetching the logs from the tomcat and sending that into the my cloud resource. Everything works well, but I have a problem when my tomcat rotates the log.

When the logs get rotated its been prefixed with date ( log gets rotated every day ). Since my script just runs every half an hour I may miss the logs when it gets rotated, because I'm fetching the logs with their static name, in the example logfile.log.

Before getting rotated the file will look like this :

logfile.log

After getting rotated, it will look like this :

logfile.log.2012-10-09

Are there any ways to get rid of this problem?

Edit:

My script :

cp /tomcat/logs/$logname $fileName 
gzip $fileName
s3cmd put $fileName.gz s3://x.x.x.x.x/$folderName

Thanks in advance.

Community
  • 1
  • 1
sriram
  • 8,562
  • 19
  • 63
  • 82
  • You should provide much more details about your script and the log files naming conventions. – weynhamz Oct 12 '12 at 13:22
  • Can we see your script? Why don't you iterate over all files that are named `logfile.log*`? – Niklas Hansson Oct 12 '12 at 13:28
  • @NiklasHansson: That has a problem there will many `logfile.log.*`. What I meant is `logfile.log.previous_dates` is also included which is not I'm looking for. – sriram Oct 12 '12 at 13:31
  • @TechliveZheng: It will stay intact. – sriram Oct 12 '12 at 13:31
  • @GroovyUser I can't help you without an extract from your script. You can't provide the crucial parts? – Niklas Hansson Oct 12 '12 at 13:34
  • So every 30 minutes you copy a log file to another server? Why should that be a problem? Just do that every 30 minutes and then, when the file gets rotated, copy that one once per day. Just make sure that the copy happens after the rotation (or as part of the rotation if you are using something like logrotate). – Christopher Schultz Oct 12 '12 at 18:06
  • @ChristopherSchultz: Can you just this an algorithm of what your saying. I'm not getting your answer properly. – sriram Oct 13 '12 at 05:38

1 Answers1

4

I think the best way to backup you logs is to do a check according to the mtime of the logfiles.

You can keep the log file mtime of the last backup somewhere, then check both rotated log files and current log file. If there is a rotated log file that newer then the last mtime stored, you could append the current log file to the rotated one and then backup. If only current log file is newer, then just backup it.

The mtime of the file could be retrieved by: LC_ALL=C stat logfile.log | grep '^Modify' | cut -d: -f2-, or the unix timestamp by date "+%s" --date="$(LC_ALL=C stat logfile.log | grep '^Modify' | cut -d: -f2-)"

weynhamz
  • 1,968
  • 18
  • 18