How does one manually rotate log without logrotate on open/live file?

Question

My Google-Fu is getting me so close but just not quite there, and I guess I'm too green in Linux to put the pieces together.

I have a very large >200GB log file, still being written to. Logrotate wont get to it in time before disk space could be a concern. Also, I don't want to fire off another round of logrotate, because I don't want it to effect all the other targets in its config. I have added the new stanza in my logrotate.conf file as such:

/log/myDevice/myDevice.log {
  compress
  daily
  rotate 360
  maxage 360
  missingok
  compresscmd /usr/bin/xz
  dateext
  compressext .xz
  copytruncate
  olddir /log/archive/myDevice
}

I would like to do these things manually:

copytruncate : to grab off the current log file, but allow syslog-ng to keep chugging along on the current open/live file
(not above in the logrotate.conf) split : to break up the current log roll into smaller chunks
xz : compress over to archive folder

I can split(1) and xz(1) just fine, but when I try to grab off the target file, no dice.

I tried sudo mv myDevice.log ZmyDevice.log && touch myDevice.log, but syslog-ng just moves with the original file (and its new name) and keeps writes merrily away. Which made a bit of since when I thought about how the file is being referenced by syslog-ng.

So I'm trying to figure out how to do some sort of a manual copytruncate in order to cut the file to another one "in place" as it were, and let syslog-ng keep chugging away at what it wants.

Yes. I want to split up and archive the current contents inside of my `myDevice.log` file. I thought about making a temporary logrotate.conf file with just this one stanza in it, and running it against the file, but I don't want the compressed file to be that of the original >200GB size. My thoughts were to use `split` to break it up first, then just `xz` them myself into the archive folder. This way the current (or what would be left as syslog-ng would still be writing to it) would get logrotate'ed tomorrow on schedule. — Ceafin, Dec 29 '14 at 18:41

score 2 · Accepted Answer · answered Dec 29 '14 at 18:44

2

Move the file, then send a SIGHUP to the syslog-ng process:

kill -HUP

which will cause it to re-open all it's open files, thus releasing the file you just moved/renamed and allowing you to carry on. It'll then open /log/myDevice/myDevice.log fresh and start writing to that.

answered Dec 29 '14 at 18:44

Craig Miskell

4,216
1
16
16

I read the `kill -HUP` answer in another SF post, could a more verbose explanation of how this works under the covers be provided? It sounds right, but I'd like to learn more about the mechanisms that drive the action. What I know about it is "Many daemons will reload their configuration files and reopen their logfiles instead of exiting when receiving this signal". But is syslog-ng known for coming back normally, or just exiting and staying exited? – Ceafin Dec 29 '14 at 18:51
I don't know in detail how syslog-ng implements SIGHUP processing under the covers, just that it claims to reopen files when it receives it: http://www.balabit.com/sites/default/files/documents/syslog-ng-ose-3.3-guides/en/syslog-ng-ose-v3.3-guide-admin-en/html/configuring_destinations_file.html, specifically "Also, after rotating the log files, reload syslog-ng OSE using the syslog-ng-ctl reload command, or use another method to send a SIGHUP to syslog-ng OSE." – Craig Miskell Dec 29 '14 at 18:58
1

This definitely did the trick! Though for anyone else in my awkward spot, the command I did was: `mv myDevice.log ZmyDevice.log && kill -HUP \`cat /var/run/syslogd.pid\`` – Ceafin Dec 29 '14 at 19:05
how to do this in windows – Santhosh Mar 10 '21 at 06:39

score 0 · Answer 2 · answered Dec 29 '14 at 22:45

Find which processes have the file open: fuser /path/to/file. Then use kill -SIGSTOP to suspend the process(es) that is writing to the file. Then rotate the file, copy it, truncate it, whatever you need to do, then resume the process(es) with kill -SIGCONT.

How does one manually rotate log without logrotate on open/live file?

2 Answers2