1

We have a particular sub-platform with around 900 web-facing RHEL VMs (on ESXi 4U2 if that helps). Each VM runs a custom application that logs to a per-VM file, we don't want these logs to get very big as we want the VMs to be small. What we've been doing to far is to get each VM to put it's file in a shared NFS mount, each named after the VM.

This has worked a treat, until yesterday when we lost half a shelf in the NetApp box running the NFS server - although it's all RAID'ed and has dual controllers we needed to switch the whole system off to swap the shelf (one aggregate has multiple members per shelf). At which point the VMs can't write their logs and we lost all this logging data.

So to the actual question, what I'd like to do is have the VMs each write to a file local to themselves (i.e. within the .vmdk), then have a process that updates the central NFS copy with any updates to this local file and clears the local file, if the NFS is down this doesn't happen and the local file gets bigger until the NFS is back, at which point it updates the central copy and then clears the local copy. Does that make sense?

Either way I've not been able to find anything like this, I'm pretty sure that rsync can't, have you any ideas?

Oh and we're having the application rewritten to use syslog but it'll take 3-6 months apparently :(

Thank you.

Chopper3
  • 101,299
  • 9
  • 108
  • 239
  • 3 to 6 months? Excuse me? All it should take is changing a `fwrite(3)` call to a `syslog(2)` call, along with associated changes which should take less than 2 weeks. You're getting taken. – Ignacio Vazquez-Abrams Jul 08 '10 at 10:21
  • I know, I know - the company who writes this application have no previous experience with syslog and will want to heavily test it, plus by dragging it out it'll cost us more - I've accepted that, it's the price you pay when one corporate customer buys from another corporate supplier - sad isn't it but a fact of life for me :( – Chopper3 Jul 08 '10 at 10:41

1 Answers1

1

Could this simpy be served with a prerotate or postrotate script in logrotate?

Wrikken
  • 981
  • 9
  • 22