I have a remote server that is continuously generating log files and saving them to a directory. Unfortunately, the log files are generated at random times, so you can't predict when the next logfile will be available.
I have a second server (Ubuntu 18.04.6 LTS) that has SSH connectivity to the log-generating machine. For reasons too complex to detail here, I need the second server to automatically copy all those log files to its own local hard drive. The second server will do some processing on the logs, then must delete those locally-saved logs to save on disc space. A logfile only needs to be processed once.
The problem: Server #2 has to periodically log into Server #1 and copy only the logfiles that have been generated since its last check-in. RSYNC could do this in a pinch, except...
...Server #2 deletes the logfiles that it downloaded previously. I believe that Rsync needs those local logfiles to be intact, or else how would it know what new files have been created since its last visit?
Does anyone know of a way to do this with RSYNC? I've looked through the delete options and don't see any obvious solution. Thank you.
