1

I'm trying to accomplish the below with RSYNC but can't quite figure out the syntax. My source contains a folder for each user, then some subfolders under Logs. I need to capture everything under the numeric folder with the sync.

SOURCE DIRECTORY STRUCTURE:  
/STORAGE/user1/db/Logs/151251/  
/STORAGE/user1/db/Logs/156123/  
/STORAGE/user1/db/Logs/117722/  
/STORAGE/user2/db/Logs/178438/  
/STORAGE/user2/db/Logs/161265/  

PREFERRED DESTINATION DIRECTORY STRUCTURE  
/LOGS/user1/151251/  
/LOGS/user1/156123/  
/LOGS/user1/117722/  
/LOGS/user2/178438/  
/LOGS/user2/161265

The command I've tried is:

rsync -azvr user@server.com:/STORAGE/*/db/Logs/*/ /LOGS/

This succeeds in copying all the files from all the numeric subfolders but I need to create the directory structure above (/LOGS/user/uniqueid/individual files).

Is there a way to accomplish this?

Sven
  • 98,649
  • 14
  • 180
  • 226
Jason
  • 381
  • 1
  • 7
  • 20

1 Answers1

3

You can use rsync filters to specify which subpaths you want transferred:

rsync -asvr --include '*/' --include 'Logs/***' --exclude '*' user@server.com:/STORAGE/ /LOGS/

This will create all the source directories on the destination (but not containing any files). To not do this you need to explicitly include all parent directories:

rsync -asvr --include '/STORAGE/' --include '/STORAGE/*/' --include '/STORAGE/*/db/' --include '/STORAGE/*/db/Logs/***' --exclude '*' user@server.com:/STORAGE/ /LOGS/

Edit: To exclude files with a certain extension, add an --exclude pattern at the front:

rsync -asvr --exclude '*.bdb' --include ...
mgorven
  • 30,615
  • 7
  • 79
  • 122
  • Thank you! But will this create the directory structure on the destination server? – Jason Feb 20 '13 at 17:09
  • @Jason Yup.​​​​ – mgorven Feb 20 '13 at 17:11
  • Thanks! I just tried that and there are no files found for sync...? I also tried `--include '*/db/Logs/***'` and that didn't seem to do the trick either? – Jason Feb 20 '13 at 17:17
  • @Jason Ah, the parent directories need to be explicitly included. See my edit for two options. – mgorven Feb 20 '13 at 17:20
  • I really appreciate your help! I'm a bit closer but still not quite there. The command is now syncing the files but it's transferring the source directory structure. So on the destination end, I'm ending up with /LOGS/user1/db/Logs/151251, etc. Any way I can have it transfer to the destination so that it eliminates the db and Logs folders and puts the numeric folders directly under the user folder (/LOGS/user1/151251, etc.)? Thanks again!! – Jason Feb 20 '13 at 17:36
  • @Jason I don't think rsync can do that. You either have to move them afterwards, or sync them individually. – mgorven Feb 20 '13 at 17:39
  • OK - thanks! I can live with the duplicate directory structure. The last thing I'm struggling with is that the Logs folder contains a few files that have a *.bdb extension. I don't want to copy those files so I've created an `--exclude "*.bdb"` but it's still copying them anyway. Am I missing something obvious? :-) Thanks again! – Jason Feb 20 '13 at 17:54
  • @Jason See edit. – mgorven Feb 20 '13 at 17:55