2

I need to copy a 700G ISCSI mounted folder to a folder on the local hard drive, while maintaining the existing permissions and ownership. The data in the folder consists of about a million mostly small files, and the new directory needs to exactly match the permissions and ownership of the original.

Using rsync -az /original_folder/ /new_folder gets all the permissions correct, but it has been over an hour "scanning for files", and has not even started coping over yet. The data is openvz disk shares, and there have been permission problems in the past when they were copied over using "cp".

Is there a faster way to copy a huge number of small files while exactly preserving the permissions and ownership? Maybe some rsync flags that can avoid the initial scan? Ideally a tool that can pick up where you left off if the process is interrupted.

John P
  • 1,679
  • 6
  • 38
  • 59
  • not that you can change it at this moment, but i remember that moving from ext3 to xfs reduced the 'scanning for files' time on a 6TB volume from 23 minutes to less than 15 secs. (now it's at 17TB, and still less than 30secs for scanning) – Javier Jul 15 '11 at 11:29

4 Answers4

3

Have you considered using tar? Something along the lines of tar -cpOC <sourcedir> . | tar -xpC <targetdir> -f - might do you.

Andreas Turriff
  • 31
  • 1
1

What version are rsync are you using? Version 3 handles small files quite a bit better. You can also use -P to get a sense for how far along you are.

As for recovering from partials.. hrm. You could go into your root dir and run something like..

for i in * ; do rsync -azP $i /newfolder ; done

then skip any subdirs that you've already copied

MrTuttle
  • 1,176
  • 5
  • 5
1

In this situation you should probably try to duplicate the volume rather than trying to duplicate the individual files.

Does your iSCSI target have a way to clone the volume?

Jodie C
  • 743
  • 6
  • 9
0

cp -aux (as root) should do the trick.

Pablo Castellazzi
  • 121
  • 2