Linux: copy one file from several directories

Question

I have a situation where I want to copy a specific file "transaction.log" from thousands of directories under linux and build a tar file containing those same specific files.

Example: I have thousands of directories under /user/foo/dirs/

/user/foo/dirs/dir1
/user/foo/dirs/dir2
/user/foo/dirs/dir3
..
..
..
/user/foo/dirs/dir50000

In each directories, there are several files and a transaction.log file. I would like to copy this transaction.log file from all the 50000 directories and store them in a tar file.

Can you please help me if there is a way to do so?

Thanks.

SOLUTION:

Okies, i found the issue, since there were some sympolic links to other disks find was not working well. solution is to use with -follow option with find to follow the symbolic links. thanks. I use the following command

find . -follow -type f -name "transaction.log" | tar --create --files-from - > /foo/Stats_transaction_Object.tar.gz

You can use wildcards. $ tar -cvf tarname.tar /user/foo/dirs/dir*/transaction.log — pce, Aug 27 '12 at 21:08

score 2 · Answer 1 · answered Aug 27 '12 at 22:23

2

The one with {} isn't bad, but it'll run a separate tar process for each file. This is a single tar process, so it's quite a bit faster:

find /where/ever -name transaction.log -print | tar --create --files-from - > /somewhere/else/foo.tar

answered Aug 27 '12 at 22:23

dstromberg

6,954
1
26
27

-print is the default and it is probably more efficient to have tar create the archive rather than have it be done from the shell redirection... – njsf Aug 27 '12 at 22:37
Thank you dstromberg. I tried your advice but the tar file got created has only 7747 directories containing 'transaction.log' file. and I have about 50K directories which contain this file. I noticed with teppic suggestion it was 7743 files. Is there some limitation on linux for doing do? – user981116 Aug 28 '12 at 11:42
-print is the default on some find's, especially GNU find. On other find's there is no default action. – dstromberg Aug 28 '12 at 17:19
I would try find /where/ever -name transaction.log -print | wc -l – dstromberg Aug 28 '12 at 17:19

score 1 · Answer 2 · answered Aug 27 '12 at 22:29

1

You can use this commands:

cd /user/foo/dirs/
find . -type f -name "transaction.log" | xargs tar -zcvf transaction_logs_backup.tar.gz

answered Aug 27 '12 at 22:29

TOC

4,326
18
21

If the length of the listing of transaction.log files exceeds xargs limit for one command invocation (usually 4K) you will have an incomplete archive. – njsf Aug 27 '12 at 22:39

score 0 · Answer 3 · answered Aug 27 '12 at 21:52

0

Assuming you want to store the directory path as well:

find /user/foo/dirs/ -name 'transaction.log' -exec tar rf /tmp/stuff.tar {} \;

This will store every transaction.log file found within that directory structure in /tmp/stuff.tar.

answered Aug 27 '12 at 21:52

teppic

8,039
2
24
37

Thank you. I tried your advice but the tar file got created has only 7743 directories containing 'transaction.log' file. and I have about 50K directories which contain this file. Is there some limitation on linux for doing do? – user981116 Aug 28 '12 at 11:41
Okies, i found the issue, since there were some sympolic links to other disks find was not working well. solution is to use with -follow option with find to follow the symbolic links. thanks. – user981116 Aug 28 '12 at 14:51

score 0 · Answer 4 · answered Aug 27 '12 at 22:37

0

find /user/foo/dirs -name transaction.log | tar -cv -T- -f /tmp/stuff.tar

answered Aug 27 '12 at 22:37

njsf

2,729
1
21
17

Linux: copy one file from several directories

SOLUTION:

4 Answers4