3

It seems like I am unable to find a direct answer to this question. I appreciate your help.

I'm trying to find all files with a specific name in a directory, read the last 1000 lines of the file and copy it in to a new file in the same directory. As an example:

Find all files names xyz.log in the current directory, copy the last 1000 lines to file abc.log (which doesn't exist).

I tried to use the following command with no luck:

find . -name "xyz.log" -execdir tail -1000 {} > abc.log \;

The problem I'm having is that for all the files in the current directory, they all write to abc.log in the CURRENT directory and not in the directory where xyz.log resides. Clearly the find with execdir is first executed and then the output is redirected to abc.log.

Can you guys suggest a way to fix this? I appreciate any information/help.

EDIT- I tried find . -name "xyz.log" -execdir sh -c "tail -1000 {} > abc.log" \; as suggested by some of the friends, but it gives me this error: sh: ./tail: No such file or directory error message. Do you guys have any idea what the problem is?

Luckily the solution to use -printf is working fine.

phuclv
  • 37,963
  • 15
  • 156
  • 475
Amir
  • 421
  • 1
  • 4
  • 14

3 Answers3

5

The simplest way is this:

find . -name "xyz.log" -execdir sh -c 'tail -1000 "{}" >abc.log' \;

A more flexible alternative is to first print out the commands and then execute them all with sh:

find . -name "xyz.log" -printf 'tail -1000 "%p" >"%h/abc.log"\n' | sh

You can remove the | sh from the end when you're trying it out/debugging.

There is a bug in some versions of findutils (4.2 and 4.3, though it was fixed in some 4.2.x and 4.3.x versions) that cause execdir arguments that contain {} to be prefixed with ./ (instead of the prefix being applied only to {} it is applied to the whole quoted string). To work around this you can use:

find . -name "xyz.log" -execdir sh -c 'tail -1000 "$1" >abc.log' sh {} \;

sh -c 'script' arg0 arg1 runs the sh script with arg0, arg1, etc. passed to it. By convention, arg0 is the name of the executable (here, "sh"). From the script you can access the arguments using $0 (corresponding to "sh"), $1 (corresponding to find's expansion of {}), etc.

ctn
  • 2,887
  • 13
  • 23
  • 1
    I'll try the second one. I previously tried sh -c but it gives this error for every occurence of the file in the directory. Any idea what the issue is? sh: ./tail: No such file or directory – Amir Jun 21 '13 at 21:55
  • By the way, I used the -printf and it worked perfectly. Thanks a lot for that. Still I'm curious to know why the other one is not working. Appreciate any comment. Thanks again. – Amir Jun 21 '13 at 22:15
  • 1
    I have no idea... try replacing `tail` with `/usr/bin/tail` in the first example. – ctn Jun 21 '13 at 22:41
  • Also, feel free to upvote answers (and/or accept the answer) that helped you :D – ctn Jun 21 '13 at 22:42
  • Or perhaps replace the `sh -c` with `/bin/sh -c` – ctn Jun 21 '13 at 22:49
  • Good points. Although they didn't work for me. I'm still looking forward to knowing what the issue is with sh -c, that's why I'll keep it open for now. Tried upvoting your helpful answer but sounds like being a new user I don't have enough reputation to do that :( – Amir Jun 21 '13 at 23:00
  • @user2510104 Try the following commands and report their output: 1. ``ls -l `which sh` `` 2. `sh -c "which tail"` 3. `echo asdf | sh -c tail` – ctn Jun 25 '13 at 11:44
  • Here are the outputs: 1. /bin/sh -> bash 2. /usr/bin/tail 3. asdf – Amir Jun 25 '13 at 23:53
  • @user2510104 `find . -name "xyz" -execdir sh -c "which tail" \;` – ctn Jun 26 '13 at 10:20
  • @user2510104 Are you running the find command from a normal terminal session (the one from which you ran the 3 commands from the previous comment)? – ctn Jun 26 '13 at 10:21
  • Yes, I run them in the same normal terminal session. When I typed in what you mentioned here, for each occurence of the file it prints /usr/bin/tail. I also tried other commands like echo or cat with redirection inside the qoutation, with no luck. It consistently (for each occurence of the found file) gives me the error message that "sh: ./echo: No such file or directory". BTW, thanks a lot for your help. – Amir Jun 26 '13 at 15:34
  • @user2510104 Another try: `find . -name "xyz.log" -execdir sh -c "tail -1000 \{} >abc.log" \;` – ctn Jun 26 '13 at 16:32
  • @user2510104 Or: `find . -name "xyz.log" -execdir sh -c 'tail -1000 {} >abc.log' \;` – ctn Jun 26 '13 at 16:33
  • @user2510104 I find your problem very intriguing. I am very curious what could be wrong. I have another idea :) `find . -name "xyz.log" -execdir echo "tail -1000 {} >abc.log" \;` – ctn Jun 26 '13 at 18:21
  • I tried all three. The first two give the same error message (sh: ./tail: No such file or directory). The third one prints the following (./tail -1000 xyz.log >abc.log). – Amir Jul 01 '13 at 20:36
  • This just crossed my mind. I previously had an issue with -execdir as it was giving me the error message that . is in my path. I took that out from the path and then I could use find with -execdir. Could this in any case be causing the issue? – Amir Jul 01 '13 at 20:39
  • @user2510104 From the behavior it seems this is definitely caused by the value of PATH somehow. It actually looks like PATH is set to only ".". What is the output of `echo $PATH` and `find . -name "xyz.log" -execdir echo '$PATH' \;` ? – ctn Jul 02 '13 at 11:40
  • @user2510104 It's not the path. I'm pretty convinced your implementation of find is bugged. It's supposed to prefix `{}` with `./` but only where `{}` occurs. It seems to prefix the entire quoted string instead. That previous echo should print `tail -1000 ./xyz.log >abc.log` not `./tail -1000 xyz.log >abc.log`. I am unable to reproduce this on my machine (findutils 4.5.11). See if you can upgrade your version of find (package `findutils`). – ctn Jul 02 '13 at 15:25
  • More precisely, it appears to be this bug: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=385896 – ctn Jul 02 '13 at 15:27
  • You nailed it. Thanks a lot. For the benefit of me and other viewers can you please briefly explain what the following is actually doing? Thanks a lot again for all the help. 'tail -1000 "$1" >abc.log' sh {} \; – Amir Jul 13 '13 at 01:00
  • Thanks a lot, buddy. Your 2nd answer worked for a similar need of mine. I wanted to strip (throught head) the 1st K of several files to a file of the same name (for each file) in another dir. -printf+|sh worked like a charm for the task. – Charles Roberto Canato Aug 01 '14 at 03:47
0

The redirect isn't passed into execdir, so abc.log shows up in the directory you run the command in. -execdir also doesn't like embedded redirects. but you can workaround the problem by passing -execdir a shell command with a redirect embedded, like this:

find . -name "xyz.log" -execdir sh -c '/usr/bin/tail -1000 {} > abc.log' \;

Much credit to this blog post (not mine):

http://www.microhowto.info/howto/act_on_all_files_in_a_directory_tree_using_find.html

Edit

I put the full path to tail in the command (assuming it's in /usr/bin on your system), since sh may load a .profile with a PATH that differs from your current shell.

lreeder
  • 12,047
  • 2
  • 56
  • 65
  • Thanks for the info. That't unfortunately not working for. Gives me the following error msg: sh: ./tail: No such file or directory. It works fine if I use -exec instead of -execdir but that doesn't serve my purpose. Any idea why -execdir is not working with that? – Amir Jun 21 '13 at 22:03
  • That's strange. It may be that sh is loading a profile which doesn't contain a path to tail. I updated my answer with the full path to tail. Hopefully this will fix your problem and is more secure anyway. – lreeder Jun 23 '13 at 15:19
  • Unfortunately that somehow didn't work out for me. Still have the same issue. Not sure what it could be. – Amir Jun 26 '13 at 01:35
0

Here's another non-find (well, sorta - it still uses find but doesn't try to shoehorn find into doing the whole thing):

while read f
do
  d=$(dirname "${f}")
  tail -n 1000 "${f}" > "${d}/abc.log"
done < <(find . -type f -name xyz.log -print)
twalberg
  • 59,951
  • 11
  • 89
  • 84
  • This fails for filenames containing newline. Use read -r -d "" and -print0 instead. Also, this is subject to a potentially exploitable race condition if there are untrusted local users -- see http://www.gnu.org/software/findutils/manual/html_node/find_html/Security-Considerations-for-find.html – Demi Sep 24 '13 at 12:38