-1

I'm currently writing a simple Bash script which finds lines in files which match the target string and outputs this both to the Terminal display but also to a file. The command at the moment is:

grep -r -n -F "myname" /Users/tom/Desktop/* | tee /Users/tom/Desktop/out.txt

This code works perfectly for what I want it to do. However, I also want to have a feature that when the command has finished, it will echo the number of results (lines) that were found to contain the specified string.

What would be the best method of doing this? I understand that using:

grep -r -n -F "myname" /Users/tom/Desktop/* | tee >(wc -l)

would enable me to get the line count whilst still enabling me to view the output in the Terminal, but, it would not allow me to save the output to a file.

Thank you in advance for any help.

Kind regards, Tom

P.S. The script deals with very large files so efficiency is very important.

Tom
  • 159
  • 1
  • 7
  • Why not run cat file | wc -l afterwards? – MrTux Jul 17 '18 at 21:25
  • @MrTux I forgot to mention, due to the size of some of the files the command is scanning, the command can take a number of minutes and doing this would be highly inefficient compared to a grep command – Tom Jul 17 '18 at 21:27
  • I don’t understand why post processing the output file is so expensive... then, why no two tee calls? – MrTux Jul 17 '18 at 21:31
  • Considered `awk` for this? – dawg Jul 17 '18 at 21:31
  • If the result is so big, why are you writing it to a slow terminal? – that other guy Jul 17 '18 at 21:34
  • @awk I haven't, no, what would you have suggested? – Tom Jul 17 '18 at 21:35
  • @thatotherguy The result is not very big, I am outputting to the terminal in order to give the user a sort of live update to what has been found due to the fact that the grep command can take a long time – Tom Jul 17 '18 at 21:38
  • If the result is not very big, then @MrTux's original suggestion also works. – that other guy Jul 17 '18 at 21:47

3 Answers3

1

tee accepts multiple filenames to write to:

grep -r -n -F "myname" /Users/tom/Desktop/* | tee >(wc -l) /Users/tom/Desktop/out.txt

If you want to

  1. Write to a file
  2. Write to the terminal
  3. Capture the resulting number of lines

You can use:

lines=$(grep -rnF "myname" /Users/tom/Desktop/* |
    tee /Users/tom/Desktop/out.txt /dev/stderr |
    wc -l)
that other guy
  • 116,971
  • 11
  • 170
  • 194
  • Thank you for your answer! Works perfectly. What method would you recommend for extracting the final line count into a variable from that command? – Tom Jul 17 '18 at 21:36
  • That was not part of the initial question, but I added a separate suggestion for it – that other guy Jul 17 '18 at 21:44
  • Thank you so much! Works absolutely perfectly. I apologise for the possibly unclear question – Tom Jul 17 '18 at 21:45
0

One option is to run wc -l out.txt after the grep | tee command.

But you can also pipe the output of the grep | tee command to another tee:

grep -r -n -F "myname" /Users/tom/Desktop/* | tee /Users/tom/Desktop/out.txt | tee >(wc -l)
axiac
  • 68,258
  • 9
  • 99
  • 134
  • What's the benefit of using multiple `tee`s? – that other guy Jul 17 '18 at 21:33
  • Thank you for your response, this works for me but I am going to stick with @thatotherguy response as I feel that it is more efficient. Thank you anyway! – Tom Jul 17 '18 at 21:36
  • @thatotherguy There is no benefit. I was writing the answer while you posted yours and I was not aware that `tee` accepts multiple file names until I read it. Learned something today. – axiac Jul 17 '18 at 21:38
0

You might consider using awk since you can match and count at the same time:

$ awk -v tgt='myname' '$0 ~ tgt {cnt++; print} END{print cnt " matches"}' file

If you want to write the matches to a new file and assign the number of matches to a variable:

cnt=$(awk -v tgt='myname' '$0 ~ tgt {cnt++; print>matches} END{print cnt}' file)
dawg
  • 98,345
  • 23
  • 131
  • 206
  • Thank you for your answer, I'll give this a try in a minute, do you know of any differences in speed between grep and awk when it comes to dealing with such large files? – Tom Jul 17 '18 at 21:40
  • `grep` and `awk` are similar in speed. If you have a lot of matches, this will be faster since you won't have `wc` have to reread the matches. – dawg Jul 17 '18 at 21:43
  • Thank you for the information. This works well for me, however, I think I will stick to the solution above as I am more familiar with grep. – Tom Jul 17 '18 at 21:47