0

Hello bash beginner question. I want to look through multiple files, find the lines that contain a search term, count the number of unique lines in this list and then print into a tex file:

  1. the input file name
  2. the search term used
  3. the count of unique lines

so an example output line for file 'Firstpredictoroutput.txt' using search term 'Stop_gained' where there are 10 unique lines in the file would be:

Firstpredictoroutput.txt Stop_gained 10

I can get the unique count for a single file using: 

grep 'Search_term' inputfile.txt | uniq -c | wc -l | >>output.txt

But I don't know enough yet about implementing loops in pipelines using bash. All my inputfiles end with *predictoroutput.txt

Any help is greatly appreciated.

Thanks in advance,

Rubal

user964689
  • 812
  • 7
  • 20
  • 40

3 Answers3

0

You can write a function called fun, and call the fun with two arguments: filename and pattern

$ fun() { echo "$1 $2 `grep -c $2 $1`"; }
$ fun input.txt Stop_gained
input.txt Stop_gained 2
kev
  • 155,172
  • 47
  • 273
  • 272
  • thanks that one works well. It would be great to have it in a loop, to avoid having to write out each input file and search term. But its workable now if I write out a list of all the possible combinations. – user964689 Mar 03 '12 at 14:53
0

You can use find :

find . -type f -exec sh -c "grep 'Search_term' {} | uniq -c | wc -l >> output.txt" \;

Although you can have issue with weird filenames. You can add more options to find, for example to treat only '.txt' files :

find . -type f -name "*.txt" -exec sh -c "grep 'Search_term' {} | uniq -c | wc -l >> output.txt" \;
Pluc
  • 901
  • 8
  • 21
  • Thanks for this, it produces an output file but currently it's empty. I've checked that the search term exists in the files. So not sure why it doesn't work. Any troubleshooting tips? – user964689 Mar 03 '12 at 14:50
  • Sorry there is no pipe between wc and the redirection. Fixed it. – Pluc Mar 03 '12 at 14:56
  • Thanks now that works, but only outputs the wc -l count not also the file name and search term. – user964689 Mar 03 '12 at 15:00
0
q="search for this"
for f in *.txt; do echo "$f $q $(grep $q $f | uniq | wc -l)"; done > out.txt
Manish
  • 3,472
  • 1
  • 17
  • 16