How to parse files according to their extension

Question

I've a situation where there are different types of files (of different extension) in some of the directories. I've no idea what type of file is present in a particular directory. Let's assume the following case:

There is a common path to all five directories 1, 2, 3, 4, 5 as:

/dir1/dir2/dir3/1/
/dir1/dir2/dir3/2/
/dir1/dir2/dir3/3/
/dir1/dir2/dir3/4/
/dir1/dir2/dir3/5/

In short, 1, 2, 3, 4, 5 are in dir3.

As an example,
directories 1 and 3 have files with extension .txt
directories 2, 4, 5 have files with extension .gz

I'm doing this as of now:

while [ $a -le 5 ]
    do
        cat /dir1/dir2/dir3/"$a"/*.txt | grep "string" > /path/output"$a"
        zcat -f /dir1/dir2/dir3/"$a"/*.gz | grep "string" > /path/output"$a"
        ((a++))
    done

Outputs are created but few of them are blank. I'm sure that inputs have that string I used in grep. I think the script I'm using is over-writing the files.

How can I perform this by searching the file extension first, and performing either cat or zcat accordingly? Please suggest?

Edit:

I've also tried the following:

while [ $a -le 5 ]
    do  

for f in /dir1/dir2/dir3/"$a"/*.txt
  do  
      cat $f | grep "string" > /path/output"$a"  
  done  
for f in /dir1/dir2/dir3/"$a"/*.gz
  do  
      cat $f | grep "string" > /path/output"$a"  
  done   

done

You edited attempt should work as long as you use `shopt -s nullglob`. This prevents one or the other `for` loop from being entered at all. If, for whatever reason, you do end up with `gz` and `txt` files in the same directory, the result of the `gz` loop will overwrite the result of the `txt` loop. — chepner, May 20 '16 at 14:41
@chepner Yes. It's wokring fine if I redirect the output to different directories - one for .txt and one for .gz. — intruder, May 20 '16 at 16:32

score 1 · Answer 1 · answered May 20 '16 at 14:14

Collect all the txt files in one array, and if that array is not empty, run cat. Otherwise, run zcat. The output of whichever one actually runs gets redirected to the output file.

shopt -s nullglob
for d in 1 2 3 4 5; do
    txt=(/dir1/dir2/dir3/"$d"/*.txt)
    if (( ${#txt[@]} )); then
        grep "string" "${txt[@]}"
    else
        zcat -f /dir1/dir2/dir3/"$a"/*.gz | grep "string"
    fi > /path/output/"$d"
done

How to parse files according to their extension

1 Answers1