cat multiple files based on ID in filename

Question

I would like to combine files with similar ID before first underscore into one file using cat. How do I do this for multiple files like below?
Thought of something like this:

for f in *.R1.fastq.gz; do cat "$f" > "${f%}.fastq.gz"; done

in

9989_L004_R1.fastq.gz
9989_L005_R1.fastq.gz
9989_L009_R1.fastq.gz
9873_L008_R1.fastq.gz
9873_L005_R1.fastq.gz
9873_L001_R1.fastq.gz

out

9989.fastq.gz
9873.fastq.gz

Related: https://stackoverflow.com/questions/41623710 – kvantour Apr 12 '19 at 10:59 — kvantour, Apr 12 '19 at 10:59

oguz ismail · Answer 1 · 2019-04-12T11:12:02.057

3

for f in *_R1.fastq.gz; do cat "$f" >> "${f%%_*}.fastq.gz"; done

>> for appending,
${f%%_*} removes the longest suffix in $f matching _*.

edited Apr 12 '19 at 11:12

answered Apr 12 '19 at 10:55

oguz ismail

1
16
47
69

Does this keep the original files as well? – user2300940 Apr 12 '19 at 10:56
I guess it should be. *_R1.fastq.gz instead of *.R1.fastq.gz – user2300940 Apr 12 '19 at 11:00

score 1 · Answer 2 · answered Apr 12 '19 at 11:03

Here is another way:

for f in *_R1.fast1.gz; do 
   [[ -f "${f%%_*}.fastq.gz" ]] || cat ${f%%_*}_*_R1.fast1.gz > "${f%%_*}.fastq.gz"
done

or if you want to have it a bit more readable:

for f in *_R1.fast1.gz; do 
   key="${f%%_*}"
   [[ -f "${key}.fastq.gz" ]] || cat ${key}_*_R1.fast1.gz > "${key}.fastq.gz"
done

cat multiple files based on ID in filename

2 Answers2

Linked