0

I have 120 pairs of files I would like to concatenate. I have the list of pair of files and the proposed merged file name in a tab delimited file as follows

Filelist.txt

/filepath/1_first.fasta /filepath/1_second.fasta > /filepath/1_merged.fasta

/filepath/2_first.fasta /filepath/2_second.fasta > /filepath/2_merged.fasta

/filepath/3_first.fasta /filepath/3_second.fasta > /filepath/3_merged.fasta 

/filepath/4_first.fasta /filepath/4_second.fasta > /filepath/4_merged.fasta

I was trying:

$cat Filelist.txt | perl -ne 'chomp;system("cat $_;")'

with the idea of handling the Filelist.txt line by line and using the cat command on the line which for the first line should be...

/filepath/1_first.fasta /filepath/1_second.fasta > /filepath/1_merged.fasta

If I just have one pair and a single line in the Filelist.txt it works fine but with multiple lines I just get ..

/filepath/2_first.fasta: No such file or directory

$

Any clues why this might not be working for every single line? Or an alternative way to do this?

Thanks

Rohan
  • 3,068
  • 1
  • 20
  • 26
user2413355
  • 1

2 Answers2

1

I would suggest this:

sed -e 's/^/cat /' Filelist.txt | bash

Assuming your Filelist.txt is really formatted that way (i.e. actually has the redirection included as shown). Maybe try the above without the | bash part first and inspect the output to make sure it looks like the right commands you want to run.

twalberg
  • 59,951
  • 11
  • 89
  • 84
  • for /filepath/1_first.fastq.gz /filepath/1_second.fastqz > /filepath/1_merged_fastq.gz without bash to test it I get filepath/1_first.fastq.gz./filepath/1_second.fastqzf>stq.gz /filepath/1_merged_fastq.gzfastq.gz almost there but not quite? Thanks for your help :) – user2413355 May 23 '13 at 17:53
  • With output like that, I'm guessing your input file doesn't really look like what is posted above. The only thing the above command should do (without the `| bash`) part, is put `cat ` at the beginning of each line. Unless you have some wonky line ending issues going on or something... Was that `Filelist.txt` created on a Windows or Mac system, maybe? – twalberg May 23 '13 at 18:37
  • Yes on a Mac, I've gone through it on nano in the terminal and checked that all separations are tabs and all line endings are just newlines and its all in the exact order I posted, below is two lines copy and pasted out of my actual Filelist.txt, thanks again for looking at this – user2413355 May 24 '13 at 07:58
  • /Volumes/resource/sequence_files/130412_RUN15/Fastqgz/GSK-0051_S5_L001_R1_001.fastq.gz /Volumes/resource/sequence_files/130404_RUN14/Fastqgz/GSK-0051_S10_L001_R1_001.fastq.gz > /Volumes/resource/sequence_files/merged_fastqs/GSK_0051R1_merged /Volumes/resource/sequence_files/130412_RUN15/Fastqgz/GSK-0051_S5_L001_R2_001.fastq.gz /Volumes/resource/sequence_files/130404_RUN14/Fastqgz/GSK-0051_S10_L001_R2_001.fastq.gz > /Volumes/resource/sequence_files/merged_fastqs/GSK_0051R2_merged – user2413355 May 24 '13 at 08:00
0

Try this:

cat * > merged_file

This will CAT all files in the directory and the output will be outputted into the merged_file.

Rijndael
  • 3,683
  • 2
  • 24
  • 26