0

I am trying to get the paste command to loop through pairs of files, pasting them together and outputing each as a unique file. I've tried a lot of things, here are a few:

 for i in *_temp4.csv; do paste *_temp4.csv *_temp44.csv > ${i}_out.csv; done
 #Each output contains each input file (rather than pairs). Obviously this is because of the * wildcard

 for i in *_temp2.csv_temp4.csv; do paste $_temp2_temp4.csv $_temp3_temp44.csv > ${i}_out.csv; done

no error, empty output files

 for i in *_temp2.csv_temp4.csv; do paste ${_temp2_temp4.csv} ${_temp3_temp44.csv} > ${i}_out.csv; done

output:

 combo15.awk: line 12: ${_temp2_temp4.csv}: bad substitution

I think I must be missing something very basic about how $ gets used, but I've been googling all night to no avail.

my entire code, for context, although I don't see why the previous lines should influence anything about this.

 for i in *.dat; do awk 'NR > 23 { print }' ${i} > ${i}_temp1.csv; done

 for i in *_temp1.csv; do awk 'BEGIN{OFS=FS=","}$2==0{$2="between"}BEGIN{OFS=FS=","}$2==1{$2="lego"}BEGIN{OFS=FS=","}$2==2{$2="pin"}BEGIN{OFS=FS=","}$2==3{$2="dice"}BEGIN{OFS=FS=","}$2==4{$2="jack"}BEGIN{OFS=FS=","}$2==8{$2="escape"}{print}'  ${i} > ${i}_temp2.csv; done

 for i in *_temp2.csv; do awk -v OFS="," '{$4 = $1 - prev1; prev1 = $1; print;}' ${i} > ${i}_temp3.csv; done  

 for i in *_temp2.csv; do awk -F "," 'BEGIN{print "new line"}{print $2}' ${i} > ${i}_temp4.csv; done

 for i in *_temp3.csv; do awk -F "," '{print $5}' ${i} > ${i}_temp44.csv; done

 for i in *_temp2.csv_temp4.csv; do paste $_temp2_temp4.csv    $_temp3_temp44.csv > ${i}_out.csv; done
Stonecraft
  • 860
  • 1
  • 12
  • 30
  • For the input files? trial_02_mid.dat trial_02_bottom.dat trial_03_top.dat All the other outputs are generated as expected, only the last line is giving me trouble. – Stonecraft Jul 19 '15 at 10:15
  • Yes, and the variations on it in my OP. – Stonecraft Jul 19 '15 at 10:18
  • I am not sure, i understood your requirement. But is this what you need? `for i in *_temp4.csv; do paste "$i" "${i/_temp4.csv/_temp44.csv}" > ${i}_out.csv; done` – anishsane Jul 19 '15 at 10:23
  • Almost! `for i in *_temp4.csv; do paste "$i" "${i/_temp4.csv}" "${i/_temp44.csv}" > ${i}_out.csv; done` almost works, but it has extra columns (each extra column is from *_temp4.csv file. The question says .dat because those are the original input files, the temp files I am trying to paste together are named .csv I am trying to use the variable (which I think is coming from the wildcard at the beginning of the line) and append it with `_temp2.csv_tem4.csv` and `temp3.csv_temp44.csv` such that it takes the unique filenames created from the previous lines. – Stonecraft Jul 19 '15 at 10:32

1 Answers1

1

Your problem is, that names of your files grow uncontrollably. This change should solve this problem:

for i in *.dat; do awk 'NR > 23 { print }' ${i} > ${i}_temp1.csv; done

for i in *.dat; do awk 'BEGIN{OFS=FS=","}$2==0{$2="between"}BEGIN{OFS=FS=","}$2==1{$2="lego"}BEGIN{OFS=FS=","}$2==2{$2="pin"}BEGIN{OFS=FS=","}$2==3{$2="dice"}BEGIN{OFS=FS=","}$2==4{$2="jack"}BEGIN{OFS=FS=","}$2==8{$2="escape"}{print}'  ${i}_temp1.csv > ${i}_temp2.csv; done

for i in *.dat; do awk -v OFS="," '{$4 = $1 - prev1; prev1 = $1; print;}' ${i}_temp2.csv > ${i}_temp3.csv; done

for i in *.dat; do awk -F "," 'BEGIN{print "new line"}{print $2}' ${i}_temp2.csv > ${i}_temp4.csv; done

for i in *.dat; do awk -F "," '{print $5}' ${i}_temp3.csv > ${i}_temp44.csv; done

for i in *.dat; do paste ${i}_temp4.csv    ${i}_temp44.csv > ${i}_out.csv; done
Sleafar
  • 1,486
  • 8
  • 10
  • Yes! That works perfectly! I was going to worry about the crazy filenames later, but that fixed it. Thank you! – Stonecraft Jul 19 '15 at 10:49