3

Problem:

cat works reads multiple files within a brace {};

cat {file1,file2}.txt

But not when the braced part is specified by a variable.

VAR="{file1,file2}"
cat $VAR.txt

I suspect, as referenced here, this is because quotes are read literally in a variable. So cat is trying to find {file1,file2}.txt which doesn't exist.

Is there a way to specify multiple files as a single element in a variable?

Context:

For my full script I want to loop cat through a set of braced and unbraced file prefixes. So simply setting VAR=( file1 file2) won't help me.

VAR=( "{file1,file2}" file3 file4 )
for i in ${VAR[@]}; do cat $i.txt > merged$i.txt; done
Community
  • 1
  • 1
James Reeve
  • 65
  • 6
  • 1
    Also be aware that `echo file{1,2,3}` will always return `file1 file2 file3` with no regard for whether those files actually *exist*... – Paul Hodges Aug 15 '18 at 21:02
  • @glenn jackman: Thanks. I've edited my post to remove the quotation marks. – James Reeve Aug 15 '18 at 21:41
  • `VAR=( {file1,file2} file3 file4 )` will work fine. Why are you quoting those strings? – Charles Duffy Aug 15 '18 at 21:42
  • And btw, it should be `for i in "${VAR[@]}"; do cat "$i.txt" > "merged$i.txt"; done` -- quotes aren't optional if you want correct operation for all possible filenames, IFS values, etc. – Charles Duffy Aug 15 '18 at 21:46
  • I quoted the string to get `{file1,file2}` as the output of `for in ${VAR[@]}; do echo $i; done`. I want `cat` to read and merge both files in one iteration of the loop. – James Reeve Aug 15 '18 at 22:02
  • Technically you can use *command substitution* as well, e.g. `var=$(echo file{1,2,3,4}.txt)` or `var=$(echo file{1..4}).txt` to store the names as a space separated string, e.g. `"file1.txt file2.txt file3.txt file4.txt"`, but that's not very useful. – David C. Rankin Aug 15 '18 at 22:10
  • Are your file groupings fixed or can they be determined programmatically? It seems like you need a multidimensional array (or a "hash of arrays"), which bash does not possess. – glenn jackman Aug 16 '18 at 00:15
  • The files grouping is fixed. The files content has no information about the group. I had to manually access the group information from the database I downloaded the files from. I created $VAR to read in the grouping information. – James Reeve Aug 16 '18 at 16:04

2 Answers2

3

Brace expansion is the first thing bash does, so you can't store it in a variable (unless you resort to eval)

Simply remove the quotes when you use an array:

$ files=( file{1..4} )
$ declare -p files
declare -a files='([0]="file1" [1]="file2" [2]="file3" [3]="file4")'

Then

cat "${files[@]}" > merged
glenn jackman
  • 238,783
  • 38
  • 220
  • 352
  • 2
    Also, don't use ALLCAPS varnames. You'll inevitably overwrite a crucial shell variable like PATH. – glenn jackman Aug 15 '18 at 20:55
  • @ glenn jackman: Thanks for the note on syntax, you've saved me from some potential future problems. Is there a way to use `declare` to treat "file1" & "file2" as the same element in the variable? I want to merge only certain files so I output `"file1+2", "file3" & "file4". – James Reeve Aug 15 '18 at 21:47
  • Not safely. It sounds like you want to be using `eval` and take the risks of malicious filenames. Also, if that's a key requirement, it belongs up in your question -- I had no idea this is what you wanted. – glenn jackman Aug 15 '18 at 22:08
  • It seems `eval` is the way to go. I'll look into it, carefully. I've also tried to edit the question to be more direct, thanks for your help. – James Reeve Aug 15 '18 at 22:37
0

Is eval what you're looking for?

VAR="{file1,file2}"
eval "cat $VAR.txt"
ewindes
  • 790
  • 7
  • 17
  • 2
    `eval` [is dangerous](http://mywiki.wooledge.org/BashFAQ/048), **especially** in the context of filenames. Encouraging its use without caveats is not great practice. – Charles Duffy Aug 15 '18 at 21:34
  • I tired it out, it work. However, it doesn't seem to work in the loop. I'll play around with the loop to find a solution, thanks. – James Reeve Aug 15 '18 at 21:37
  • 1
    @JamesReeve, ...lots of things *look like* they work in bash when only tested in the simple cases but have subtle, or even dangerous, bugs. If you're lucky, those subtle bugs just mean your script fails when operating on filenames with spaces, but misuse of `eval` often means your script fails by executing filenames as code -- so if you some joker ran `touch '$(rm -rf ~)'`, unsafe `eval`-using code can destroy your whole home directory. – Charles Duffy Aug 15 '18 at 21:39