Assign to a bash array variable indirectly, by dynamically constructed variable name

Question

Bash script to create multiple arrays from csv with unknown columns.

I am trying to write a script to compare two csv files with similar columns. I need it to locate the matching column from the other csv and compare any differences. The kicker is I would like the script to be dynamic to allow any number of columns to be entered and it still be able to function. I thought I had a good plan to solve this but turns out I'm running into syntax errors. Here is a sample of a csv I need to compare.

IP address, Notes,  Nmap-SSH,   Nmap-SMTP, Nmap-HTTP, Nmap-HTTPS,
10.0.0.1,   ,       open,       closed,     open,     open,
10.0.0.2,   ,       closed,     open,       closed,   closed,

When I read the csv file I was planning to look for "IF column == open; then; populate this column's array with the IP address" This would have given me 4 lists in this scenario with the IPs that were listening on said port. I could then compare that to my security device configuration to make sure it was configured properly. Finally to the meat, here is what I thought would accomplish creating the arrays for me to search later. However I ran into a snag when I tried to use a variable inside an array name. Can my syntax be corrected or is there just a better way to do this sort of thing?

#!/bin/bash
#
#
# This script compares config_cleaned_<ip>.txt output against ext_web_env.csv and outputs the differences
#
#
# Read from ext_web_env.csv file and create Array
#
        FILENAME=./tmp/ext_web_env.csv
#
        index=0
#
        while read line
          do
# How many columns are in the .csv?
        varEnvCol=$(echo $line | awk -F, '{print NF}')
            echo "columns = $varEnvCol"

# While loop to create array for each column

                while [ $varEnvCol != 2 ]
                  do
# Checks to see if port is open; if so then add IP address to array
                   varPortCon=$(echo $line | awk -F, -v i=$varEnvCol '{print $i}')
                        if [ $varPortCon = "open" ]
                          then
                                arr$varEnvCol[$index]="$(echo $line | awk -F, '{print $1}')"
# I get this error message "line29 : arr8[194]=10.0.0.194: command not found"
                        fi
           echo "arrEnv$varEnvCol is: ${arr$varEnvCol[@]}"
# Another error but not as important since I am using this to debug "line31: arr$varEnvCol is: ${arr$varEnvCol[@]}: bad substitution"
                   varEnvCol=$(($varEnvCol - 1))
                done
                index=$(($index + 1 ))
          done < $FILENAME

UPDATE

I also tried using the eval command since all the data will be populated by other scripts.

but am getting this error message:

./compare.sh: line 41: arr8[83]=10.0.0.83: command not found

Here is my new code for this example:

 if [[ $varPortCon = *'open'* ]]
   then
    eval arr\$varEnvCol[$index]=$(echo $line | awk -F, '{print $1}')
 fi

Answer 1

arr$varEnvCol[$index]="$(...)"

doesn't work the way you expect it to - you cannot assign to shell variables indirectly - via an expression that expands to the variable name - this way.

Your attempted workaround with eval is also flawed - see below.

---

tl;dr

If you use bash 4.3 or above:

declare -n targetArray="arr$varEnvCol"
targetArray[index]=$(echo $line | awk -F, '{print $1}')

bash 4.2 or earlier:

declare "arr$varEnvCol"[index]="$(echo $line | awk -F, '{print $1}')"

Caveat: This will work in your particular situation, but may fail subtly in others; read on for details, including a more robust, but cumbersome alternative based on read.

The eval-based solution mentioned by @shellter in a since-deleted comment is problematic not only for security reasons (as they mentioned), but also because it can get quite tricky with respect to quoting; for completeness, here's the eval-based solution:

eval "arr$varEnvCol[index]"='$(echo $line | awk -F, '\''{print $1}'\'')'

See below for an explanation.

---

Assign to a bash array variable indirectly:

bash 4.3+: use declare -n to effectively create an alias ('nameref') of another variable

This is by far the best option, if available:

declare -n targetArray="arr$varEnvCol"
targetArray[index]=$(echo $line | awk -F, '{print $1}')

declare -n effectively allows you to refer to a variable by another name (whether that variable is an array or not), and the name to create an alias for can be the result of an expression (an expanded string), as demonstrated.

bash 4.2-: there are several options, each with tradeoffs

NOTE: With non-array variables, the best approach is to use printf -v. Since this question is about array variables, this approach is not discussed further.

• [most robust, but cumbersome]: use read:

IFS=$'\n' read -r -d '' "arr$varEnvCol"[index] <<<"$(echo $line | awk -F, '{print $1}')"

• IFS=$'\n' ensures that that leading and trailing whitespace in each input line is left intact.
• -r prevents interpretation of \ chars. in the input.
• -d '' ensures that ALL input is captured, even multi-line.
  • Note, however, that any trailing \n chars. are stripped.
  • If you're only interested in the first line of input, omit -d ''
• "arr$varEnvCol"[index] expands to the variable - array element, in this case - to assign to; note that referring to variable index inside an array subscript does NOT need the $ prefix, because subscripts are evaluated in arithmetic context, where the prefix is optional.

• <<< - a so-called here-string - sends its argument to stdin, where read takes its input from.

• [simplest, but may break]: use declare:

declare "arr$varEnvCol"[index]="$(echo $line | awk -F, '{print $1}')"

• (This is slightly counter-intuitive, in that declare is meant to declare, not modify a variable, but it works in bash 3.x and 4.x, with the constraints noted below.)
• Works fine OUTSIDE a FUNCTION - whether the array was explicitly declared with declare or not.

• Caveat: INSIDE a function, only works with LOCAL variables - you cannot reference shell-global variables (variables declared outside the function) from inside a function that way. Attempting to do so invariably creates a LOCAL variable ECLIPSING the shell-global variable.

• [insecure and tricky]: use eval:

eval "arr$varEnvCol[index]"='$(echo $line | awk -F, '\''{print $1}'\'')'

• CAVEAT: Only use eval if you fully control the contents of the string being evaluated; eval will execute any command contained in a string, with potentially unwanted results.
• Understanding what variable references/command substitutions get expanded when is nontrivial - the safest approach is to delay expansion so that they happen when eval executes rather than immediate expansion that happens when arguments are passed to eval.
• For a variable assignment statement to succeed, the RHS (right-hand side) must eventually evaluate to a single token - either unquoted without whitespace or quoted (optionally with whitespace).
• The above example uses single quotes to delay expansion; thus, the string passed mustn't contain single quotes directly and thus is broken into multiple parts with literal ' chars. spliced in as \'.
• Also note that the LHS (left-hand side) of the assignment statement passed to eval must be a double-quoted string - using an unquoted string with selective quoting of $ won't work, curiously:
  • OK: eval "arr$varEnvCol[index]"=...
  • FAILS: eval arr\$varEnvCol[index]=...