Iterate over several associative arrays

Question

For my current use case I'm creating an scp script which will copy log files from one server to one or more other servers.

I.e.

server1:/my/path1/log-files.* --> log_server1:/log/path1/server1
server1:/my/path2/log-files.* --> log_server2:/log/path/server1
server1:/my/path3/log-files.* --> log_server1:/log/path2/server1

I would like to be able to use Associative arrays (Arrays) in bash (version 4) for the log file configuration, and loop over all of the A. Arrays by putting their names into an indexed array.

But I'm stumped on how I'm referencing a named A. Array using a variable as the name of the A. Array.

Example:

#!/bin/bash
# GNU bash, version 4.1.2(1)-release (x86_64-redhat-linux-gnu) from RedHat/CentOS 6.4
declare -A log_server1 log_server2
log_server1=([name]="ls1" [user]="user")
log_server2=([name]="ls2" [user]="user")

declare -A log1 log2 log3
log1=([log_server]="log_server1" [path]="/my/path1" [file]="log-files" [rpath]="/log/path1/server1")
log2=([log_server]="log_server2" [path]="/my/path2" [file]="log-files" [rpath]="/log/path/server1")
log3=([log_server]="log_server1" [path]="/my/path3" [file]="log-files" [rpath]="/log/path2/server1")

logs=(log1 log2 log3)

for log in ${logs[@]}
do
  # How can I now refer to the A. Array by the name of "log1", etc ?
  ...
done

gniourf_gniourf · Accepted Answer · 2013-12-01T15:36:37.540

You can use indirect expansion, but it's really ugly!

#!/bin/bash

declare -A log_server1=([name]="ls1" [user]="user")
declare -A log_server2=([name]="ls2" [user]="user")

declare -A log1=([log_server]="log_server1" [path]="/my/path1" [file]="log-files" [rpath]="/log/path1/server1")
declare -A log2=([log_server]="log_server2" [path]="/my/path2" [file]="log-files" [rpath]="/log/path/server1")
declare -A log3=([log_server]="log_server1" [path]="/my/path3" [file]="log-files" [rpath]="/log/path2/server1")

logs=( log1 log2 log3 )

for log in "${logs[@]}"; do
    l_ls=$log[log_server]
    l_p=$log[path]
    l_f=$log[file]
    l_rp=$log[rpath]
    echo "array $log:"
    echo "    log_server => ${!l_ls}"
    echo "    path => ${!l_p}"
    echo "    file => ${!l_f}"
    echo "    rpath => ${!l_rp}"
done

In the reference manual section I linked above, you'll read:

If the first character of parameter is an exclamation point (!), a level of variable indirection is introduced. Bash uses the value of the variable formed from the rest of parameter as the name of the variable; this variable is then expanded and that value is used in the rest of the substitution, rather than the value of parameter itself. This is known as indirect expansion. The exceptions to this are the expansions of ${!prefix} and ${!name[@]} described below. The exclamation point must immediately follow the left brace in order to introduce indirection.

Question. Why don't you, instead, create associative arrays log_server, path, file and rpath with keys log1, log2 and log3? as in:

#!/bin/bash

declare -A log_server1=([name]="ls1" [user]="user")
declare -A log_server2=([name]="ls2" [user]="user")

declare -A log_server path file rpath

log_server[log1]="log_server1"
path[log1]="/my/path1"
file[log1]="log-files"
rpath[log1]="/log/path1/server1"

log_server[log2]="log_server2"
path[log2]="/my/path2"
file[log2]="log-files"
rpath[log2]="/log/path/server1"

log_server[log3]="log_server3"
path[log3]="/my/path3"
file[log3]="log-files"
rpath[log3]="/log/path2/server1"

for log in "${!log_server[@]}"; do
    echo "log server $log:"
    echo "    log_server => ${log_server[$log]}"
    echo "    path => ${path[$log]}"
    echo "    file => ${file[$log]}"
    echo "    rpath => ${rpath[$log]}"
done

Hell-lo... I looked around this quite some time, but this sollution did not come easily - thus your conclusion - ugly - is valid. It wont be very readable for others, I'll say... — sastorsl, Dec 01 '13 at 17:15
Your suggestion for an other way of working around this particular problem is interesting. Allthough, your initial answer answers my question in a more general way with regards to associative arrays and looping over more than one array. As you saw, I'm also using an associative array for my log server host definition, which also must be referenced. I'm almost concluding: Not the tool for the job, but it's an interesting venture into bash none the same. — sastorsl, Dec 01 '13 at 17:19
I see where I went wrong with regards to indirect expansion (not part of my question). I wrote `l_ls=${log[log_server]} ; l_ls=${!l_ls}` where I should have dropped the braces. — sastorsl, Dec 01 '13 at 17:26

score 0 · Answer 2 · answered Dec 01 '13 at 17:35

Presenting my own answer. I'm expecting some healthy critisism :-) However, the main question was how to use identical associative arrays, and looping over them in a unified manner.

Suggestions on how to achieve the same will be greatly appreciated:

#!/bin/bash
# GNU bash, version 4.1.2(1)-release (x86_64-redhat-linux-gnu) from RedHat/CentOS 6.4
declare -A log_server1 log_server2
log_server1=([name]="ls1" [user]="user")
log_server2=([name]="ls2" [user]="user")

declare -A clog1 clog2 clog3
clog1=([log_server]="log_server1" [path]="/my/path1" [file]="log-files" [rpath]="/log/path1/server1/")
clog2=([log_server]="log_server2" [path]="/my/path2" [file]="log-files" [rpath]="/log/path/server1/")
clog3=([log_server]="log_server1" [path]="/my/path3" [file]="log-files" [rpath]="/log/path2/server1/")

for log in ${!clog*}
do
    l_ls=$log[log_server] ; l_p=$log[path] ; l_f=$log[file] ; l_rp=$log[rpath]
    l_ls=${!l_ls} ; l_p=${!l_p} ; l_f=${!l_f} ; l_rp=${!l_rp}
    r_n=$l_ls[name] ; r_u=$l_ls[user]
    r_n=${!r_n} ; r_u=${!r_u}
    echo "Array $log:"
    cmd="   scp ${l_p}/${l_f}* ${r_u}@${r_n}:${l_rp}"
    echo "${cmd}"
done

Result:

$./bash-A-Array.sh
Array clog1:
   scp /my/path1/log-files* user@ls1:/log/path1/server1/
Array clog2:
   scp /my/path2/log-files* user@ls2:/log/path/server1/
Array clog3:
   scp /my/path3/log-files* user@ls1:/log/path2/server1/

Iterate over several associative arrays

2 Answers2