Bash script to create variables from and run commands on each item in an external list

Question

On my local machine I have a text file containing a list of IP addresses and directory names.

list.txt

# Comment for Server 1
XXX.XX.XXX.X
DIRECTORY-NAME
# Comment for Server 2
XXX.XX.XXX.X
DIRECTORY-NAME
# Comment for Server 3
XXX.XX.XXX.X
DIRECTORY-NAME

I am trying to come up with something basically like:

#!/bin/sh
export NEWPASSWORD=m0&*a$%3JbFD*#U#N2
input="list.txt"
for every third line in input
  do
    set variable DIRECTORYNAME
for every second line in input
  do
    set variable IPADDRESS
    ssh root@$IPADDRESS cd /srv/users/myusername/apps/$DIRECTORYNAME/public/ wp user update JohnDoe --user_pass="$NEWPASSWORD"
done

Is something like this even possible? I did see this but that doesn't quite apply here.

Idelic · Answer 1 · 2020-11-29T17:22:08.367

2

You can try this:

while read
do
  read dir
  read ip
  # Your command using $dir and $ip
done < list.txt

The read after while reads the comments and discards them.

Each read command reads a single line from standard input (which in this case we've redirected from list.txt). For a more detailed explanation, see the documentation.

edited Nov 29 '20 at 17:22

answered Nov 20 '20 at 16:51

Idelic

121
3

I'm not quite clear on how that works but it does indeed work. So simple it's beautiful. :-) – Jay Nov 24 '20 at 20:11
In trying to understand how your code works, can I ask.... if list.txt contained 2 lines of comments before the server IP and the directory name, instead of a single line, what would your "do" "read" bits look like then? – Jay Nov 28 '20 at 01:49
1

You'd need to just add another `read` with no arguments right before the `read dir` line. I added a brief description of `read` and a link to the documentation. – Idelic Nov 29 '20 at 17:22

parkamark · Answer 2 · 2020-11-19T22:39:52.963

0

This will probably work:

$ awk '(NR-1)%3 {print}' list.txt | xargs -n2 | while read host dir; do echo host=$host dir=$dir; done
host=XXX.XX.XXX.X dir=DIRECTORY-NAME
host=XXX.XX.XXX.X dir=DIRECTORY-NAME
host=XXX.XX.XXX.X dir=DIRECTORY-NAME

Explanation of code:

Use awk to filter out comment lines leaving the remainder. grep -v '^#' would have also worked here.
Pipe to xargs which will re-format the single column of alternate values into 2 columns, the first being the host, the second being the directory.
Pipe into a BASH while loop, reading those values into independent variables.

Caveat: The 3rd step creates a BASH sub-shell, so you may find that variables defined outside this (like your password) are not available. The quick work around for this, in your case, is to define the static password in the sub-shell code instead. Having said that, you are exporting the password so I guess that may circumvent this problem?

See https://stackoverflow.com/questions/13726764/while-loop-subshell-dilemma-in-bash for further information about BASH sub-shells.

edited Nov 19 '20 at 22:39

answered Nov 19 '20 at 22:19

parkamark

1,128
7
11

That almost works however once I change the code to do more than just echo it results in only the first item in the list being processed. eg: This doesn't work. ```awk '(NR-1)%3 {print}' list.txt | xargs -n2 | while read host dir; do ssh root@$host cd /srv/users/username/apps/$dir/public/ && echo I am in $dir on $host; done``` – Jay Nov 20 '20 at 00:34
1

Looks like you need to add `-n` to ssh per https://stackoverflow.com/questions/37200100/bash-while-loop-stops-after-first-iteration-when-subshell-is-called – parkamark Nov 20 '20 at 01:52
I did try with ``-n`` but that also gave an error. I forget what the exact wording of the error was at the moment. But something negative about "psuedo shell". – Jay Nov 23 '20 at 23:49

Bash script to create variables from and run commands on each item in an external list

2 Answers2