Many concurrent SSH commands in a bash script?

Question

My Bash-Foo is not strong. Right now I have something like

function update_project {
  for i in server-{1,2,3,4} ; do
    echo "Updating $i"
    ssh $i "git pull"
  done
}

The number of servers is growing every day, and since each update takes about 20 seconds, I'd like to do the requests concurrently. What's the best way to do this, while still being able to see the output (e.g. failed merges)?

score 2 · Answer 1 · answered Jun 26 '10 at 23:52

2

func will allow you to send a command to an arbitrary number of machines, and let you watch the output.

answered Jun 26 '10 at 23:52

Ignacio Vazquez-Abrams

45,939
6
79
84

score 2 · Accepted Answer · answered Jun 26 '10 at 23:54

2

Why are you trying to reinvent the wheel? Just use Capistrano to deploy your project code. It's designed for this exact purpose and runs the deploys in parallel across all of the configured machines.

answered Jun 26 '10 at 23:54

rodjek

3,327
17
14

score 1 · Answer 3 · answered Jun 27 '10 at 00:17

1

There are several parallel ssh projects. Look for pssh.

answered Jun 27 '10 at 00:17

Joel K

5,853
2
30
34

score 1 · Answer 4 · answered Jun 27 '10 at 00:22

1

One of the other answers is the correct thing to do, but to directly answer your question, just add an ampersand at the end of your command:

ssh $i "git pull" &

answered Jun 27 '10 at 00:22

Dennis Williamson

62,149
16
116
151

The difficulty comes when trying to work out which of the hosts the merge failed on, isn't it? – Jonathan Leffler Jun 27 '10 at 02:55
@Jonathan: Yes, the output would be a jumble. – Dennis Williamson Jun 27 '10 at 03:31

score 0 · Answer 5 · answered May 19 '12 at 11:51

If you have GNU Parallel http://www.gnu.org/software/parallel/ installed you can do this:

parallel --slf hostfile --nonall --tag git pull

You can install GNU Parallel simply by:

wget http://git.savannah.gnu.org/cgit/parallel.git/plain/src/parallel
chmod 755 parallel
cp parallel sem

Watch the intro videos for GNU Parallel to learn more: https://www.youtube.com/playlist?list=PL284C9FF2488BC6D1

Many concurrent SSH commands in a bash script?

5 Answers5