7

I regularly do things where I loop through a list of servers to take some action. For example:

for s in `cat servers.txt` ; do
    echo; echo $s
    ssh $s 'do something'
done

I'm wondering (from a shell perspective) if there's an easier way of doing this than cat servers.txt

Yes I know about tools like mcollective, capistrano, etc - I'm often doing this to fix up mcollective problems :-)

7
  • oops, easy to read over that comment "from a shell perspective"... forget about parallel-ssh.
    – hookenz
    Jul 7, 2014 at 2:13
  • Can you clarify what you mean by "easier/better"? If by this you mean you would like to parallelize it, yes you can use xargs to parallelize the ssh connections. Jul 16, 2014 at 0:03
  • 1
    "Easier" - less typing, or a "better/more correct" way. I hadn't considered doing it in parallel - how would you do that using xargs? I usually use multixterm (one of the scripts in the expect package). Jul 16, 2014 at 6:13
  • Ah, didn't read that first comment ... still, I feel parallel ssh is a valid answer, since it's run from a shell.. but yeah, not really "from a shell perspective"
    – Andrew
    Jul 16, 2014 at 7:11
  • 2
    @SoniaHamilton: no matter which method, it's going to be about the same amount of typing. The only improvement I would suggest is to add within the loop a line before your ssh line, such as this: trap 'continue 3';. This will allow you to use `Ctrl-\` to kill the current ssh session without killing the loop. Jul 16, 2014 at 17:09

8 Answers 8

5

My quick and dirty... where servers.txt has a series of hosts or IPs, one-per-line.

#!/bin/bash

SERVER_LIST=/path/to/servers.txt

while read REMOTE_SERVER
do
        ssh $REMOTE_SERVER "do_something_cool"
done < $SERVER_LIST
4
  • Does this really satisfy her requirement for an "easier way"? All you're doing is replacing "cat" with a redirect operator "<". Jul 15, 2014 at 23:49
  • Yes... because she's already aware of the commonly available orchestration tools. Sometimes something like this is needed to fix Puppet or Mcollective issues.
    – ewwhite
    Jul 15, 2014 at 23:51
  • 1
    She's looking for an "easier" way of doing it from within the shell. You only give her an alternate method, not an easier method. I suspect she isnt' clear herself on what she means by "easier". She needs to explain what she thinks are the flaws or shortcomings of her method ... Jul 15, 2014 at 23:53
  • 6
    @MichaelMartinez I dunno, man. The OP doesn't need to do anything at this point... I gave the example of what I use for this task. And usually the task is not something I wish to do in parallel. You're entirely welcome to post your own question and answer if you feel that the original post here is misleading or the answer(s) insufficient.
    – ewwhite
    Jul 16, 2014 at 12:42
7

I use ClusterSSH.
It opens up a lot of small shells, and you can type to all of them in the same time. Really handy when you want to execute the same command on a lot of servers, but still see the output.
I use it like this: clusterssh $(~/get-servers.sh), but obviously you can do something like clusterssh $(cat servers.txt)
The result looks like this:

enter image description here

It's also available as a Debian package.

1
  • ClusterSSH is a wonderful hack :) Also allows you to type into individual spawned terminals if absolutely required. Nov 3, 2021 at 16:47
5

Please do yourself a favor and use something designed for this. You already know about mcollective, but you and I both know that it needs some infrastructure to work. As do puppet and chef.

clusterssh, parallel ssh and dancer shell are small simple improvements over a shell for loop. They don't need more infrastructure.

But there's also ansible, which let's you do that, but also write reusable "playbooks" with several steps. It does need python installed in addition to sshd, but in practice I've never had to separately install that, it's always been available.

Ansible is the only configuration management system I've tried that also works well as a deployment and orchestration tool (puppet needs mcollective and maybe capistrano/fabric for that, ...)

(Yes, Puppet and Chef and all the rest can be run without central servers, but you need to install packages on the hosts to manage, which ansible doesn't need)

1
  • Thanks ptman, lots of useful tools (in addition to the multixterm tool I already use). Jul 17, 2014 at 0:19
4

Here is how to use xargs to parallelize these ssh sessions:

cat servers.txt | xargs -IH -n1 -P0 ssh H 'some command to run'

You can also add the -n or -f options to ssh to redirect the stdin from /dev/null or put the session in the background. If you have to type a password for each host, then this doesn't help you much, but if you are using ssh keys then this works very nicely.

2
  • Thanks Michael, I wasn't aware of the -I option in xargs - nice. Jul 17, 2014 at 0:17
  • Example for those who wonder how to do it with Ansible: ansible -i servers.txt all -a "uname -a" Mar 25, 2023 at 15:30
4

To execute a simple task across a wide range of servers without using any tool designed for this purpose, regardless they require previous infrastructure or not, you can use a simple shell script called mussh, distributed as a package in many distributions.

You can call it with a list of hosts, a list of commands, store both things in files, and quite a few more options, like ssh-agent integration, proxy support, ... Check the manpage for all the details.

An example could be as simple as:

$ mussh -H host_list.txt -C command_list.txt
2
  • Interesting. I've never seen that one before.
    – ewwhite
    Jul 16, 2014 at 22:24
  • Thanks dawud, I didn't know of mussh. Jul 17, 2014 at 0:17
2

Have you considered using parallel-ssh? https://code.google.com/p/parallel-ssh/

I generally fall back to using that if/when our mco or puppet setup is broken. It is another dependency to manage, but totally worth it if you have large fleet of boxen to manage - with the added bonus of being able to choose how many machines to work on in tandem/parallel, or even doing one at a time, as you're used to with bash.

0

Undeleted my answer. Although it's not really clear what you're asking for.


There is a project called parallel SSH which provides parallel versions of ssh, scp and rsync.

So the advantage is that you don't need to do any shell script you just provide it the list of servers to run the command on and it'll do it in parallel.

This is great if you've got a long running set of ssh commands to do as it will do them all in parallel offering a potentially big speed up.

e.g.

parallel-ssh -h myhosts.txt "echo 'hello world'"
-1

copied from Sarav AK at middlewareinventory and modified below

#!/bin/bash
# Script to connect to a list of servers with same username and password 
# and get hostname, uptime and query installed OS version
# you would need the username and password for the remote hosts
# this requires sshpass to be installed on local machine
echo "Enter the Remote username"
read rmtuname

echo "Enter the Remote password"
read -s rmtpasswrd


# read IP addresses from a predefined host list 
for server in `cat /full/path/to/serverlist.txt`
do
        # Printing the remote hostlist 
        echo "Processing Host "$server

        # Write a local shell script for temporary usage and save in current location
        cat << 'EOF' > ./TestScript.sh
                #!/bin/bash
                echo "Running in $0"
                echo "on Hostname = `hostname` and " 
                echo "at TimeStamp = `date` for Uptime of `uptime`"
                echo "for RedHat version `cat /etc/redhat-release`"
                echo "for Debian base `lsb_release -a`"
                exit 0
EOF
        chmod a+x TestScript.sh

        # SCP - copy the script file from current location to remote server tmp location 
        sshpass -p$rmtpasswrd scp -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no TestScript.sh $rmtuname@$server:/tmp/TestScript.sh

        # Take Rest for 3 Seconds
        sleep 3

        # SSH to remote Server  and Execute a Command [ Invoke the Script ] 
        sshpass -p$rmtpasswrd ssh   -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no $rmtuname@$server "/tmp/TestScript.sh"
done

... a snippet of the result/return from the ubuntu host looks like this

...... 
Processing Host 10.0.0.99
Warning: Permanently added '10.0.0.99' (ED25519) to the list of known hosts.
Warning: Permanently added '10.0.0.99' (ED25519) to the list of known hosts.
Running in /tmp/TestScript.sh
on Hostname = UBUNTU-HOST1 and 
at TimeStamp = Sat 25 Mar 14:47:57 UTC 2023 for Uptime of  14:47:57 up 15 days, 18:56,  1 user,  load average: 0.28, 0.19, 0.18
cat: /etc/redhat-release: No such file or directory
for RedHat version 
No LSB modules are available.
for Debian base Distributor ID: Ubuntu
Description:    Ubuntu 22.04.1 LTS
Release:    22.04
Codename:   jammy
Processing Host 10.0.0.110
......

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .