Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I'm interested in learning about tools and techniques used to manage many Linux machines. (That is, deploying and maintaining updates.)

One way that I thought of to do this is write a Bash script that uploads another script to the servers and executes the script for each server sequentially. For example:

foreach server
     connect to server and scp update_script.sh to ~/scripts
     ssh user@server -e "sh ~/scripts/update_script.h"

And update_script would use apt-get/aptitude or yum, or whatever to update packages on the server.

Are there better ways to do things like this?


locked by HopelessN00b Dec 5 at 11:48

This question exists because it has historical significance, but it is not considered a good, on-topic question for this site, so please do not use it as evidence that you can ask similar questions here. This question and its answers are frozen and cannot be changed. More info: help center.

Pretty much a duplicate of serverfault.com/questions/2533/… –  Philip Durbin May 27 '09 at 19:58

9 Answers 9

Try Capistrano. It works just like your bash foreach loop above but it is based on Ruby instead of bash. Capistrano is used for operational tasks (a la: put server in maintenance mode, take out of maintenance mode)

+1 for Puppet. It's a good fit for idempotent operations that leave a system in a known state.

pcampbell thanks for adding a Capistrano link. I didn't have enough points when I posted it. –  KevinRae Aug 23 '09 at 5:23

If you want to be able to run commands in parallel on a cluster of Linux systems, one of the following may be of interest:

As a general way of configuring a large network of systems - you probably want to be using tools already mentioned such as cfengine and puppet.


Check out Func for running "things" on several servers at once. Google for "func redhat"

Also if you like puppet you should check out Chef opscode dot com. It solves a few of puppets problems and the config files are written in ruby instead of a DSL. also ohai is a more informative version of facter. Bcfg2 is also another config management tool written in python. Its cool for high security or auditing because every single package can be accounted for, so if something gets added outside of bcfg2 it flags an alert. Its down side is the config files them selves are written in XML.

I had a bunch of links but I guess new users cant post hyperlinks.


I use cfengine to administrate ~150 Linux machines. I have to log in to any given machine less than once a week, and cfengine does the rest. Adding users, removing them, installing packages, etc.


ClusterSSH is what you're looking for. It provides a way to broadcast commands to all nodes in a cluster. Think of it like BashReduce sans Reduce.


How big of a group is "many"? As mentioned already, Webmin is good if it does what you want - you can run arbitrary commands and do some common admin tasks (like synchronizing local users) using https. So, you have less overhead than the ssh method. Webmin's cluster tools are good for "tens" of machines; I've never tried it on larger groups. There's also cfengine, which was mentioned - it can be used for small to extremely large groups of machines (as in, thousands), because it can do tiered management as needed (one master, then some sub-masters, etc). I'm currently using this to manage a network which is about 3500 Unix machines of different flavors. As the other poster said, it's a pain to initially learn, but works very well.

If your systems are homegeneous, or at least fairly homogeneous groups, and you don't have a whole lot (like, under a few hundred), there are several good cluster management toolsets out there. The Oscar project has some admin tools already assembled for use in managing a cluster: http://svn.oscar.openclustergroup.org/trac/oscar, and there are other similar projects whose names escape me (I'll probably remember as soon as I post).

In simpler terms, there are a few paralell ssh tools discussed at the Linux Journal: http://www.linux.com/archive/feature/151340. Like the other ssh-based cluster tools though, you'll still start running into problems when you try to open too many concurrent ssh connections. Depending on your hardware, my esperience is that you'll probably want to keep paralelism below 20-30 simultaneous ssh links.

Basically, there are a whole lot of pre-made solutions for this, and there are just as many home-grown solutions. Look around freshmeat.net and google and you'll find several. Or roll your own; it's not a particularly difficult challenge to solve acceptably if you're just doing 10-20 machines... :)


Someone else already mentioned Puppet.

In the same vein, I can recommend Cfengine. The learning curve can be a little steep, but once you get the hang of it, it's great. I use it to manage about 50 servers and can't believe I ever got along without it.


Try puppet

Another excellent (truly excellent) tool is Webmin, if you add several servers running webmin together (in the webmin interface) you can push updates and view package configurations in its cluster pages.

An alternative, which is more geared to rolling out images, is SystemImager

+1 puppet rules –  user7321 Jul 20 '09 at 7:08

I don't know much about script deployment even if you idea does not seem to bad to me. However, if you want to monitor your linux machines in your process of managing them, I strongly recommand Nagios for this task.

Nagios is a bit a pain to configure (like any linux software, mind you) if you want it to handle more than the usual default tasks, but you have an extensive documentation on their website: nagios.sourceforge.net/docs/3_0/toc.html

Of course, it is free ;)