How To Setup A Centralized Repo To Distribute Files To Servers

Question

Assume I have dozens, even hundreds of servers. What is the best method of pushing a central copy of files to a given directory to all servers? Example, we update a bash script, and put it on a central repo. We then want the repo to push the updated bash script to all servers automatically. Assume also that the repo can ssh into all servers via public key.

I know I could whip up some hacked shell script to loop through a list of servers and scp, but there has to be a better way to manage this. How about rysnc? Would LOVE a GUI interface, like, add servers, then simply upload a file, and it gets scp'd to all the servers in the group.

Thanks.

Answer 1

In addition to the other answers for centralized configuration (we use chef internally)..

You can also use a svn file repository like subversion for its file versioning capabilities and its HTTP front end.

For example we have a apache web server farm with httpd on 5 nodes, and I keep a copy of httpd.conf and /etc/httpd/conf.d/www.myserver.com.conf in the svn repo, this allows us to roll out changes to the instances easily, and also to roll-back in the case of a f**k up.

It also represents a component in your asset database, and can be used to confirm that all your instances are running the same configuration if you deploy it automatically.

Obviously in this modern times there are whole bunch of alternatives for a repository such as git, mercurial or a davfs interface to some fileserver that has a local versioning strategy.

Answer 2

If you want a heavy weight solution that will centrally manage server configuration (and, if you're dealing with dozens or hundreds of boxes, you should be looking at this sort of thing), take a look at Chef or Puppet. This may be overkill for your environment, and there is a steep learning curve, but I can't imagine dealing with that many boxes in any other way.

If you just have a directory of bash scripts, you can possibly mount a network share of that directory.

Actually, I think we're going to need more information about your environment. Do you have direct control of all those servers? Are you instead saying "I distribute some software to a bunch of customers, and want to keep that software up to date" (in which case, how is your software packaged)? What exactly is your situation?

Answer 3

You should investigate a Configuration Management system like Puppet, Bcfg2, Chef, CFEngine, or others

Puppet for example, covers all your listed use cases, and provide additional functionality like stages roll-outs, auditing/reporting, complex work-flows etc.