Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

We'd like some ideas on how to automatically configure Ubuntu virtual machines as staging servers for our application, test out the staging code, and then script deployments to the production server (rather than a person installing everything on the staging server via ssh, copying files over, hand-editing configurations, etc.)

We can move to AWS if need be but we're okay on Azure for now --- we basically get an Ubuntu VM with SSH access on Azure, so it shouldn't be that different from others ways of configuring linux-based servers I imagine

What we want a script to do is basically:

  • Go "in" to the fresh virtual machine via SSH
  • Setup nginx, MariaDB, php, other apt-get installs
  • Deploy a copy of our wordpress code from a Git repository
  • Import a sample staging database into MariaDB with wordpress config info and other information
  • Use the cloudflare API to setup a DNS hostname for the server (optional, can do this manually if needed)
  • Let the person testing the new code poke around

Then if it works fine we want to automate pushing the new code in Git to our production server without manually editing wordpress config files and so forth. But that's a bit separate from the question of setting up the staging environment through a script

The tools I've come across so far are these (can't provide links because of low account rep)

  • Chef
  • Puppet
  • Vagrant
  • Salt Stack
  • Ansible
  • Juju

Before diving right into them I was wondering if someone has a 10,000ft view of what is applicable to our needs.

share|improve this question

4 Answers 4

up vote 2 down vote accepted

What you're really looking for are solutions to two different (but closely-related) problems that stack on top of each other:

  1. Configuration management
  2. Deployment automation

There's some overlap between the two, and as you're starting out it will probably be a little bit fuzzy which tools to use for which parts of your infrastructure. Here are some general guidelines to get you started, though:

Configuration management tools generally follow a resource-oriented, idempotent model. That is, you model your system as a set of resources that have state, and you run your configuration management tool continuously to see if anything looks different from your specification. If the resource is not in that state, you have some type-specific logic to bring the resource into that state. A package or a user account is a simple and obvious kind of resource, but you might also have ElasticSearch templates, entries in your SELinux database, or Nagios hosts that can also be declared as resources in your configuration management system of choice.

As a quick example, a package may be installed or not installed, and it may have a version attached. A configuration management tool will be able to take a configuration specification that says package X should be installed, see that it's not currently installed, and take the right steps to bring it from not installed to installed state (obviously, by installing it).

Idempotence means that you can apply the same configuration ten, a hundred, or a thousand times and get the exact same result -- only the things that need changing are actually changed. This is in contrast to a script that might, say, append a line to a configuration file a hundred times (meaning you end up with the same line in that file a hundred times).

The disadvantages of a configuration management system using this model are that the associations between your resources are fairly loose, they don't map well to long-running tasks like database imports or schema migrations, and they typically don't give you a ton of control over exactly what order things are applied in across multiple systems in a stack.

Puppet and Chef are examples of configuration management systems.

Deployment automation tools are for performing ad-hoc tasks. In other words, you run them explicitly when you need to do something. (Okay, sometimes you run them implicitly from a trigger, like from a continuous integration system.) These often orchestrate multiple systems in a predictable way; for example, you might only want one of your web servers to run a database migration during an upgrade, and you might want to do your app server upgrade only if the database migration succeeds, and you might want your app servers to restart three at a time so you don't bring down your whole application during the update. Most importantly, you only want this process to kick off exactly when you tell it to.

Capistrano and Fabric are popular examples of deployment automation tools. For single servers, you can just as easily use things like Makefiles all the way across your app.

In your particular case, you likely want a configuration management system handling things like the install of your database system and PHP, and the configuration of your web server. On the other hand, you likely want a deployment tool to handle the creation of your databases and the population of your test data into them. The download of your app code from Git can easily be modeled by either a configuration management or a deployment automation tool, depending on which suits your needs better. And regardless of which method you choose, people have differing opinions on how the config files for your application should make their way onto the server.

The most important thing when you're working with tools like these is that they're a complete waste if you're not using them correctly. In other words, if you're using a fancy automated deployment method to put your app on staging, and your production server doesn't actually look exactly like your fancy-pants staging server, you've blown a whole lot of effort for almost no gain. Do things right, and they'll serve you very well.

share|improve this answer

I'm more of a RH guy (and I would use kickstart to do the basics + puppet to do the rest) but anyway, try puppet. It should be easy to get started as some time ago Wikimedia has published GIT with all their puppet config files:

http://blog.wikimedia.org/2011/09/19/ever-wondered-how-the-wikimedia-servers-are-configured/

share|improve this answer

I know this answer might sound strange... but consider using Hudson, you can make it do whatever you want to, just by some tweaking .. and writing the scripts..

share|improve this answer
    
This does not provide an answer to the question. To critique or request clarification from an author, leave a comment below their post - you can always comment on your own posts, and once you have sufficient reputation you will be able to comment on any post. –  Wesley May 22 '13 at 21:14

What you are looking at are two things -

  • configuration management
  • deployment automation

I would suggest using Chef and you can use chef-bootstrap which is OS agnostic.

First part you want to just setup mariadb, nginx, php and other packages, which is just plain and simple with chef.

Second part is first time setup - like importing database, setting up dns, allow user to login so setup ssh keys, you can still use chef and put a checks so that we know these scripts need to run first time, like first boot in many systems

For third part, which is continuous deploy, you need to figure out triggers (like every git checkin should trigger a git refresh on local machine, which can be done using simple scheduled cron job which is matching the remote head number with local head number, if they are in mismatch then its just a matter of pulling the code down)

Since you are using php, you don't have restart the server or things like that, so its all good to just do a git pull.

so what jgoldschrafe has said makes sense; I would just do it with bunch of chef scripts running on client site. If you have multiple nodes I would setup chef server as well and run those as chef client.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.