Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

Say I have application servers, database servers, and a few dns-round-robin load balancers. All this powered by images created with Packer with deployment managed with Terraform. How do I change the image of the database servers without nuking their data when the instances get destroyed and recreated?

The simplest thing I can think of would be to turn off writes, snapshot the database, and then restore the snapshot to the new servers. But it feels really wrong to rely on manual fiddling like that, and it also feels wrong to take the service down for a simple upgrade. There is a cleaner and better way, right?

(P.S. I can't tag this with terraform or with packer because the tags doesn't exist yet, and it takes 300 reputation to create a tag. Anybody care to fix this?)

share|improve this question

1 Answer 1

up vote 3 down vote accepted

There's no simple answer to this question.

Using an architecture designed around images (commonly referred to as "immutable infrastructure") works fantastically for stateless services, like your application servers.

It's most definitely possible to expand that to your stateful services with the right tools, failover systems and upgrade paths, but those are usually overkill for simple systems (as the one you describe).

One thing to keep in mind when using these tools is that you don't have to go "all in". Packer and Terraform are very much designed to work only where you want them. They don't enforce a pattern across all of your systems on purpose.

Practically speaking, the best way to handle this problem is maintain your database servers differently, outside of Packer (building the initial image, yes! But not necessarily upgrading them in the same way as the stateless web servers) or outsource managing the state to someone else. Notable options include Heroku Postgres or AWS RDS.

To round it out – yes, it's possible, but with our current tooling it's probably more trouble then it's worth at smaller scale or with simple architectures.

Packer and Terraform can still be a huge boon in other aspects of the very same infrastructure – Terraform, for example, could provision a Heroku database for use in your DigitalOcean Application servers in a very straight forward way. Packer can handle upgrading and releasing your application server images, and likewise for development.

share|improve this answer
    
Ah, the simplest way to win is by refusing to play the game. I bet Hashicorp will come out with software for managing databases someday. Anyway, I can get on with my new infrastructure now, many thanks! –  jpadvo Jul 31 at 19:55

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.