Server Fault is a question and answer site for system and network administrators. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

We're building an API-based web solution on Python which is going to be delivered to individual customers. Each customer is going to have a separate dataset.

Currently, it's not a problem to deploy an instance of the application + database for each customer, but this is going to become nightmare with more customers: imagine hundreds of such instances and administration expenses on that. Obviously, the solution should become more "centralized".

However, the data is quite sensitive, so we can't keep everything in a single database: a compromized server and DB password will lead to the leakage of all data. Such risks are unacceptable.

What are the right technologies to consider when developing a distributed API server platform? Which smart practices will make the solution more "centralized" so we get rid of that many instances but still keep the data isolated?

share|improve this question

closed as too broad by Sven, Rex, Ward, mdpc, Jenny D Mar 15 '14 at 11:18

There are either too many possible answers, or good answers would be too long for this format. Please add details to narrow the answer set or to isolate an issue that can be answered in a few paragraphs.If this question can be reworded to fit the rules in the help center, please edit the question.

up vote 3 down vote accepted

In short, AWS + Cloudformation + Ansible (or some other CM software).

With cloudformation, you can define your "nuts and bolts" infrastructure - networks, servers, RDS instances, etc., and deploy this infrastructure in a reliable, repeatable manner. This same cloudformation manifest would be deployed once for each customer.

Once the infrastructure is deployed, use a configuration management system to configure your servers, deploy code, etc. Ansible is what we're using, and I've found it to be heads and shoulders above its competition (Puppet, Chef, Saltstach, etc.) as far as ease-of-use and flexibility. Ansible is a python project, and being a python shop, it would be easy for you to come up to speed on it.

Additionally, consider seeing if you might be able to deploy your application using AWS's Elastic Beanstalk. If this is possible, it would relieve you of having to handle deployment, server upgrades, and scaling.

If you are concerned about maintaining a 1:1 ratio of customers to server instances, consider deploying several instances of your application on a single server using Docker. This would allow a reasonable amount of segregation between application instances, while still making it fairly easy to deploy in a controlled fashion.

share|improve this answer
Wow, thanks! Right, there are many tools for automated software deployment, and services to manage the servers for us. Still, my main concern is about the total number of applicatin+DB instances: is 1:1 the only way of doing that with my restrictions? – kolypto Mar 13 '14 at 0:47
@kolypto - if you want full isolation of your application instances, then yes, you'll need one server for each customer. Likewise, you'll need one database server (RDS Instance) for each customer. – EEAA Mar 13 '14 at 0:50

Not the answer you're looking for? Browse other questions tagged or ask your own question.