2

We're building an API-based web solution on Python which is going to be delivered to individual customers. Each customer is going to have a separate dataset.

Currently, it's not a problem to deploy an instance of the application + database for each customer, but this is going to become nightmare with more customers: imagine hundreds of such instances and administration expenses on that. Obviously, the solution should become more "centralized".

However, the data is quite sensitive, so we can't keep everything in a single database: a compromized server and DB password will lead to the leakage of all data. Such risks are unacceptable.

What are the right technologies to consider when developing a distributed API server platform? Which smart practices will make the solution more "centralized" so we get rid of that many instances but still keep the data isolated?

kolypto
  • 11,058
  • 12
  • 54
  • 66

1 Answers1

3

In short, AWS + Cloudformation + Ansible (or some other CM software).

With cloudformation, you can define your "nuts and bolts" infrastructure - networks, servers, RDS instances, etc., and deploy this infrastructure in a reliable, repeatable manner. This same cloudformation manifest would be deployed once for each customer.

Once the infrastructure is deployed, use a configuration management system to configure your servers, deploy code, etc. Ansible is what we're using, and I've found it to be heads and shoulders above its competition (Puppet, Chef, Saltstach, etc.) as far as ease-of-use and flexibility. Ansible is a python project, and being a python shop, it would be easy for you to come up to speed on it.

Additionally, consider seeing if you might be able to deploy your application using AWS's Elastic Beanstalk. If this is possible, it would relieve you of having to handle deployment, server upgrades, and scaling.

If you are concerned about maintaining a 1:1 ratio of customers to server instances, consider deploying several instances of your application on a single server using Docker. This would allow a reasonable amount of segregation between application instances, while still making it fairly easy to deploy in a controlled fashion.

EEAA
  • 109,363
  • 18
  • 175
  • 245
  • Wow, thanks! Right, there are many tools for automated software deployment, and services to manage the servers for us. Still, my main concern is about the total number of applicatin+DB instances: is 1:1 the only way of doing that with my restrictions? – kolypto Mar 13 '14 at 00:47
  • @kolypto - if you want full isolation of your application instances, then yes, you'll need one server for each customer. Likewise, you'll need one database server (RDS Instance) for each customer. – EEAA Mar 13 '14 at 00:50