is it better to use several small machines or fewer big ones to deploy a stack?

Question

Is it better to split the different components of a stack over several smaller machines or get 1 or 2 bigger machines and put several components on a same machine?

My thought is that if I start with several machines, it will be easier to scale by increasing the power of each machine when needed. I am not ready for horizontal scaling anyway for now.

I have a budget of $20/month to start.

I chose DigitalOcean because it looks flexible with their 512MB droplet so here are my options:

1, 2, 3 or 4 Machines of 512MB
2 Machines of 1GB
1 Machine of 2GB    
1 or 2 Machines of 512MB + 1 Machine of 1GB

Here is my stack:

Nginx
+
Gunicorn with 3 workers  (=4 processes, memory footprint: ~50MB/process)
+
3 or 4 RQ workers  (memory footprint of each worker: ~50MB)
+
Redis as Cache (limit to 100MB to start)
+
Redis as DataStore (small size: it is not meant to store a lot of data, mainly used to store the queues for the workers)
+ 
PostgreSQL  (not sure how much RAM I would need)

The main drawback I see of using several small machines is that since the OS is installed on each machine, each machine has a set of resource dedicated to the OS which is "lost" for the application. The main advantage I see is that it looks easier to scale vertically since each machine can be upgraded separately.

What would be a good trade-off?

Option 1

512MB: Nginx, Gunicorn, Redis Cache
512MB: RQ Workers, Redis DataStore
1GB or 512MB: PostgreSQL

I am not sure whether it's worth it to give 1GB for PostgreSQL only... Maybe I should just use a 512MB droplet as well? Also, I am wondering whether the 512MB machine with Nginx, Gunicorn and the Redis cache would not be too small.

So I could do that:

Option 2

1GB: Nginx, Gunicorn, Redis Cache
512MB: RQ Workers, Redis DataStore
512MB: PostgreSQL

Option 3 using 2 bigger machines

1GB: Nginx, Gunicorn, (RQ Workers), Redis Cache, (Redis DataStore)
1GB or 512MB: PostgreSQL, (RQ Workers here?), (Redis DataStore here?)

Answer 1

One way to spread an application across many machines is to run each layer on separate hardware. For example a web server and a database server can run on a single machine or on different machines.

Doing one or the other isn't going to require major design changes in the software stack. Mostly one has to watch out for the additional communication latency introduced by splitting it on different machines. If you for example have an application using many database queries to render a web page, then the large number of roundtrips will lead to a measurable slowdown of your application. But this is something which you can design around.

The amount of scaling you get to do from running different layers on different machines is not much. The number of layers in a software stack tends to grow slower than the number of users. So you'll simply not be able to scale by that approach alone.

However splitting layers on different hardware may prepare you for scaling in the other direction.

If you have one web server communicating with one database server, there is not much stopping you from setting up two web servers communicating with one database server. In other words, each layer of your software stack can be scaled independently.

However any layer maintaining state is difficult to scale across more machines. In particular the database may prove to be the hardest part to scale.

In terms of size of individual machines, I would aim for the size giving you most performance for the money. There will be a size which is optimal from that perspective. Bigger machines will be too expensive, and smaller machines will not be sufficiently cheaper.

But there is also a question about whether to design for scalability now or later. You may waste your time designing for scalability, which you are never going to need. On the other hand if you don't design for scalability across many machines, you may find yourself scrambling to find a single machine large enough to host that one part of your system, which you cannot spread out (a database is a likely candidate as the part needing one giant machine at some point).

Answer 2

This is an interesting question and the last point @kasperd made is the most pertinent (so +1 for that alone). IMO (and many others) - this type of problem is a good one but definitely not something you should be doing unless you are really anticipating growth - otherwise you are wasting time and cash. With that said here are a few considerations and options.

Effectively there are a few questions rolled into this question and the main one being whether to scale up or to scale out (with cost as a consideration). Scaling up is usually going to be more limited (and potentially quite a bit more costly if you need redundancy) than scaling out and with the variety of PaaS options available scale out is less complex than in the past but that said you need to look at the purpose of the application you are delivering to come to a concrete conclusion on what is best.

Digital Ocean doesn't have any autoscaling functionality at present so if you envisage spikes in demand you will have manually do this.

In terms of architecture many bottlenecks within a system will be the database but this really depends on how "data driven" your app would be so without knowing what the exactly the application is or does I cannot really recommend having a larger instance for your database.

In some regards managing multiple small instances (patching / maintenance etc.) adds complexity, however, if all application tiers are separated out it could, potentially, in many instances, make it easier to find bottlenecks (note: it could also make it more complex if it ends up being a network performance issue).

My ultimate advice would be to start as small as possible and put your entire stack on one small instance and use a tool like Forge to manage your stack when needed. There is nothing stopping you from separating your tiers at a later stage, scaling out and switching off the services you do not require on a given box (once the tiers are separated).

This gives you the flexibility to opt for vertical or horizontal scaling should it be necessary, provides you the ability to separate concerns easily and I suppose most importantly because you have no metrics to work off so you are effectively working with the unknown.