This requires quite a bit of application analysis. When figuring out how well a given system will scale, there are a few questions to ask yourself to help identify scaling pain-points:
- If I suddenly see an order of magnitude more traffic, where will I see problems, and how quickly can I address them?
- How about two orders of magnitude?
- If my site is up for a year at a magnitude increase, where will I see problems due to size?
These questions will help you figure out where any single-points may be tripping you up, and identify tiers that require excessive resources to scale out. Some things will be hard to model, such as a year's worth of DB growth and how that impacts DB performance. Others will be easy, such as app-server performance.
This helps in the 2 or 3 tier analysis since it'll let you know if you need to separate the first two tiers at all, and will help you figure out how far down the road you'll need a discrete first tier. If it looks like two orders of magnitude increase for a year causes problems in the first tier, design your tier 1/2 to be split later and do it later as needed.