VPS hosting for a user-generated content site like fmylife.com?

Question

I'm in the process of creating my very first website and I wanted to make a clone of fmylife.com and bought a script called fmyscript for that purpose.

The requirements to run fmyscript are as follows:

Apache web server with mod_rewrite enabled.
MySQL version 4.0+
GD Library 2+ with freetype support.
PHP5 with the following configuration
    safe_mode = off
    short_open_tag = on
    output_buffering = on
    register_argc_argv = On
    session.gc_maxlifetime = 15000 (Recommended or higher)
    register_globals = off
    magic_quotes_gpc = on
    allow_url_fopen = on

I wanted to host my site on a VPS by linode.com. My questions are:

1. Can I run a script like that on a VPS?
2. What are the up- and downsides to VPS when it comes to hosting websites with user-generated content?
3. On linode.com it says that 'all incoming traffic is free.' What exactly does this mean? For example, if I ordered the 'Linode 512' plan with 200GB of transfer and had 1 - 2 million visitors a day, that would be incoming traffic, right? That's my guess anyway.

Basically, I want a fast-running site, that can support hundreds of thousands of clicks a day and now I'm looking for the best hosting solution for such a site.

Please excuse me if my writing is confusing, English is not my native language.

Thank you in advance for your help and have a Merry Christmas!

Answer 1

1-2 million visitors a day is kind of a lot and I doubt that you would be able to serve that off of a VPS just like that.

To answer your questions:

1 ) Yes, you can do that on a VPS. But it will require that you have enough basic understanding of servers to set it up. A VPS will allow you to do this, since you can become root on it.

2 ) Compared to what? Compared to a random shared hosting solution, it gives you mostly upsides; full control of features, usually better resource allocation, usually dedicated resources and such. The "downsides" are that you will need to maintain the server software yourself, you will need technical know-how to run it.

3 ) Incoming traffic is very little compared to the outgoing traffic. Basically, only the requests are incoming. Unless of course, if you build an imagehosting site you will have a reasonable amount of incoming traffic. All content submitted by users is incoming traffic upon submission. When serving it to other users it is outgoing traffic.

I would assume that 1-2 million visitors a day would consume a lot more traffic than 200 GB a month. But then again, it depends on your average request size.

A site I am operating has around 13.000 unique visitors a day. That site consumes around 280 GB traffic a month.

To conclude it all, to me it sounds like you are not currently fit to go in production with a website hosted on a server you operate. I am not trying to be judgmental, but the questions you have asked gives me a hint.

Bringing that amount of visitors to your site requires massive marketing. If I were you I would put my site on a shared webhost that complies with your requirements and then maybe purchase a small VPS and start learning.

Answer 2

Firstly, to answer your questions: In terms of functionality, you should find little difference between a dedicate server, a VPS, or a cloud computing (e.g. Amazon EC2) solution. All will offer you root access, and let you install whatever you need and setup the configurations to your needs. The differences between them lie in way resources are distributed between users (and the cost) - and in the case of cloud solutions, the ease of deployment.

Apache, MySQL, and PHP (and GD) is a very common setup - and all the specific configurations for PHP are easily made in php.ini. (By the way, GD is for image manipulation - e.g. generating CAPTCHAs, random avatars, thumbnails, etc.)

As for user generated content - one of the considerations, is the required processing. Image manipulation (even creating thumbnails in large quantities), converting videos, etc. require a lot of processing (both CPU and RAM)- and a VPS is unlikely to handle that strain. Moreover, user generated content can amount to a lot of storage depending on the type. There is a solution to storage in the form of cloud storage (e.g. Amazon S3, which you can mount as local storage with S3fuse) which would give you unlimited storage - where you pay for whatever you use.

Incoming traffic refers to all data that goes to your server, while outgoing traffic is the data that your server transmits. Incoming traffic includes:

• Downloading software (e.g. Apache, MySQL)
• User uploading content (e.g. images, videos, etc)
• User requesting a page (the request contains some headers, additional data (e.g. GET/POST), cookies, etc)
• FTP content uploaded to your server

Outgoing traffic includes:

• All the content your server generates - all the pages, images, etc that are viewed on your site
• Files downloaded from your server
• Emails sent from your server

Come up with an estimate for your average page size. The average web page today is over 600kB - but let's say you have really optimized pages with great caching, and only send 100kB per page view. At 1 million visitors a day (views will hopefully be more than visitors) - you are looking at 100kB * 1M = 100GB/day - So around 3TB/mo, very conservatively - more likely closer to 10TB/mo.

A fast running site depends on a few things - firstly, the complexity of the script - a simple script or a static file will always be served faster than a complex dynamic script. Secondly, the degree of optimization, if you are looking at millions of hits a day, you want to cache whatever dynamic content you can to lessen the loads on your server. Finally, the hardware you run on - running a dynamic (PHP-MySQL) site on a single VPS server with 512MB of RAM (and Apache) and trying to serve a few million hits a day is unlikely at best.

I've run a PHP-MySQL site getting 60k hits/day fairly easily on a VPS with 1GB of RAM - but the user content was primarily text based, not images/videos).

1 million hits a day is around 11.5 hits per second. If each request takes your server 1 second to process, that means you need on average 12 apache threads running. If each take 30 MB of RAM - you have already used 360MB (a good estimate would be that peak traffic is at least 3 times higher - which this VPS would not be able to handle), you also need to consider MySQL which is going to need a healthy bit of memory to server this quantity of requests. Most sites with database intensive applications getting high volumes of traffic will have database servers separate from their application servers.

Bottom line: your VPS will certainly run your script - but in all liklihood, it won't be able to serve a few million hits a day.

Now for the flip side of the argument - live for today, dream for tomorrow.

Not to be cynical, but you said 'this is your very first site' - given that, a few million hits a day is rather optimistic. Unless you have some definite assurances of such traffic (an established user base, a viral marketing campaign, a contract with a significant web presence, etc), it is likely that a few hundred - or at the outset, a few thousand hits a day would be something to start focussing on. Planning for something of the few million hits a day scale without some certainty of its occurrence is going to result in you wasting your resources on something that may not be needed for a long time. Moreover, you will always be able to scale up later. Hopefully as your traffic increases, so does your revenue, and you can look into larger deployments.

Finally a few general recommendations: Not to make assumptions, but the nature of your question suggests that you are just starting off with servers. If you are looking to setup your own server(s), I would suggest a great way to learn is by getting a copy of Virtualbox, and setting up a CentOS server (or Ubuntu if that is more to your liking). Install PHP, Apache, MySQL, and setup your script - test it out and see how things work, and resolve any difficulties you run into. The advantage of this is that it is free - you can take your time to learn and iron out the wrinkles without paying for hosting, you will either find it is easy enough to do or that you need to investigate other approaches. For a good guide, I would suggest the Perfect Server tutorials from HowToForge.

If you are really looking for high concurrency, and don't mind putting in a bit of extra effort, you will find that nginx and php-fpm should offer you a better solution for running on a VPS (the rewrite rules that Apache uses can usually be translated to those that Nginx recognizes). If you do decide to go with Apache though, you definitely want to look into FastCGI - especially php-fpm instead of using mod_php. Also you may look into Varnish to cache your dynamic pages once your page views start increasing significantly.

Another option to look into are some of the cloud offerings. The advantage is that they can scale with your traffic - when you start out small, you will incur a fairly small monthly cost - as you grow, your cost will increase (but hopefully so will your revenue) - they are excellent to learn with (very easy to take snapshots, restore something if you make a mistake, launch more servers if demand requires, etc). The big downside is that you usually can't cap your costs - if your site actually does end up using 10TB of bandwidth, you will end up with a very hefty bill (over $1200, just for bandwidth) - you definitely want another solution if you are in that scenario. However, those servers (e.g. Amazon EC2, Rackspace, etc) are great to learn on and run for just the time you need them.