Is more than a virtual dedicated server needed initially for a user-generated video sharing site?

Question

I'm building a hybrid flash and HTML5 user-generated video site. I'm a novice but have some more knowledgeable developers helping me. I've bought a virtual dedicated server with 30 GB of disk space and 500 GB of bandwidth. My expectation is that we'll get 1000 videos uploaded and 10,000 downloaded year 1. Each video will be 3-5 minutes long - not high def for the most part. We don't have much money to spend. Do I need to get a storage solution like S3 and a content delivery network like Cloudfront or will I probably be ok that first year just using the VDS?

Answer 1

Ok, let's work this out.

You don't mention what codec or bit-rate you're going to use so I'm going to assume you pick something like H.264 at around 1.5Mbps (as that's what I do for SD content incidentally).

So 1.5Mbps works out at around 187.5KBps, multiply this by 60 for 60 seconds in a minute, then by 4 for the average between a 3 and 5 minute clip means each played back clip will need a bare minimum of 45MB of downstream bandwidth each - not including any signalling or general site usage (catalogue, front page etc.).

Multiply this by your 10k playback estimate and you get 450GB per year or, if extremely evenly distributed across the year, 35.5GB per month.

Now you don't state what your 500GB allowance really is, is that per month? if so then you should be fine as you'll be using less than 10% of your allowance but be aware that your estimate may be very wrong, what if 10M users come to your site, what are your plans then? also the other thing to consider is that your users WON'T be smoothly distributed across the year, you'll see huge bubbles of super busy periods and then nothing at all - you need to have worked this out.

Given you have no money I'd go ahead with what you have but you need to start designing your next phase, where do you go once this server is overused, how do you migrate to this next phase, where is your data at this point, how do you ensure catalogue coherency, how will you load-balance etc.

Video delivery is just about one of the most complex IT problems we face today, I know, it's my job, and I'm pleased you're giving it a go but you need to think about tomorrow today.

Answer 2

I've bought a virtual dedicated server

Lets stick to reality here. You RENT a virtual server. No buy. Nothing dedicated here, either.

What you say is like saying "I purchased a dedicated taxi for a trip".

My expectation is that we'll get 1000 videos uploaded and 10,000 downloaded year 1. Each video will be 3-5 minutes long - not high def for the most part. We don't have much money to spend. Do I need to get a storage solution like S3 and a content delivery network like Cloudfront or will I probably be ok that first year just using the VDS?

Well, depenmds totally on what you do on the server and how the load distributes. If you reencode you may be in for a bad surprise CPU wise.

500gb a month (that is per month, right)? can and can not be good enough - depends on how large a video file is (which you do not say - but it depends on resolution and bit rate, which you possiobly control). You also dont say whether you can up the VPS or not.

• If you do not reencode CPU will likely not be an issue.
• If you do you are dead when people mass upload unless you can handle that and people are willing to wait.
• Bandwidth should be easy to scale. As should be RAM, CPU for low CPU cenarios (no reencoding). VPS can go quite high, thoug not all providers support that.

Finally: noone cares what money you ahve or not. Saving is nice, bubut if you can not run a site, you can not.

In general: a site like you describe is not even worth metioning from the CPU / Memory point of view, bandwidth may be. Very small. 10.000 downloads a year, over 365 days are on average 27.4 per day. About 1.1 per hour. Say peak is 5 times - that is 6 downloads per hour. The server may send you an email about boring jobs and falling asleep.

OTOH make that bleu ray level discs which you want to reencode and you arein for pain CPU wise.

You have to say - our crystal balls are broken.