simple question here, I know these large sites like digg etc. have a wall of memcache servers that handle requests before hitting their actual master/slave db's.
What would these servers look like, would they be just bare bone linux servers with like 32GB of ram?
Are there any security concerns that need to be addressed? I was just thinking if someone could somehow grab objects from memcache, you could very well have a User object stored that has the users username/pwd in it!
There are a lot of ways you can set up your memcached servers depending on the way your application works. For very large-high traffic sites that can do a lot of caching, your scenario may be good. In our case we actually use the front-end webs in our memcache cluster.
Regarding security, since memcached has none, you need to make sure that access to the ports it uses are properly firewalled. You could also use your own ports rather than the defaults. The only way then that someone could access your objects would be to compromise your server that accesses memecached, in which case you probably have bigger problems.
Memcache doesn't answer HTTP requests directly, it has no capacity to do so; the way I've set this up is to use nginx's memcache module to retrieve pages directly from cache if available, talking to memcache instances on a private network.
I typically don't use dedicated machines for memcache, but rather look at the resource usage profiles of existing servers and put memcached instances where there's spare memory capacity. For instance, file storage servers typically need a lot of disk IO but often don't use all their RAM (once you've filled RAM with your active set of files, more memory for disk cache is wasted), whilst memcached uses RAM but no disk IO -- a perfect set of complimentary use cases. As a site gets larger, though, dedicated machines running with lots of RAM, small CPU, and probably PXE booted (so there's no local HDDs, and you can provision new ones in near-zero time) make sense as memcached servers.
