For my current setup, I have a single client server using Tornado, a standalone database server and another standalone server for my website.
I'm looking at having a second client server process running on the same system (to take advantage of its multiple cores) and I would like some advice in locating which server my "clients" have connected to. Each client can have multiple connections (instances).
I've already looked at using memcached to hold a list of user identifiers and link them to which server(s) they are connected to, but that doesn't seem like it would scale very well (eg having six digits of connected users).
I see the same issue with database lookups. I have already optimized my server as much as possible, without going into micro-optimization and I personally frown upon that.
Current server methodology:
On connect: Accept connection, rate limit for max connections per IP. Append client instance to a list named "clientList".
On data from client: Rate limit for max messages per second. Append data to a client work queue. If client has a thread dedicated toward its work queue: return, its work will be chewed by the current thread otherwise create a new thread for this users work queue, start it.
TLDR: How do I efficiently store which servers a client has connected to in order to forward messages to that user.
