2

I recently inherited a small Linux-based load-balanced web server set up. Unfortunately, it was never completely brought on-line, so I am working on doing that.

The basic set up is:

  • LVS-based load-balancer. Also acts as a simple NAT for the web servers.
  • Two Apache/PHP5 web servers.
  • MySQL database server.

The database server is inside the main company firewall while the other three are in the DMZ. Basic load balancing works, but I've had trouble persisting connections with LVS and since session storage for PHP5 is using local files, we lose sessions. I might be able to get LVS to persist sessions, but since connections are generally to large numbers of NATed clients behind the same firewall with sometimes lengthy delays between requests, I think it might be a losing proposition. I've decided to instead switch to memcached for session storage, as it will be quick to set up.

My question with this is where should the memcached service live? I cannot add another server so it has to live either with the load-balancer or the database server. At the moment, I'm leaning toward putting it with the database server, as it has no external connections, but I'm concerned the two might conflict to some extent. Is there a best-practice for this sort of situation? Or any other guidance?

Michael Johnson
  • 148
  • 6

2 Answers2

2

If I were you, I would save the memory on the database server for the database. For resilience, I would install memcached on both your webservers. One small problem with using memcached for sessions is that restarting your memcached will destroy your sessions. You might look at something like memcachedb instead, which persists objects.

David Pashley
  • 23,497
  • 2
  • 46
  • 73
  • Sharding across the webservers is an excellent idea. – womble Oct 20 '09 at 08:48
  • Would this not produce inconsistent session failures if one of the webservers was to disappear? – Dan Carley Oct 20 '09 at 09:03
  • Yes, but only half your sessions if you use memcached. Memcachedb also replicates your objects, which would solve this problem. – David Pashley Oct 20 '09 at 10:27
  • Half of your session data is a considerable amount to lose. I would say that only makes memcachedb viable. – Dan Carley Oct 20 '09 at 10:49
  • The only problem I have with this is it is actually the web servers that are memory hungry, not the database server. Traditionally, the database server has been underused while the single web server this arrangement is replacing easily used 8GB. However, I like the idea of splitting it between the two web servers, so I'll look into that. – Michael Johnson Oct 20 '09 at 13:32
  • 1
    Usually, you want to configure your database so as much of it can fit in memory. If your dataset is considerably smaller than your memory, it might be a good use for it. – David Pashley Oct 20 '09 at 13:49
  • @David: I agree. But that machine has 16GB of RAM and the active set of the database is considerably smaller than that. Given the amount of data stored in the sessions and the expected number of users, I don't think memcached will be using more than a few MB. So we can still optimize the database's memory usage and shoehorn in memcached. – Michael Johnson Oct 20 '09 at 15:53
  • Sorry, that's what I meant when I said "it might be a good use for it." Too many "it"s . Should have read "[memcached] might be a good use for [the database server]". – David Pashley Oct 20 '09 at 17:21
  • I'm marking this answer as the solution, even though we're not going to use it. In general, I think it's right solution for this problem, especially going with memcachedb. But due to the unusual circumstance of our web servers needing more memory than the database server, it's just not quite right for us specifically. – Michael Johnson Oct 22 '09 at 02:49
-1

I would place it on the load balancer. For three reasons:

  1. That machine probably isn't utilising a lot of CPU/RAM at the moment (if that isn't the case then you'll need to rethink).
  2. If the load balancer goes down then sessions will be no good to you anyway.
  3. Sessions have a bit more of a logical association with connections than they do databases.
Dan Carley
  • 25,617
  • 5
  • 53
  • 70
  • Putting memcached on the load balancer is dangerous; if you misconfigure memcached (or it gets an unpleasant amount of load) you can do bad things to the performance of the load balancer. – womble Oct 20 '09 at 08:47
  • The same could be said for placing and misconfiguring it anywhere. – Dan Carley Oct 20 '09 at 08:52