0

I've read an article about Distributed Hash Tables and seems it's possible to implement such a thing like memcache with APC. As you know APC is much more faster than memcache if we're fetching keys from a single server. So if we make APC distributed we have both performance and distribution. I need some thoughts to start it. Could someone who is familiar with Hash tables explain how to do that? How to make APC like memcache?
If you know something about keyspace partitioning and Overlay network that would be much more better.

Alireza
  • 6,497
  • 13
  • 59
  • 132
  • 1
    Isn't APC just a whole lot faster exactly because it's not a server but a local chunk of memory set aside for aching/object storage. Ie, won't your idea invalidate itself by distributing apc? – fvu Jul 20 '12 at 18:09
  • @fvu, If it's faster just because it's on the localhost, then why memcache is not as fast as APC when it's on the localhost like APC? – Alireza Jul 20 '12 at 18:12
  • apc = chunk of memory that gets accessed, and that's it. Memcache is a server, ie localhost or not, there **is** network traffic involved, the client must encode a request, send it, the server decodes it, encodes its answer, sends it through the network and client decodes it. Now, "distributed apc" will have to be a server, and will very much resemble memcache's current model, and with the model inherit the behavior and performance levels, no way around it. Memcache is, for the work it does, already pretty fast, I really think you're looking for the wrong culprit here. – fvu Jul 20 '12 at 20:54
  • I think I'm going to re-cast that in an answer, and elaborate the under the hood part a bit, back in 15m – fvu Jul 20 '12 at 21:04

1 Answers1

1

Although at the surface both softwares provide a comparable service, their underpinnings are entirely different, and that explains the dramatic difference in performance.

APC is basically a system that allows you to store objects (be it user objects or parsed opcode chunks) in shared memory. Shared memory, in all systems I know, is as fast as local RAM once you obtained a pointer to it.

So, in short, what APC has to do to write or read an object is:

  • request shm access and obtain a pointer to it
  • calculate object offset and size in the shm
  • memcpy that memory zone into a buffer or vice versa
  • done

Simple, and taking into account that memory bandwidth nowadays is 10's of gigabytes per second, quick.

Due to its distributed nature in a memcache scenario more needs to be done:

  • client encodes and transmits request
  • server receives and decodes request
  • server calculates object offset and size in memcached's memory
  • server memcpy's that memory zone into a buffer or vice versa
  • server transmit buffer
  • client receives and decodes buffer

Now, if we want to distribute APC, the client and server will need to talk to each other. And all of a sudden we find ourselves in a scenario that, with the exception of a few less important details, is identical to the one used by memcache. And all the expensive operations will become necessary again, ie all the copying around, sending through the network stack included.

That's also an explanation why even with a memcache instance running on localhost, without horribly slow gigabit ethernet between the nodes, there is a considerable overhead in what needs to be done to make a distributed system work.

And that's why I'm convinced you're looking at the wrong suspect here, make APC distributed and it will be in the same performance/throughput category.

fvu
  • 32,488
  • 6
  • 61
  • 79
  • OTOH, it seems like it might be possible to speed up memcached if you could access it via SHM (falling back to the network if it's not there), though you might have to drop some consistency guarantees. – tc. Jul 20 '12 at 21:59
  • @tc. Can certainly be done, and eg Informix offers shm as an alternative communication path between server and client but the benefits of such a setup are hard to estimate (as it only makes sense in a multinode setup - otherwise just stick to APC - which means that a sizeable portion of total traffic will still go over the network), and code complexity would increase a lot. – fvu Jul 20 '12 at 22:27
  • Fair enough, convinced. I think the best approach would be to use both APC and memcache. APC for opCode caching and memcache for data caching(when we have multiple servers). – Alireza Jul 21 '12 at 04:35