Stack Overflow
Stack Overflow

Questions tagged [consistent-hashing]

Consistent Hashing, a process discovered by Karger et al. at MIT, is a special kind of hashing such that when a hash table is resized only K/n keys need to be remapped unlike normal hashing techniques

Definition

Consistent hashing is a special kind of hashing such that when a hash table is resized and consistent hashing is used, only K/n keys need to be remapped on average, where K is the number of keys, and n is the number of slots. In contrast, in most traditional hash tables, a change in the number of array slots causes nearly all keys to be remapped.

Consistent hashing is used in many places. The most famous being OpenStack's Object Storage component called as Cinder

Source

Wikipedia page on Consistent Hashing

126 questions
4
votes
2 answers

Memcached consistent hashing & network partitioning, how to solve?

My understanding is when using memcached in "consistent hashing" mode, when a node is down, the clients will remap the key with the algo. Consider I have 2 clients (A,B) and two servers (C,D) What if client A think server C is down, but B think C is…
Howard
  • 19,215
  • 35
  • 112
  • 184
4
votes
1 answer

Consistent hashing: Where is the data-structure of ring kept

We have N cache-nodes with basic consistent-hashing in a ring. Questions: Is data-structure of this ring stored: On each of these nodes? Partly on each node with its ranges? On a separate machine as a load balancer? What happens to the ring when…
Ivan Voroshilin
  • 5,233
  • 3
  • 32
  • 61
3
votes
2 answers

Is there any disadvantage of Consistent Hashing?

Admittedly, consistent hashing is a widely used technology in distributed caching applications. It offers a good solution when the number of nodes changes dynamically. And when the virtual node is combined, the load balancing problem will also be…
Geni
  • 687
  • 3
  • 10
  • 22
3
votes
1 answer

Python hash_ring not distributing uniformly, what are consistent hashing alternatives?

I'm using hash_ring package for distributing objects among servers. I've assumed that distribution would be uniform, as it's based on MD5 hashes. Unfortunately it's not the case. I'm using random keys which are generated using uuid.uuid4(). I've…
vartec
  • 131,205
  • 36
  • 218
  • 244
3
votes
2 answers

Is relational database able to leverage the way of consistent hashing to do the partition table?

Assume we have a user table to be partitioned by user id as integer 1,2,3...n . Can I use the way of consistent hashing used to partition the table? The benefit would be if the number of partitions is increased or decreased, old index can be the…
Clark Bao
  • 1,743
  • 3
  • 21
  • 39
3
votes
0 answers

what is the difference between consistent hashing and sticky sessions?

My understanding of consistent hashing is that it is used to minimize rehashing overhead and that request is served to the next nearest worker in the ring to the hash value of the request. So this guarantees same requests to hash to the same worker…
Sameer
  • 757
  • 1
  • 14
  • 35
3
votes
1 answer

Consistent Hashing to a Dynamic Set of Servers Based on TCP Payload

I have a scenario where I need to route TCP traffic to a dynamic set of backend servers (Kubernetes pods to be exact but we can ignore that for purposes of this post) through a proxy like HAProxy or nginx. The traffic needs to be routed based on a…
Lee Hampton
  • 410
  • 5
  • 13
3
votes
1 answer

Zookeeper-Kafka and Consistent hashing

I am learning Zookeeper and I was stuck in middle with some confusion. I gone through various forums and questions and none clear my confusion and came to SO finally to get some clarification on the following things. As I understand Zookeeper…
Jagan
  • 63
  • 1
  • 7
3
votes
2 answers

Consistent Hashing: what about rehashing?

As you may know, consistent hashing is a great idea when dealing with DHT. The main idea is to not suffer too much when a new node is added or deleted. From the original Paper: When a machine is added to or removed from the set of caches, the…
santiagobasulto
  • 11,320
  • 11
  • 64
  • 88
3
votes
1 answer

How to checksum the file to be uploaded with javascript?

I want to checksum the files at browser side before uploading, then checksum and compare at server side to make sure the consistent. But how can I get the pure binary data of the file and checksum it? I tried the way below, but doesn't work: let…
Suge
  • 2,808
  • 3
  • 48
  • 79
3
votes
1 answer

What is the difference between consistent hashing and cone hashing?

What I know is: Consistent hashing: uniform distributed storage system Cone hashing: non uniform distributed storage system I want to know: How it works? What is the use of it? What is the difference between this two types of hashing? I am not…
Hemanth Venkatappa
  • 39
  • 1
  • 1
  • 8
3
votes
4 answers

Can circular hashing remain consistent as the target set grows?

Circular hashing algorithms provide consistency given a static set of targets. For instance: I have an initial set of targets, let's call them A, B and C. I have a key, let's call it x I have a circular hashing function, let's call it hash(key,…
Jonathan Azoff
  • 171
  • 4
3
votes
2 answers

Is there a guaranteed fair variation on consistent hashing?

I'm looking for something like Consistent Hashing, but with a guarantee that a distribution ends up as fair as possible (not just on average for random keys) - is there such a thing and where can I find it if so? Edit: In my specific case, the set…
SoftMemes
  • 5,602
  • 4
  • 32
  • 61
2
votes
1 answer

Does a Cassandra node get assigned a new token every time it restarts?

From my limited knowledge, Cassandra assigns a random token for every new node in the ring. The ring position is important because data is replicated in the SimpleStrategy according to the position. So what happens when the node restarts and wants…
Mainul Islam Mahi
  • 35
  • 3
2
votes
1 answer

Using custom hash function with HAProxy LoadBalancer

I have configured a load balancer using HAProxy on frontend and on the backend I have 4 server serving the request using Apache web server. #HAProxy for web servers frontend web-frontend bind IPADDRESS_LB:80 mode http default_backend…
user10798329
1 2
3
…
8 9