Stack Overflow
Stack Overflow

Questions tagged [consistent-hashing]

Consistent Hashing, a process discovered by Karger et al. at MIT, is a special kind of hashing such that when a hash table is resized only K/n keys need to be remapped unlike normal hashing techniques

Definition

Consistent hashing is a special kind of hashing such that when a hash table is resized and consistent hashing is used, only K/n keys need to be remapped on average, where K is the number of keys, and n is the number of slots. In contrast, in most traditional hash tables, a change in the number of array slots causes nearly all keys to be remapped.

Consistent hashing is used in many places. The most famous being OpenStack's Object Storage component called as Cinder

Source

Wikipedia page on Consistent Hashing

126 questions
6
votes
4 answers

why do we need consistent hashing when round robin can distribute the traffic evenly

When the load balancer can use round robin algorithm to distribute the incoming request evenly to the nodes why do we need to use the consistent hashing to distribute the load? What are the best scenario to use consistent hashing and RR to…
user3718420
  • 151
  • 1
  • 3
  • 14
6
votes
1 answer

Consistent hashing as a way to scale writes

I am trying to figure out if I am on the right track. I am building a (real-time) statistics/analytics service and I use redis to store some sets and hashes. Now let's assume I have some success and I need to scale out. The hash ring technique looks…
Sergio Tulentsev
  • 226,338
  • 43
  • 373
  • 367
6
votes
2 answers

Stale data handling with memcached/consistent hashing

Assume I have two memcached nodes (node A, B) at the beginning, and when I add a new node C, a portion of the keys are remapped and thanks to consistent hashing only some of them. Let's assume a value with key "foo" originally at server A is now…
Ryan
  • 10,041
  • 27
  • 91
  • 156
5
votes
4 answers

If consistent hash is efficient,why don't people use it everywhere?

I was asked some shortcommings of consistent hash. But I think it just costs a little more than a traditional hash%N hash. As the title mentioned, if consistent hash is very good, why not we just use it? Do you know more? Who can tell me some?
jingege
  • 51
  • 3
5
votes
1 answer

Difference between replicas and virtual nodes in consistent hashing

This is perhaps specific to an implementation that I'm looking at (node-hashring), but what is the difference between virtual nodes (vnodes) and replicas in a consistent hash ring? The original Akamai paper does not seem to describe vnodes…
ZachB
  • 13,051
  • 4
  • 61
  • 89
5
votes
0 answers

Algorithm to rebalance virtual nodes in consistent hashing solution in PostgreSQL?

I'm using PostgreSQL 9.3 and I have implemented Consistent hashing with virtual nodes using plproxy. In current setting node addition and rebalancing among nodes is working ( data movement between servers is done using PostgrelSQL foreign data…
Viraj
  • 5,083
  • 6
  • 35
  • 76
5
votes
0 answers

Ensuring a hash function is well-mixed with slicing

Forgive me if this question is silly, but I'm starting to learn about consistent hashing and after reading Tom White blog post on it here and realizing that most default hash functions are NOT well mixed I had a thought on ensuring that an arbitrary…
Slater Victoroff
  • 21,376
  • 21
  • 85
  • 144
4
votes
2 answers

Consistent hashing, why are Vnodes a thing?

My understanding of consistent hashing is that you take a key space, hash the key and then mod by say 360, and place the values in a ring. Then you equally space nodes on that ring. You pick the node to handle this key by looking clockwise from…
Luke De Feo
  • 2,025
  • 3
  • 22
  • 40
4
votes
0 answers

How to solve hot partition problem in consistent hashing Load Balancing

I am trying to understand how different Load Balancing strategy works. One way is to use consistent hashing algorithm where we divide the entire space into multiple virtual nodes and each physical node takes a set of vnodes. I am not able to…
Bishnu
  • 383
  • 4
  • 14
4
votes
2 answers

Understanding Consistent Hashing

I've been looking at consistent hashing algorithms for PHP over the past couple of days. I wish to gain a better understanding of how consistent hashing actually works, so I can utilize it on some upcoming projects. It appears to me that Flexihash…
EstelS
  • 153
  • 1
  • 6
4
votes
2 answers

Rendezvous vs consistent hashing

I read on Wikipedia: Unlike consistent hashing, HRW (Highest Random Weight, aka Rendezvous Hashing) requires no precomputing or storage of tokens. Why? My understanding is that: In consistent hashing, one hashes objects and sites independently…
Josh
  • 11,979
  • 17
  • 60
  • 96
4
votes
1 answer

Consistent hashing domino effect

I'm trying to figure out how a system like Cassandra which uses consistent hashing handles cascading node failures. I know there's this concept of virtual nodes and keys are mapped to virtual nodes. Virtual nodes are in turn mapped to actual…
frodo
  • 1,561
  • 3
  • 21
  • 36
4
votes
1 answer

Why does elasticsearch still use simple routing value using modulo?

Just wondering why elasticsearch still use that simple routing value approach for deciding which shard the data must be stored to. Actually this approach is limiting us to change the number of shards in the future. If elasticsearch uses an approach…
indraep
  • 143
  • 1
  • 1
  • 9
4
votes
2 answers

Different results from Murmur3 from Scala and Guava

I am trying to generate hashes using the Murmur3 algorithm. The hashes are consistent but they are different values being returned by Scala and Guava. class package$Test extends FunSuite { test("Generate hashes") { println(s"Seed =…
Saket
  • 3,079
  • 3
  • 29
  • 48
4
votes
6 answers

Does python-memcached support consistent hashing & binary protocol?

Python-memcached is the official supported memcached driver for Django. Does it support Consistent hashing Binary protocol If it does, how do I use those features within Django? I couldn't find any documentation.
Continuation
  • 12,722
  • 20
  • 82
  • 106
1
2
3
…
8 9