Stack Overflow
Stack Overflow

Questions tagged [consistent-hashing]

Consistent Hashing, a process discovered by Karger et al. at MIT, is a special kind of hashing such that when a hash table is resized only K/n keys need to be remapped unlike normal hashing techniques

Definition

Consistent hashing is a special kind of hashing such that when a hash table is resized and consistent hashing is used, only K/n keys need to be remapped on average, where K is the number of keys, and n is the number of slots. In contrast, in most traditional hash tables, a change in the number of array slots causes nearly all keys to be remapped.

Consistent hashing is used in many places. The most famous being OpenStack's Object Storage component called as Cinder

Source

Wikipedia page on Consistent Hashing

126 questions
0
votes
2 answers

Implement Consistent Hash Algorithm in Java

I am trying to implement a consistent hash based algorithm in java using the following reference for sharding keys to redis - Stanford Theory on CH I am trying to understand the best way to generate the hascode for a node and a key. Currently I am…
user1619355
  • 429
  • 1
  • 4
  • 17
0
votes
1 answer

AKKA router using consistenthashing as routing but needs to deliver message to specific actor

I am trying to find a solution where I have AKKA Actor instance containing specific key value pair as instance data. What I need is to be able to update the instance data by targeting it with the key it has. One Actor will have one key and it is…
Ram
  • 325
  • 4
  • 22
0
votes
2 answers

Cluster sharding client not connecting with host

After recent investigation and a Stack over flow question I realise that the cluster sharding is a better option than a cluster-consistent-hash-router. But I am having trouble getting a 2 process cluster going. One process is the Seed and the other…
S. Harrap
  • 31
  • 3
0
votes
1 answer

Does Nginx respect the weight attribute with consistent hashing?

More specific, will this work ? upstream backend { hash $request_uri consistent; server backend1.example.com weight=1; server backend2.example.com weight=2; } will backend2.example.com receive twice as much traffic ? Also, what happens…
mihaic
  • 135
  • 1
  • 7
0
votes
1 answer

Akka ConsistentHashingRoutingLogic not routing to the same dispatcher thread consistently

I am trying to use Akka's ConsistentHashingRoutingLogic to guarantee that messages with the same key are routed to the same Actor. It is important that messages with the same key are processed in FIFO ordering. Messages with different keys can be…
jbx
  • 21,365
  • 18
  • 90
  • 144
0
votes
0 answers

Partition data consistently among worker processes

I am looking for ideas on how I can equally distribute work on among predefined worker processes I have an array of 'n' documents stored in Mongo and 'm' worker processes Currently I have written an algorithm which is processed by each worker on…
Laavaa
  • 575
  • 9
  • 19
0
votes
1 answer

Does using many small tables in Cassandra lower the free disk space requirements?

Presently, we have tables which are over 100 gigabytes each. It was communicated to me that if we had many small tables this would be easier to handle. The idea being that Cassandra requires an amount of free space approximately equal to the table…
Eric Urban
  • 3,671
  • 1
  • 18
  • 23
0
votes
1 answer

Optimizing keyspace partitioning during node joins in a distributed hash table

When a node joins a DHT network, it seems optimal for the new node to evenly divide the largest interval on the consistent hash's circle in order to minimize remapping. However, this is only optimal for 2n nodes (assuming with start with n=1); all…
ZachB
  • 13,051
  • 4
  • 61
  • 89
0
votes
1 answer

Consistent hashing SHA1 modulo operation

I hope some guru here could help me out I am writing a C/C++ code to implement Consistent Hashing using SHA1 as the hashing algorithm. I need to implement module operation as follow : 0100 0011....0110 100 mod 0010 1001 = ? If the divisor (0000…
tmh1999
  • 63
  • 5
0
votes
1 answer

Do I need to checksum the file at browser side then checksum and compare them at server side?

In my website , some important files will be uploaded, do I need to calculate the file's checksum with JavaScript at the browser side, then calculate again with golang and compare them at the server side to make sure the files got by server are…
Suge
  • 2,808
  • 3
  • 48
  • 79
0
votes
1 answer

Add rings to consistent hashing circle to represent data

I am implementing consistent hashing and hence drawing a circle with sectors as shown in the Circle Demo. The sectors represents the Nodes. The HTML withing which my Circle resides is :
fnaticRC ggwp
  • 955
  • 1
  • 11
  • 20
0
votes
1 answer

Could anyone help provide a tutorial on hashing?

Recently I've read some papers on hashing techniques. It seems that hashing is everywhere. In computer science, the hash table is commonly used as a efficient look-up data structures. In encryption, the hashing is in the techniques such as md5…
mining
  • 3,557
  • 5
  • 39
  • 66
0
votes
4 answers

Secure integer hashing for order number

Let say I have a table Orders with auto-incremented id, e.g. 1, 2, 3, 4..., and they are current queried as http://www.example.com/order?id={1,2,3..} Now, I want to hash primary key [1, 2, 3, ..] into another number called Order Number so our…
Ryan
  • 10,041
  • 27
  • 91
  • 156
0
votes
1 answer

Is there a way to generate hash integer within a predefined range for a string in PostgreSQL?

For sharding I need good hash algorithm. I need to generate hash integer value for string within 0-2^31 range. Is this possible ? Following answer give a way to generate integer from md5 algorithm. Hashing a String to a Numeric Value in…
Viraj
  • 5,083
  • 6
  • 35
  • 76
0
votes
0 answers

Evenly distributing byte array into buckets

I'm trying to implement a minimal perfect hash function, and one of the primary ways to to this is to be able to take the known universe of keys and split them into decently even buckets of a small size (the smaller the better within reason). The…
Daniel Imberman
  • 618
  • 1
  • 5
  • 18
1 2 3
…
8
9