Comments on: Consistent hashing http://michaelnielsen.org/blog/consistent-hashing/ Mon, 07 May 2012 21:35:26 +0000 hourly 1 http://wordpress.org/?v=3.3 By: Michael Nielsen http://michaelnielsen.org/blog/consistent-hashing/comment-page-1/#comment-30818 Michael Nielsen Sun, 26 Jun 2011 00:28:02 +0000 http://michaelnielsen.org/blog/?p=613#comment-30818 @ZD - thanks for pointing this out, it's now fixed. (Funny, some people have trouble with left and right, which I find very instinctive. But I always have to pause to think about clockwise versus counterclockwise, and I sometimes mess them up.) @ZD – thanks for pointing this out, it’s now fixed.

(Funny, some people have trouble with left and right, which I find very instinctive. But I always have to pause to think about clockwise versus counterclockwise, and I sometimes mess them up.)

]]> By: ZD http://michaelnielsen.org/blog/consistent-hashing/comment-page-1/#comment-30817 ZD Sat, 25 Jun 2011 16:45:41 +0000 http://michaelnielsen.org/blog/?p=613#comment-30817 Found your post via "crazy websurfer" logic via <a href="http://www.reddit.com/r/programming/comments/i8k4l/write_your_first_mapreduce_program_in_20_minutes/" title="Write your first mapreduce program in 20 minutes" rel="nofollow">Reddit</a>. In your text you say "counterclockwise" for your first example, but then proceed to distribute the key to the next <i>clockwise</i> machine, machine 1; is this a mistake or am I missing something? Found your post via “crazy websurfer” logic via Reddit.

In your text you say “counterclockwise” for your first example, but then proceed to distribute the key to the next clockwise machine, machine 1; is this a mistake or am I missing something?

]]> By: Pregel | DDI http://michaelnielsen.org/blog/consistent-hashing/comment-page-1/#comment-30784 Pregel | DDI Fri, 17 Jun 2011 21:20:50 +0000 http://michaelnielsen.org/blog/?p=613#comment-30784 [...] To implement Pregel on a cluster, we need a way of assigning vertices to different machines and threads in the cluster. This can be done using a hashing scheme, as was done in the code above to assign vertices to different worker threads. It can also be done using other approaches, if desired, such as consistent hashing. [...] [...] To implement Pregel on a cluster, we need a way of assigning vertices to different machines and threads in the cluster. This can be done using a hashing scheme, as was done in the code above to assign vertices to different worker threads. It can also be done using other approaches, if desired, such as consistent hashing. [...]

]]> By: SQL Server Scale Out Solutions | Brent Ozar PLF http://michaelnielsen.org/blog/consistent-hashing/comment-page-1/#comment-30726 SQL Server Scale Out Solutions | Brent Ozar PLF Wed, 08 Jun 2011 13:00:54 +0000 http://michaelnielsen.org/blog/?p=613#comment-30726 [...] of evenly distributing data (roughly) across that fixed length output value. This is usually called consistent hashing. However the consistent hashing is generated, it’s easy to divide the total range of hashed [...] [...] of evenly distributing data (roughly) across that fixed length output value. This is usually called consistent hashing. However the consistent hashing is generated, it’s easy to divide the total range of hashed [...]

]]> By: Michael Nielsen http://michaelnielsen.org/blog/consistent-hashing/comment-page-1/#comment-30320 Michael Nielsen Wed, 23 Feb 2011 23:08:02 +0000 http://michaelnielsen.org/blog/?p=613#comment-30320 I don't see why more replicas will imply more rehashing. It's true that more machines will be involved in the rehashing, but each machine will rehash fewer keys, and the total number of keys rehashed will be the same. I don’t see why more replicas will imply more rehashing. It’s true that more machines will be involved in the rehashing, but each machine will rehash fewer keys, and the total number of keys rehashed will be the same.

]]> By: Santiago http://michaelnielsen.org/blog/consistent-hashing/comment-page-1/#comment-30317 Santiago Wed, 23 Feb 2011 15:34:37 +0000 http://michaelnielsen.org/blog/?p=613#comment-30317 Hello Michael. I've been thinking about the replicas "dilema". It's obvious that more replicas will make the keys to "distribute" better. Here you can find a pretty straightforward chart: http://www.lexemetech.com/2007/11/consistent-hashing.html The thing is that, if you have to add or remove a node, then, more replicas will imply more rehashing between the nodes. A big number of replicas would mess the consistent hashing "concept". What do you think? Hello Michael. I’ve been thinking about the replicas “dilema”.

It’s obvious that more replicas will make the keys to “distribute” better. Here you can find a pretty straightforward chart: http://www.lexemetech.com/2007/11/consistent-hashing.html

The thing is that, if you have to add or remove a node, then, more replicas will imply more rehashing between the nodes. A big number of replicas would mess the consistent hashing “concept”.

What do you think?

]]> By: Michael Nielsen http://michaelnielsen.org/blog/consistent-hashing/comment-page-1/#comment-29761 Michael Nielsen Thu, 06 Jan 2011 13:11:53 +0000 http://michaelnielsen.org/blog/?p=613#comment-29761 HA: In the formula with each node responsible for at most (1+e)K/N keys, with e = O(log N), where does the number of replicas appear? The variation should decrease as the number of replicas increases. Do you have a bound on a constant c such that e < c log N? Without knowing the constant, I don't see how to tell whether this rule applies or not. Unfortunately, I haven't done a detailed analysis of the relationship between the number of replicas and the variation. HA: In the formula with each node responsible for at most (1+e)K/N keys, with e = O(log N), where does the number of replicas appear? The variation should decrease as the number of replicas increases. Do you have a bound on a constant c such that e < c log N? Without knowing the constant, I don’t see how to tell whether this rule applies or not.

Unfortunately, I haven’t done a detailed analysis of the relationship between the number of replicas and the variation.

]]> By: HA http://michaelnielsen.org/blog/consistent-hashing/comment-page-1/#comment-29688 HA Thu, 06 Jan 2011 03:34:18 +0000 http://michaelnielsen.org/blog/?p=613#comment-29688 With increased replicas, it gives a more uniform load balance of keys across machines. This is the distribution with 20 replicas, 8 machines, 256 keys: Machine 0 has 31 keys Machine 1 has 33 keys Machine 2 has 41 keys Machine 3 has 26 keys Machine 4 has 25 keys Machine 5 has 35 keys Machine 6 has 36 keys Machine 7 has 29 keys Is there a rule of thumb on how to choose number of replicas to get good load balancing of keys (for instance, less than 2X difference across machines) ? Chord paper (from SIGCOMM '01) mentions the following about consistent hashing - For any set of N nodes and K keys, with high probability each node is responsible for at most (1+e)K/N keys with a bound of e=O(logN). And e can be reduced to an arbitrarily small constant by having each node run O(logN) replicas. But that doesn't seem to apply for the example I'm using. Do you have any inputs on why that is? Thanks! With increased replicas, it gives a more uniform load balance of keys across machines. This is the distribution with 20 replicas, 8 machines, 256 keys:
Machine 0 has 31 keys
Machine 1 has 33 keys
Machine 2 has 41 keys
Machine 3 has 26 keys
Machine 4 has 25 keys
Machine 5 has 35 keys
Machine 6 has 36 keys
Machine 7 has 29 keys

Is there a rule of thumb on how to choose number of replicas to get good load balancing of keys (for instance, less than 2X difference across machines) ?

Chord paper (from SIGCOMM ’01) mentions the following about consistent hashing – For any set of N nodes and K keys, with high probability each node is responsible for at most (1+e)K/N keys with a bound of e=O(logN). And e can be reduced to an arbitrarily small constant by having each node run O(logN) replicas.
But that doesn’t seem to apply for the example I’m using. Do you have any inputs on why that is?

Thanks!

]]> By: Michael Nielsen http://michaelnielsen.org/blog/consistent-hashing/comment-page-1/#comment-29578 Michael Nielsen Wed, 05 Jan 2011 16:52:03 +0000 http://michaelnielsen.org/blog/?p=613#comment-29578 HA: What happens when you increase the number of replicas, say to 20? HA: What happens when you increase the number of replicas, say to 20?

]]> By: HA http://michaelnielsen.org/blog/consistent-hashing/comment-page-1/#comment-29489 HA Wed, 05 Jan 2011 00:08:16 +0000 http://michaelnielsen.org/blog/?p=613#comment-29489 Thanks for sharing your program. I tried it out and found that the distribution of keys to machines isn't balanced for N keys that are number 0...N For example, for 8 machines and 4 replicas, if we map 256 keys numbered 0..255 using this, we get following distribution of keys to machines: Machine 0 has 18 keys Machine 1 has 66 keys Machine 2 has 39 keys Machine 3 has 44 keys Machine 4 has 11 keys Machine 5 has 21 keys Machine 6 has 27 keys Machine 7 has 30 keys There is as much as 6X difference in number of keys assigned to machines. Am I missing something? How can the load balancing of keys across machines be made more uniform? Thanks for sharing your program. I tried it out and found that the distribution of keys to machines isn’t balanced for N keys that are number 0…N

For example, for 8 machines and 4 replicas, if we map 256 keys numbered 0..255 using this, we get following distribution of keys to machines:
Machine 0 has 18 keys
Machine 1 has 66 keys
Machine 2 has 39 keys
Machine 3 has 44 keys
Machine 4 has 11 keys
Machine 5 has 21 keys
Machine 6 has 27 keys
Machine 7 has 30 keys

There is as much as 6X difference in number of keys assigned to machines. Am I missing something? How can the load balancing of keys across machines be made more uniform?

]]>