5

What has been happening with the four big P2P distributed hash table (DHT) overlay networks — Pastry, CAN, Chord, and Tapestry — since they all came out in 2001?

I know that academic projects continued for a few years, and sporadic maintenance releases still appear for some of them, but did any ever end up in major-scale, non-academic use? Is there still an active dev community around any of them?

I've take a few journeys through Google and Wikipedia, but there's no real information about what has happened recently, and their web sites are all moribund.

Update: I see that Chimera (the successor to Tapestry) is still under active development, with recent research publications: http://current.cs.ucsb.edu/projects/chimera/index.html

Update #2: given someone's -1 for the question, I should be more clear about the programming side - I'm interested in a general-purpose P2P overlay-network library and associated standards that would make a firm foundation for a P2P social networking app. All the existing ones I've looked at, including Chimera, seem too weakly-developed and supported and/or too out-of-date to form a solid infrastructure layer. I'd like to know what other options I have.

Update #3: Mainline DHT seems to generate a few questions here. It's based on Kademlia, and as far as I now, has specialized use mainly as a distributed search protocol for Bittorrent.

Canuck
  • 565
  • 4
  • 13
  • 1
    This is a perfectly reasonable question, and the OP has clearly done some research. Not sure what the -1 was for, so +1. – halfer Oct 26 '11 at 15:49
  • The BitTorrent DHT has ~7M nodes, I would say that counts as major-scale use. It's not used for searches as you say though, it's simply a lookup mechanism to find peers participating in a specific swarm (identified by an SHA1 hash). Some other filesharing networks use DHTs too. – the8472 May 06 '12 at 20:26
  • 1
    The response (or lack thereof) to this question is pretty disappointing. I can hardly believe that the only large scale use of p2p overlays is in bittorrent and the old version of skype... Also, fwiw, chimera's last release seems to be from 2008; hardly active development :/ – Frew Schmidt Jul 21 '13 at 16:02
  • Thanks for the comment, Frew. I've given up on this area for now, but here's hoping some smart PhD student, postdoc fellow, or Googler with 20% time stumbles on this page, and realizes what a great opportunity this is get do some high-profile R&D that actually matters. – Canuck Jul 21 '13 at 20:32

1 Answers1

2

did any ever end up in major-scale, non-academic use?

DHT's were designed to solve a set of issues which were described in a theoretical/abstract way. In 2001, people did not know much about the practical aspect of P2P communication. Proper NAT traversal solutions were not available (or even understood correctly).

None of the DHT designs addressed the practical aspect of P2P communication (i.e., How do I actually contact another device over the Internet? How do I find it? How do I establish a connection to it? How do I deal with NATs, Proxies, public and private addresses? etc...).

Hence, there was not enough 'context' for them to thrive, which explains their current status. Today, all the pieces of the puzzle/solution are available, but no one has connected them in a unified framework.

I wish I had more time to finish the Chaupal project.

Jérôme Verstrynge
  • 57,710
  • 92
  • 283
  • 453
  • Thanks - I suspect your answer saves me from a few dead ends. – Canuck Nov 04 '11 at 02:35
  • @Canuck Let's say, the theoretical results suggested by DHT's won't necessarily meet practical measurements, because they rely on an idealized world. That being said, some still suggest interesting solutions to real issues. It's just that DHT's are not a complete P2P solution/framework. – Jérôme Verstrynge Nov 04 '11 at 03:35