11

We are developing a system with the following requirements.

  • There are N systems that each generate data that is unique to themselves
  • Each system requires the data from every other system to perform its end goal
  • These systems are talking to each other on an unreliable network.
  • It is expected that some systems will be completely unavailable for extended periods of time (but they may be in contact with some of there peers who are in contact with the rest of the network)

To put it another way, each system needs to replicate its data to N peer systems. Ideally, this will be done in an intelligent manner.

I have considered looking into database synchronization frameworks, but I am concerned that it is overkill for this problem. I don't think there is any possibility for row conflicts because each system's data is entirely independent of other systems.

The question is, do you know of any frameworks that could help solve this problem? Or possibly a way to phrase this issue that might help me down a path to discover a solution.

Finally, ideally, this framework would be in C++ (and potentially, java).

Charles
  • 50,943
  • 13
  • 104
  • 142
Justin Breitfeller
  • 13,737
  • 4
  • 39
  • 47

5 Answers5

2

0mq. It is a C framework with a C++ interface. It notably supports EPGM (reliable multicast over UDP) and N-to-N connections. Though, there will be work to do for your special use case.

J.N.
  • 8,203
  • 3
  • 29
  • 39
  • Do you have any further pointers into how I might accomplish my task with this library? I started reading through the guide, but it has quite a few options available and I'm not sure where you think would be a good place to start. – Justin Breitfeller Mar 08 '12 at 15:40
2

SymmetricDS.org

The solution you are looking for sounds a lot like the open source software SymmetricDS.

"SymmetricDS is an asynchronous data replication software package that supports multiple subscribers and bi-directional synchronization. It uses web and database technologies to replicate tables between relational databases, in near real time if desired. The software was designed to scale for a large number of databases, work across low-bandwidth connections, and withstand periods of network outage."
-SymmetricDS.org

Symmetric was designed to be used as a Java library, as well as a stand alone application. Used with a lightweight database like H2, you could avoid your overkill scenario. H2 can optionally be run embedded within an application and can store data in memory or to disk.

Disclaimer: I recently started working for JumpMind, the company that develops this software.

Austin Brougher
  • 526
  • 2
  • 9
  • Interesting. We actually started looking at this earlier. Will this work in a situation where every node is of "equal" importance? IE, the website has an image that implies the existence of a master node instead of peers talking to each other. Finally, we are looking to potentially run this software on Android. Is it possible to write something to extend SymmetricDS support for SQLite? – Justin Breitfeller Mar 08 '12 at 20:57
  • SymmetricDS can work when every node is of "equal" importance. It is referred to as "multi-master replication". SymmetricDS 3 will support Android and SQLite. The target release is within the next few months. If you would like to work with the developers directly you can go to the JumpMind contact page. http://jumpmind.com/contact-us – Austin Brougher Mar 08 '12 at 21:21
  • SymmetricDS isn't a true P2P solution, but it might work for you. The natural way to use it is to have a single (or several) highly available centralized node (maybe in the cloud) that acts as a proxy to your client nodes. If instead, you wanted nodes to sync directly to each other, then they would have to be available on the network via HTTP and the configuration would be a bit more tricky. – chenson42 Mar 09 '12 at 00:38
  • I see. I think that might be the sticking point for us. Unfortunately, in our situation the network is an ever changing topology and it is very unlikely any single node would be highly available. – Justin Breitfeller Mar 09 '12 at 14:46
  • Since this is the most likely avenue we will explore, I'll accept this answer. For now, we are going to write something specific to our use case. – Justin Breitfeller Apr 18 '12 at 19:53
1

Interesting problem. Many of the issues you've described lend themselves particularly well to the BitTorrent protocol.

MattDavey
  • 8,897
  • 3
  • 31
  • 54
  • I've reached my vote limit for today, but that's a clever idea !! – J.N. Mar 08 '12 at 14:29
  • Does BT work well with unreliable NETWORKS (i.e. no guarantee of traffic delivery), or simply unreliable hosts (no guarantee of host availability)? Those are different problems. And while I agree it could be a clever solution, it would also require some kind of other interface to tell the various hosts what are available (since it's a pull protocol vs a push protocol). – Will Hartung Mar 08 '12 at 14:44
0

What you are looking for is called a "distributed database", and they are extensively used even in production system; http://www.project-voldemort.com/ for example, is used by linkedin

As p2p network like DHT and Kadmelia ARE key->value database, there are also some P2P database, where new node are automatically added and the failure resistence of any node is strong, as those network resistance and scalability is proven

So just look on your preferred search engine for "p2p database" and "distributed database", and you will find a lot of implementation.

Lesto
  • 2,260
  • 2
  • 19
  • 26
0

It seems you want to implementing a reliable broadcast for your peer communication. Check out the library J.N. provided, and if it is not sufficient (or you want to modify it) there are some algorithms in this book.

Check Causal Order Broadcast and Total Order Broadcast.

My teacher at the univ did implement such a library, I will update when I find it.

UmNyobe
  • 22,539
  • 9
  • 61
  • 90