2

Does it make sense to break up the data model of an application into different database systems? For example, the application stores all user data and relationships in a graph database (ideal for storing relationships), while storing other data in a document database, such as CouchDB or MongoDB? This would require the user graph database to reference unique ids in the document databases and vice versa.

Is this over complicating the data model and application? Or is this using the best uses of both types of database systems for scaling your application?

Community
  • 1
  • 1
onejigtwojig
  • 4,771
  • 9
  • 32
  • 35
  • a similar question has been asked. http://stackoverflow.com/questions/5817182/using-mongodb-as-our-master-database-should-i-use-a-separate-graph-database-to-i/5829228#5829228 – onejigtwojig Aug 14 '11 at 13:02

3 Answers3

4

It definitely can make sense and depends fully on the requirements of your application. If you can use other database systems for things in which they are really good at.

Take for example full text search. Of course you can do more or less complex full text searches with a relational database like MySql. But there are systems like e.g. Lucene/Solr which are optimized for such things and can search fast in millions of documents. So you could use these systems for their special task (here: make a nifty full text search), then you return the identifiers and maybe load the relational structured data from the RDBMS.

Or CouchDB. I use couchDB in some projects as a caching systems. In combination with a relational database. Of course I need to care about consistency, but it it's definitely worth the effort. It pushed performance in the projects a lot and decreases for example load on the server from 2 to 0.2. :)

disco crazy
  • 31,313
  • 12
  • 80
  • 83
  • thanks for your answer. I do want to mention that in both of your examples, full text search and couchdb, you are using multiple database systems that will basically be storing the same/duplicate data. You are simply using the additional database for faster querying performance. My question is primarily for asking whether it's useful to break up your data model into multiple systems, where those systems are storing different sets of data or rather different parts of the data model. – onejigtwojig Aug 12 '11 at 20:50
  • 1
    Hmm. Yes, it depends. For example in Solr I am not duplicating the data. Parts of the data are in Solr and the other data in a relational database. I mean in a current project which is really document heavy because of crawled data, I store lots of parts e.g. in Solr and some structured data, which are still part of the model in a relational database. But in this case the Solr data is not duplicating anything except of unique IDs for reference. :) – disco crazy Aug 12 '11 at 21:00
3

Something like this is for instance called cross-store persistence. As you mentioned you would store certain data in your relational database, social relationships in a graphdb, user-generated data (documents) in a document-db and user provided multimedia files (pictures, audio, video) in a blob-store like S3.

It is mainly about looking at the use-cases and making sure that from wherever you need it you might access the "primary" or index key of each store (back and forth). You can encapsulate the actual lookup in your domain or dao layer.

Some frameworks like the Spring Data projects provide some initial kind of cross-store persistence out of the box, mostly integrating JPA with a different NOSQL datastore. For instance Spring Data Graph allows it to store your entities in JPA and add social graphs or other highly interconnected data as a secondary concern and leverage a graphdb for the typical traversal and other graph operations (e.g. ranking, suggestions etc.)

Michael Hunger
  • 41,339
  • 3
  • 57
  • 80
1

Another term for this is polyglot persistence.

Here are two contrary positions on the question:

Pro: "Contrary to that, I’m a big fan of polyglot persistence. This simply means using the right storage backend for each of your usecases. For example file storages, SQL, graph databases, data ware houses, in-memory databases, network caches, NoSQL. Today there are mostly two storages used, files and SQL databases. Both are not optimal for every usecase."

Con: "I don’t think I need to say that I’m a proponent of polyglot persistence. And that I believe in Unix tools philosophy. But while adding more components to your system, you should realize that such a system complexity is “exploding” and so will operational costs grow too (nb: do you remember why Twitter started to into using Cassandra?) . Not to mention that the more components your system has the more attention and care must be invested figuring out critical aspects like overall system availability, latency, throughput, and consistency."

onejigtwojig
  • 4,771
  • 9
  • 32
  • 35