Querying across heterogeneous triplestores

Question

What I currently believe to be true:

Triplestores are all about 'linked data'; data that is decoupled from any given application and can be augmented through afferent links formed by new data. This concept drives the Web.
URIs allow linked data to be hosted in different heterogeneous environments.
For two machines to interoperate, they must both work to an agreed standard.
W3C have defined such standards. Specifically: RDF (for formatting) and SPARQL (for querying).

Question:

Is it possible to run a query that spans multiple heterogeneous RDF triplestores?

Example:

We have 3 RDF triplestores running on different triplestore software.
Triplestore A contains nodes which reference other nodes in triplestores B and C.
We run such a query that should pass over nodes from all 3 stores.

Joshua Taylor · Accepted Answer · 2013-10-31T14:33:44.827

3

Yes, this is quite possible, and it's what the SPARQL 1.1 Federated Query W3C recommendation is about. In a SPARQL query you use the SERVICE keyword to specify different SPARQL endpoints that you want to query. An example from the linked recommendation is

PREFIX foaf:   <http://xmlns.com/foaf/0.1/>
SELECT ?name
FROM <http://example.org/myfoaf.rdf>
WHERE
{
  <http://example.org/myfoaf/I> foaf:knows ?person .
  SERVICE <http://people.example.org/sparql> { 
    ?person foaf:name ?name . } 
}

The service keyword there says that the triple ?person foaf:name ?name should be retrieved from <http://people.example.org/sparql>. In your case, you might end up with something like:

PREFIX ex: <http://example.org/>
SELECT ?person ?age ?weight ?resume WHERE {
  values ?person { ex:Bill ex:John }
  SERVICE <http://jobs.example.org/sparql>   { ?person ex:resume ?resume } 
  SERVICE <http://age.example.org/sparql>    { ?person ex:age    ?age    } 
  SERVICE <http://weight.example.org/sparql> { ?person ex:weight ?weight } 
}

You'll have to run this query somewhere, though. If you run it locally, then all three triplestores can be specified as services, but if you're running the query directly against one of them, then you could have just the other two as services. This will all depend on having some SPARQL engine that supports federated queries, of course. I expect that most do, these days, but I only have experience with Jena (which does support federated queries).

edited Oct 31 '13 at 14:33

answered Oct 31 '13 at 14:28

Joshua Taylor

84,998
9
154
353

2

The service keyword is kind of a joke for any real deployment scenario (imo). So while that will work for the OP, I'd advise against ever using it. A federated query service, against which you pose a normal SPARQL query and it figures out what parts of the query go where is a better approach. Encoding where the data is in the query is brittle and kind of defeats a lot of the value data integration via RDF brings to the table. – Michael Oct 31 '13 at 15:48
@Michael: That sounds exactly like what I want to do. How do I achieve this? How do I avoid specifying the locations upfront in the query? I envision a system whereby nodes in the RDF store you are querying reference nodes from other RDF stores, and that the query starts at the entry node, and crawls the graph much like a spider. – Lawrence Wagerfield Nov 01 '13 at 13:47
2

I dont advise doing it by hand, federated query is not something you can (correctly) throw together in an afternoon. There are solutions out there, I know Sesame has some sort of federation support and I'm sure commercial vendors do as well. But the tl;dr of it is, you have the logical ontology that represents your domain and queries are wrt to that. A separate query service takes that query and decomposes it into (hopefully) smaller queries to the individual sources which contains different parts of the data, based on some configuration/mapping. – Michael Nov 01 '13 at 14:15
@Michael you might have a look at [How can I query over multiple endpoints without adding SERVICE clauses to my query?](http://answers.semanticweb.com/questions/24999/how-can-i-query-over-multiple-endpoints-without-adding-service-clauses-to-my-query) from answers.semanticweb.com. – Joshua Taylor Nov 06 '13 at 13:58
Yes, I'm aware of Sesame's federation support, it's a good open source option for @LawrenceWagerfield. Our solution is akin to what I described, but is not yet a part of Stardog – Michael Nov 06 '13 at 17:26
@Michael Oops, sorry, I meant to comment at Lawrence, not you. Thanks for replying with a comment that he'll get notified on! – Joshua Taylor Nov 06 '13 at 17:42

Querying across heterogeneous triplestores

1 Answers1