Illegal utf-8 exception while querying apache jena with tdbquery

Question

I use Apache Jena to query RDF Data from the Billion Triple Challange 2014 Dataset. I loaded the dataset into Jena with tdbloader. I especially use queries which contain property paths with tdbquery. When I start such a query I often get the Exception:

 Exception
org.apache.jena.atlas.RuntimeIOException: java.io.IOException: Illegal UTF-8: 0xFFFFFF80
    at org.apache.jena.atlas.io.IO.exception(IO.java:216)
    at org.apache.jena.atlas.io.BlockUTF8.exception(BlockUTF8.java:279)
    at org.apache.jena.atlas.io.BlockUTF8.toCharsBuffer(BlockUTF8.java:152)
    at org.apache.jena.atlas.io.BlockUTF8.toChars(BlockUTF8.java:75)
    at org.apache.jena.atlas.io.BlockUTF8.toString(BlockUTF8.java:97)
    at org.apache.jena.tdb.store.nodetable.NodecSSE.decode(NodecSSE.java:101)
    at org.apache.jena.tdb.lib.NodeLib.decode(NodeLib.java:105)
    at org.apache.jena.tdb.lib.NodeLib.fetchDecode(NodeLib.java:81)
    at org.apache.jena.tdb.store.nodetable.NodeTableNative.readNodeFromTable(NodeTableNative.java:186)
    at org.apache.jena.tdb.store.nodetable.NodeTableNative._retrieveNodeByNodeId(NodeTableNative.java:111)
    at org.apache.jena.tdb.store.nodetable.NodeTableNative.getNodeForNodeId(NodeTableNative.java:70)
    at org.apache.jena.tdb.store.nodetable.NodeTableCache._retrieveNodeByNodeId(NodeTableCache.java:128)
    at org.apache.jena.tdb.store.nodetable.NodeTableCache.getNodeForNodeId(NodeTableCache.java:82)
    at org.apache.jena.tdb.store.nodetable.NodeTableWrapper.getNodeForNodeId(NodeTableWrapper.java:54)
    at org.apache.jena.tdb.store.nodetable.NodeTableInline.getNodeForNodeId(NodeTableInline.java:67)
    at org.apache.jena.atlas.iterator.Iter$4.next(Iter.java:308)
    at org.apache.jena.sparql.engine.main.iterator.QueryIterGraph$QueryIterGraphInner.nextIterator(QueryIterGraph.java:152)
    at org.apache.jena.sparql.engine.main.iterator.QueryIterGraph$QueryIterGraphInner.hasNextBinding(QueryIterGraph.java:126)
    at org.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:111)
    at org.apache.jena.sparql.engine.iterator.QueryIterRepeatApply.hasNextBinding(QueryIterRepeatApply.java:74)
    at org.apache.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:111)
    at org.apache.jena.tdb.solver.OpExecutorTDB1.optimizeExecuteQuads(OpExecutorTDB1.java:212)
    at org.apache.jena.tdb.solver.OpExecutorTDB1.execute(OpExecutorTDB1.java:148)
    at org.apache.jena.sparql.engine.main.ExecutionDispatch.visit(ExecutionDispatch.java:66)
    at org.apache.jena.sparql.algebra.op.OpQuadPattern.visit(OpQuadPattern.java:95)
    at org.apache.jena.sparql.engine.main.ExecutionDispatch.exec(ExecutionDispatch.java:46)
    at org.apache.jena.sparql.engine.main.OpExecutor.exec(OpExecutor.java:118)
    at org.apache.jena.tdb.solver.OpExecutorTDB1.exec(OpExecutorTDB1.java:87)
    at org.apache.jena.sparql.engine.main.OpExecutor.execute(OpExecutor.java:232)
    at org.apache.jena.sparql.engine.main.ExecutionDispatch.visit(ExecutionDispatch.java:130)
    at org.apache.jena.sparql.algebra.op.OpSequence.visit(OpSequence.java:75)
    at org.apache.jena.sparql.engine.main.ExecutionDispatch.exec(ExecutionDispatch.java:46)
    at org.apache.jena.sparql.engine.main.OpExecutor.exec(OpExecutor.java:118)
    at org.apache.jena.tdb.solver.OpExecutorTDB1.exec(OpExecutorTDB1.java:87)
    at org.apache.jena.sparql.engine.main.OpExecutor.execute(OpExecutor.java:393)
    at org.apache.jena.sparql.engine.main.ExecutionDispatch.visit(ExecutionDispatch.java:267)
    at org.apache.jena.sparql.algebra.op.OpProject.visit(OpProject.java:47)
    at org.apache.jena.sparql.engine.main.ExecutionDispatch.exec(ExecutionDispatch.java:46)
    at org.apache.jena.sparql.engine.main.OpExecutor.exec(OpExecutor.java:118)
    at org.apache.jena.tdb.solver.OpExecutorTDB1.exec(OpExecutorTDB1.java:87)
    at org.apache.jena.sparql.engine.main.OpExecutor.execute(OpExecutor.java:415)
    at org.apache.jena.sparql.engine.main.ExecutionDispatch.visit(ExecutionDispatch.java:275)
    at org.apache.jena.sparql.algebra.op.OpDistinct.visit(OpDistinct.java:47)
    at org.apache.jena.sparql.engine.main.ExecutionDispatch.exec(ExecutionDispatch.java:46)
    at org.apache.jena.sparql.engine.main.OpExecutor.exec(OpExecutor.java:118)
    at org.apache.jena.tdb.solver.OpExecutorTDB1.exec(OpExecutorTDB1.java:87)
    at org.apache.jena.sparql.engine.main.OpExecutor.execute(OpExecutor.java:403)
    at org.apache.jena.sparql.engine.main.ExecutionDispatch.visit(ExecutionDispatch.java:307)
    at org.apache.jena.sparql.algebra.op.OpSlice.visit(OpSlice.java:50)
    at org.apache.jena.sparql.engine.main.ExecutionDispatch.exec(ExecutionDispatch.java:46)
    at org.apache.jena.sparql.engine.main.OpExecutor.exec(OpExecutor.java:118)
    at org.apache.jena.tdb.solver.OpExecutorTDB1.exec(OpExecutorTDB1.java:87)
    at org.apache.jena.sparql.engine.main.OpExecutor.execute(OpExecutor.java:89)
    at org.apache.jena.sparql.engine.main.QC.execute(QC.java:52)
    at org.apache.jena.sparql.engine.main.QueryEngineMain.eval(QueryEngineMain.java:53)
    at org.apache.jena.tdb.solver.QueryEngineTDB.eval(QueryEngineTDB.java:112)
    at org.apache.jena.sparql.engine.QueryEngineBase.evaluate(QueryEngineBase.java:136)
    at org.apache.jena.sparql.engine.QueryEngineBase.createPlan(QueryEngineBase.java:106)
    at org.apache.jena.sparql.engine.QueryEngineBase.getPlan(QueryEngineBase.java:87)
    at org.apache.jena.tdb.solver.QueryEngineTDB$QueryEngineFactoryTDB.create(QueryEngineTDB.java:169)
    at org.apache.jena.sparql.engine.QueryExecutionBase.getPlan(QueryExecutionBase.java:582)
    at org.apache.jena.sparql.engine.QueryExecutionBase.startQueryIterator(QueryExecutionBase.java:526)
    at org.apache.jena.sparql.engine.QueryExecutionBase.execResultSet(QueryExecutionBase.java:567)
    at org.apache.jena.sparql.engine.QueryExecutionBase.execSelect(QueryExecutionBase.java:184)
    at org.apache.jena.sparql.util.QueryExecUtils.doSelectQuery(QueryExecUtils.java:196)
    at org.apache.jena.sparql.util.QueryExecUtils.executeQuery(QueryExecUtils.java:78)
    at arq.query.queryExec(query.java:218)
    at arq.query.exec(query.java:160)
    at jena.cmd.CmdMain.mainMethod(CmdMain.java:93)
    at jena.cmd.CmdMain.mainRun(CmdMain.java:58)
    at jena.cmd.CmdMain.mainRun(CmdMain.java:45)
    at tdb.tdbquery.main(tdbquery.java:33)
Caused by: java.io.IOException: Illegal UTF-8: 0xFFFFFF80
    ... 71 more

Is my query wrongly encoded or the dataset?

For instance is used a query: SELECT DISTINCT ?o { GRAPH ?g {A B* ?o}} LIMIT 3 where A and B are valid IRIs and it returned a valid result. But I want all results for that query so I deleted the LIMIT 3 and got the exception.

I also used the query directly:

sudo ./tdbquery -time -loc=/path/to/database "SELECT DISTINCT ?o {GRAPH ?g {A B* ?o}}"

and from a file

sudo ./tdbquery -time -loc=/path/to/database --query=/path/to/query.sparql

Sorry if any important information is missing to the question. Any Idea why I get the exception and how I can handle it?

It is likely the data is bad UTF-8 and that it managed to get loaded anyway. Which format of which file did you load? The other possibility is that loading was interrupted and restarted. Jena ships with a command "arq.utf8" that tries to check files for UTF-8 correctness. — AndyS, Jun 01 '16 at 21:45
Thank you for your comment. What exactly do you mean by format? The data was in nquads : file.nq.gz . I decided to load the data again into the store because the loading process was indeed interrupted. Thus I hope that this will fix the problem. — SNaidra, Jun 02 '16 at 15:19
`.nq.gz` is compressed n-quads. TDB treats that quite laxly and might have been that bad data got into TDB but the fact there had been an interrupted load is much more important. — AndyS, Jun 03 '16 at 22:41

Illegal utf-8 exception while querying apache jena with tdbquery

0 Answers0