Adding Individuals in Jena OntMOdel and accessing it. Exception ObjectFileStorage.read Impossibly large object

Question

I am trying to add some individuals to my existing Ontology (OntModel) with an objective to add the values/literals for DatatypeProperty with a specific datatype known at runtime from the range of the datatypeproperty. My OntModel is backed by a TDBStore, linked to a dataset(so any changes made to the OntModel ins reflected back to my TDBStore/dataset). Individuals are added as per the following code :

Individual ind =oc.createIndividual(namespace+nameOfIndividual); // oc is OntClass object
Literal l = ontModel.createTypedLiteral("1230",dp.getRange().getURI()); //dp is a DatatypeProperty object
ind.addLiteral(dp,l);

When the code executes the the literal is added and a sparql query :

"SELECT *  WHERE {  ?s :ABCConstant ?o }";  //:ABCConstant is the datatypeProperty for which the literal is added by the above code.

Gives me the following result:

------------------------------------------------------------------------------------------------------------------------------------------------
| s                                                                                     | o                                                    |
================================================================================================================================================
| <http://www.semanticweb.org/ontologies/2012/10/Ontology.owl#individual1ABCDConstant>    | "10000.0"^^<http://www.w3.org/2001/XMLSchema#int> |
| <http://www.semanticweb.org/ontologies/2012/10/Ontology.owl#individual2ABCDConstant> | 1230                                                 |
------------------------------------------------------------------------------------------------------------------------------------------------

But when i try to use the same query in a second run(without creating any individual this time) I get the following Exception when the program tries to display/access the result:

Exception in thread "main" com.hp.hpl.jena.tdb.base.file.FileException: ObjectFileStorage.read[nodes](181624)[filesize=248062][file.size()=248062]: Impossibly large object : 1634628966 bytes > filesize-(loc+SizeOfInt)=66434
    at com.hp.hpl.jena.tdb.base.objectfile.ObjectFileStorage.read(ObjectFileStorage.java:346)
    at com.hp.hpl.jena.tdb.lib.NodeLib.fetchDecode(NodeLib.java:78)
    at com.hp.hpl.jena.tdb.nodetable.NodeTableNative.readNodeFromTable(NodeTableNative.java:178)
    at com.hp.hpl.jena.tdb.nodetable.NodeTableNative._retrieveNodeByNodeId(NodeTableNative.java:103)
    at com.hp.hpl.jena.tdb.nodetable.NodeTableNative.getNodeForNodeId(NodeTableNative.java:74)
    at com.hp.hpl.jena.tdb.nodetable.NodeTableCache._retrieveNodeByNodeId(NodeTableCache.java:103)
    at com.hp.hpl.jena.tdb.nodetable.NodeTableCache.getNodeForNodeId(NodeTableCache.java:74)
    at com.hp.hpl.jena.tdb.nodetable.NodeTableWrapper.getNodeForNodeId(NodeTableWrapper.java:55)
    at com.hp.hpl.jena.tdb.nodetable.NodeTableInline.getNodeForNodeId(NodeTableInline.java:67)
    at com.hp.hpl.jena.tdb.lib.TupleLib.triple(TupleLib.java:126)
    at com.hp.hpl.jena.tdb.lib.TupleLib.triple(TupleLib.java:114)
    at com.hp.hpl.jena.tdb.lib.TupleLib.access$000(TupleLib.java:45)
    at com.hp.hpl.jena.tdb.lib.TupleLib$3.convert(TupleLib.java:76)
    at com.hp.hpl.jena.tdb.lib.TupleLib$3.convert(TupleLib.java:72)
    at org.apache.jena.atlas.iterator.Iter$4.next(Iter.java:317)
    at org.apache.jena.atlas.iterator.Iter$4.next(Iter.java:317)
    at org.apache.jena.atlas.iterator.Iter$4.next(Iter.java:317)
    at org.apache.jena.atlas.iterator.Iter.next(Iter.java:915)
    at com.hp.hpl.jena.util.iterator.WrappedIterator.next(WrappedIterator.java:94)
    at com.hp.hpl.jena.util.iterator.WrappedIterator.next(WrappedIterator.java:94)
    at com.hp.hpl.jena.util.iterator.FilterIterator.hasNext(FilterIterator.java:55)
    at com.hp.hpl.jena.graph.compose.CompositionBase$2.hasNext(CompositionBase.java:94)
    at com.hp.hpl.jena.util.iterator.NiceIterator$1.hasNext(NiceIterator.java:103)
    at com.hp.hpl.jena.util.iterator.WrappedIterator.hasNext(WrappedIterator.java:90)
    at com.hp.hpl.jena.sparql.engine.iterator.QueryIterTriplePattern$TripleMapper.hasNextBinding(QueryIterTriplePattern.java:151)
    at com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:112)
    at com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.hasNextBinding(QueryIterRepeatApply.java:81)
    at com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:112)
    at com.hp.hpl.jena.sparql.engine.iterator.QueryIterBlockTriples.hasNextBinding(QueryIterBlockTriples.java:64)
    at com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:112)
    at com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBinding(QueryIteratorWrapper.java:40)
    at com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:112)
    at com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBinding(QueryIteratorWrapper.java:40)
    at com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:112)
    at com.hp.hpl.jena.sparql.engine.ResultSetStream.hasNext(ResultSetStream.java:75)
    at com.hp.hpl.jena.sparql.resultset.ResultSetMem.<init>(ResultSetMem.java:97)
    at com.hp.hpl.jena.query.ResultSetFactory.makeRewindable(ResultSetFactory.java:420)
    at com.hp.hpl.jena.sparql.resultset.TextOutput.write(TextOutput.java:149)
    at com.hp.hpl.jena.sparql.resultset.TextOutput.write(TextOutput.java:132)
    at com.hp.hpl.jena.sparql.resultset.TextOutput.write(TextOutput.java:120)
    at com.hp.hpl.jena.sparql.resultset.TextOutput.format(TextOutput.java:67)
    at com.hp.hpl.jena.query.ResultSetFormatter.out(ResultSetFormatter.java:122)
    at com.hp.hpl.jena.query.ResultSetFormatter.out(ResultSetFormatter.java:74)
    at com.hp.hpl.jena.query.ResultSetFormatter.out(ResultSetFormatter.java:65)
    at senseXploreApi.SparqlQuery.sparqlQuery(SparqlQuery.java:82)
    at senseXploreApi.TryMain.main(TryMain.java:39)

Note: Other queries to view the added individuals execute fine. But Queries like: "SELECT * WHERE { ?s ?o 1230}" lead to the same error.

Also query like : "SELECT * WHERE { ?s ?o 10000.0}" OR "SELECT * WHERE { ?s ?o 10000}" don't give any error but return nothing in result.

the literal 10000 was added using the statement :

ind.addLiteral(opp,new Integer(10000));

Please help me!! where am i wrong.. Is the procedure used to create individuals is wrong? if yes! then what can be other possible ways to add literals with specific datatype known at runtime?

score 3 · Accepted Answer · answered Oct 28 '13 at 09:34

3

The Impossibly Large Object message means that your TDB database is (partially) corrupted, specifically part of the node table that maps between database internal identifiers and the original RDF terms is corrupted. Any query that touches that part of the node table will see this error, other queries may still run without issue.

The only way to recover your database is to rebuild it from the original data, once the corruption has happened it cannot be repaired.

Corruption can happen in various ways, the most common of which are:

Accessing TDB in a non-transactional manner and failing to call sync() on the dataset after making changes (or more likely in your example failing to call close() on the Model after making changes)
Having multiple JVMs accessing the same TDB dataset at the same time, if your application requires this use a server like Fuseki to centralise access to the database.

See the TDB Transactions for how to use TDB transactionally or see the Fuseki documentation if you need to expose a TDB database to multiple JVMs simultaneously.

answered Oct 28 '13 at 09:34

RobV

28,022
11
77
119

I am using a single JVM. No threads, no concurrent accesses. Though I'am not performing any sync calls, but I ensure that the model is closed at the end of the program. still I am facing such data corruption, which is not advisable for my application because my application will be storing new individuals during runtime, any such data corruption will lead to loss of runtime collected data. Further I plan to use the application on a web server, is Fuseki meant for that purpose?? For handling multiple query requests on the TDB?? – Dibyanshu Jaiswal Oct 29 '13 at 05:20
@djthequest Without seeing your full code it is impossible to say why the corruption is happening. There has been a recent bug about `close()` not always doing the right thing so I would strongly recommend using transactions in your code as they make it very hard for the data to become corrupted – RobV Oct 29 '13 at 10:00
1

@djthequest Running the application on a web server does not necessitate Fuseki if there is only one web app accessing the database but that is really an entirely separate question and you should ask it as such – RobV Oct 29 '13 at 10:01

Adding Individuals in Jena OntMOdel and accessing it. Exception ObjectFileStorage.read Impossibly large object

1 Answers1