OutOfMemoryError caused when db4o database has 15000+ objects

Question

I am using db4o 8.0.

I have a class

PostedMessage{
    @Indexed
    long receivedTime;
    @Indexed
    long sentTime;
    ...    
    //getter methods and setter methods for all the fields.
}

I persist the PostedMessage objects to db4o database. I have already saved 15000+ objects to db4o database. And now when I run following query, it results in OutOfMemoryError.

    //Query to get PostedMessages between "start" and "end" dates.
    Query q = db.query();
    q.constrain(PostedMessage.class);
    Constraint from = q.descend("receivedTime").constrain(new Long(start.getTimeInMillis())).greater().equal();
    q.descend("receivedTime").constrain(new Long(end.getTimeInMillis())).smaller().equal().and(from);
    q.execute();//results in OutOfMemoryError

To avoid the OutOfMemoryError, I need to add indexes to the fields of PostedMessage class. Read This.

I have a server/client configuration. I don't have control over pre-configuring the ObjectContainer before opening it.

I will have to apply/append the indexing CommonConfiguration after the ObjectContainer is just opened and provided to me.

I know how to create the config.

    EmbeddedConfiguration appendConfig = Db4oEmbedded.newConfiguration();
    appendConfig.common().objectClass(EmailMessage.class).objectField("receivedTime").indexed(true);
    appendConfig.common().objectClass(EmailMessage.class).objectField("sentTime").indexed(true);

I am not able to figure out how to apply this config to already opened ObjectContainer. How can I add indexes to the just opened ObjectContainer?

Is EmbeddedConfigurationItem's apply() method the answer? If it is, can I get a sample code that shows how to use it?

Edited : Added @Indexed annotation later to the question.

Answer 1

Look in Reference doc at @Indexed

Answer 2

cl-r's suggestion of using TA/TP worked like a charm in my case. See his comment above.

You have also to install Transparent Activation/Transparent Persistence to avoid to load unnecessary object in memmory. Look at chapter 10&11 in tutorial (in the doc/tutorial directory of the downloaded db4o[Version].zip] - cl-r

Answer 3

In my particular case, I need to iterate over the ObjectSet returned by the query.

It was found that using IMMEDIATE and SNAPSHOT query modes also solved the OutOfMemoryError problem. Also the timings were equally well. LAZY mode is not the solution for me.

It took about 8000 to 9000 ms to retrieve any 100 PostedMessages out of 100000 saved PostedMessages. e.g. 1 to 100, 1001 to 1100, 99899 to 99999.

Answer 4

You should add indexes for your queries. Otherwise db4o has to scan over all objects.

You can do it with an annotation, like this:

import com.db4o.config.annotations.Indexed;

PostedMessage{
    @Indexed
    long receivedTime;
    long sentTime;

Or as you do, with the configuration:

EmbeddedConfiguration config = Db4oEmbedded.newConfiguration();
config.common().objectClass(EmailMessage.class).objectField("receivedTime").indexed(true);
config.common().objectClass(EmailMessage.class).objectField("sentTime").indexed(true);
ObjectContainer container = Db4oEmbedded.openFile(config,"your-data.db4o");

You cannot add this configuration when the container is already running. Only when opening it. When the indexes are not there yet, they will be added while opening the database. You need to get control over it, when opening. Or use the annotation above.