3

I'm using GraphDb Free 8.6.1 in research project, I'm running it with default configuration on linux server having 4GB memory.

However, it has started to throw exceptions pointing to insufficient memory:

Caused by: org.eclipse.rdf4j.repository.RepositoryException: Query evaluation error: Insufficient free Heap Memory 238Mb for group by and distinct, threshold:250Mb, reached 0Mb (HTTP status 500)
    at org.eclipse.rdf4j.http.client.SPARQLProtocolSession.execute(SPARQLProtocolSession.java:1143)
    at org.eclipse.rdf4j.http.client.SPARQLProtocolSession.executeOK(SPARQLProtocolSession.java:1066)
    at org.eclipse.rdf4j.http.client.SPARQLProtocolSession.sendTupleQueryViaHttp(SPARQLProtocolSession.java:834)
    at org.eclipse.rdf4j.http.client.SPARQLProtocolSession.getTupleQueryResult(SPARQLProtocolSession.java:763)
    at org.eclipse.rdf4j.http.client.SPARQLProtocolSession.sendTupleQuery(SPARQLProtocolSession.java:391)
    at org.eclipse.rdf4j.repository.http.HTTPTupleQuery.evaluate(HTTPTupleQuery.java:69)

Please, can you help me to identify the problem? How can I properly configure GraphDB?

Ahmed Ashour
  • 5,179
  • 10
  • 35
  • 56
Peter Kostelnik
  • 63
  • 6

2 Answers2

7

The behavior you observe is part of memory optimization of distinct/group by operations. The error message itself is related to the default threshold of 250 mb and it's there to let you know you need to adjust your memory. When the free heap memory became less than the threshold, a QueryEvaluationException is thrown so to avoid running out of memory due to hungry distinct/group by operation. You can adjust the threshold to minimize those errors, by reducing it with with passing the following argument when starting GraphDB "-Ddefaut.min.distinct.threshold=XXX" (which could be set to the amount of memory in bytes for the threshold).

Insufficient free Heap Memory 238Mb for group by and distinct, threshold:250Mb, reached 0Mb
238Mb = free heap space reported by the JVM
250Mb = the default threshold below which the protection should raise an exception to prevent OME
0Mb = the current buffer used for distinct and group by

I suspect another operation takes most of your RAM and once you run any DISTINCT/GROUP BY query it immediately stop because of the OME protection.

Konstantin Petrov
  • 1,058
  • 6
  • 11
  • Thank you very much, i assumed, that problem will be here. Can you, please, give me some advise, to what value should i set the distinct threshold for this case? – Peter Kostelnik Aug 30 '18 at 12:04
  • It really depends on the dataset, the queries and the returned results. To minimize those errors probably the best thing to do is to lower its value to "-Ddefaut.min.distinct.threshold=104857600". But you should give it a try and experiment with the value. – Konstantin Petrov Aug 30 '18 at 14:03
  • 2
    Note the (not your) typo in the name of the parameter. Since GDB-8.8 release it should be `-Ddefault.min.distinct.threshold` instead of `-Ddefaut.min.distinct.threshold` (missing `l` in default). It has been fixed/changed in GDB-8.8 release. – dedek Aug 22 '19 at 15:37
0

This answer would have helped me.

Example, if your machine has 32 GB RAM:

/opt/graphdb-free/app/graphdb-free.cfg (cutout)

[JVMOptions]
-Xms20G
-Xmx20G
-XX:PermSize=4G
-XX:MaxPermSize=4G
-Dfile.encoding=UTF-8
-Djava.net.preferIPv4Stack=true
--add-exports
jdk.management.agent/jdk.internal.agent=ALL-UNNAMED
--add-opens
java.base/java.lang=ALL-UNNAMED

Via GUI and Settings:

graphdb.page.cache.size    10G
qräbnö
  • 2,722
  • 27
  • 40