Garbage Collection Overhead limit - Hibernate

Question

I am using Hibernate to query my MySQL database to find all of the segments in a particular journey. (For example a journey from A->D might have segments A->B, B->C, C->D.)

I run this query very frequently and it completes very quickly. The database is indexed in such a way that performance is very good. The code for the query is as follows:

public List<Segment> getSegmentsForUID(String trainUid) {
    Query segmentQuery = session.createQuery("select segment from Segment segment inner join segment.journey as journey where segment.journey " +
                                             "in (:journies) and journey.trainUid = '" + trainUid + "'");
    segmentQuery.setParameterList("journies", dailyJournies);
    queryCount++;
    return segmentQuery.getResultList();
}

Where dailyJournies is set in the initilisation of the class and is never changed again.

My problem is that after doing this query many times (usually in the region between 2,000 and 3,000 queries) I always receive a: Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded on the return segmentQuery.getResultList(); line.

Since no other part of this class is being touched after initilization and since the exception is always thrown on this line, I can't imagine it being a problem elsewhere.

My original thought after a bit of digging was that the Hibernate session was growing too large so I started to close and reopen the session after each 1,000 queries but this didn't make any difference. I've also tried using the following code:

Cache cache = session.getSessionFactory().getCache();
cache.evictAllRegions();

To clear the session cache manually but this didn't do anything either.

Has anyone else come across this before or is there something glaringly obvious that I'm missing?

Profile the application; determine what you are holding on to. — Boris the Spider, Dec 23 '17 at 12:06
@BoristheSpider Profiling on Netbeans shows that the number of surviving generations increases steadily over time. From the object profiler, I can see that char[] is using the most memory and this also seems to steadily increase over time. Is there another tool I can use to link this back to the code? — Tom, Dec 23 '17 at 12:29
unrelated to your issue: you shouldn't use string processing there. you're already using named parameters, why not use it for the `trainUid` too? — the8472, Dec 23 '17 at 18:52
@the8472 okay, thanks, I didn't realise that was the standard practice. What are the reasons for avoiding string processing in this case? — Tom, Dec 23 '17 at 19:40
generally to avoid SQL injection. https://stackoverflow.com/q/14102334/1362755 additionally fixed strings can allow prepared statements to be created and used by the RDBMS to optimize performance. — the8472, Dec 23 '17 at 19:53

score 0 · Accepted Answer · answered Dec 23 '17 at 13:24

0

It seems I wasn't finding the right information before.

Apparently this is an issue that others have experienced as well (https://github.com/ow2-proactive/scheduling/issues/2870).

For anyone stumbling across this question, I fixed it by adding the following two lines to my hibernate config file:

<property name="hibernate.query.plan_cache_max_size">16</property>
<property name="hibernate.query.plan_parameter_metadata_max_size">128</property>

Which is what was suggested in GitHub. I think the issue stems from keeping a session AND a sessionFactory open for many queries. I'm guessing the problem could also be fixed by creating a new sessionFactory for each 1,000 entries, but this solution seems cleaner.

answered Dec 23 '17 at 13:24

Tom

321
5
16

2

You should _never_ be creating a new `SessionFactory` - they're very heavy and designed to live for the entire lifecycle of the application. You seem to have run into the second of the two hard problems - caching. – Boris the Spider Dec 23 '17 at 13:56
@BoristheSpider Thanks for clarifying that. I assume that closing and reopening a session is an ok thing to do though? – Tom Dec 23 '17 at 14:07
1

Yes, exactly - sessions are desgined to be lightweight and short lived; usually having a session alive for anything more than a single "transaction" (either physical or logical) is an antipattern. – Boris the Spider Dec 23 '17 at 14:09
I guess, you caused the problem by generating new string queries by concatenating the `trainUid`. You can see that Hibernate uses caching in order to minimize the overhead and by supplying still new queries, you made this caching useless and actually harmful. +++ Note that the values you're using are described as "very low", so they may be far from optimal. – maaartinus Dec 29 '17 at 16:52

Garbage Collection Overhead limit - Hibernate

1 Answers1