Facing OutOfMemory issue in Java

Question

I am facing out of memory issue due to high heap allocation.I verified it from HP Diagnostics tool and it is pointing to a section in my code where I am adding elements in an arraylist. I am not able o figure out how else can I write this code so that objects are released early. Below is the code :

    private List<UpperDTO> populateRecords(List<BaseEntity> baseEntityList,List<DataEntity> dataEntityList) {

    List<UpperDTO> masterDTOList  = new ArrayList<UpperDTO>();
    if(baseEntityList !=null && baseEntityList.size()>0){
        BigDecimal conId = null;
        for(BaseEntity baseEntity :baseEntityList){
            conId = baseEntity.getConsignmentId();
            ArrayList<StatusData> statusDataList = new ArrayList<StatusData>();
            if(dataEntityList !=null && dataEntityList.size()>0){
                for(DataEntity data : dataEntityList){
                    if(conId.equals(data.getConsignmentId())){
                        //making null to supress from the response 
                        data.setConsignmentId(null);
                        statusDataList.add(TrackServiceHelper.convertStatusDataToDTO(data));
                    }
                }
            }
            masterDTOList.add(TrackServiceHelper.populateDTO(baseEntity, statusDataList));  
        }
    }
    return masterDTOList;
}

public static UpperDTO populateDTO(TrackBaseEntity baseEntity,
        List<StatusData> statusList) {

    UpperDTO upperDTO = new UpperDTO();
    //Setter methods called
    upperDTO.setStatusData(statusList);
    return upperDTO;

}

The issue is pointed at following line in the code :

    masterDTOList.add(TrackServiceHelper.populateDTO(baseEntity, statusDataList));

This is rest api which receives messages from JMS Queues and MDB listens to these messages. I am not able to simulate this in my local or Dev environments as the issue comes during performance testing when the number of requests are high. How can I fix this?

This is the stacktrace of Collection Leak from HP Diagnostics:

Chart   Collection Class    Contained Type  Probe   Collection Growth Rate  Collection Size Leak Stack Trace    Maximum Size
0, 0, 255   java.util.ArrayList com.rex.ih2.dtos.UpperDTO   gtatsh645   3,848   122,312 java.util.ArrayList.add(ArrayList.java:413)
com.rex.ih2.utils.AppDAO.populateConsignment(AppDAO.java:168)
com.rex.ih2.utils.AppDAO.searchConsignment(AppDAO.java:93)
com.rex.ih2.service.AppService.fetchConDetail(AppService.java:131)
com.rex.ih2.service.AppService.getConDetail(AppService.java:69)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:76)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:607)
org.apache.webbeans.intercept.InterceptorHandler.invoke(InterceptorHandler.java:297)
org.apache.webbeans.intercept.NormalScopedBeanInterceptorHandler.invoke(NormalScopedBeanInterceptorHandler.java:98)
com.rex.ih2.service.TrackService_$$_javassist_0.getConsignmentDetail(TrackService_$$_javassist_0.java)
com.rex.ih2.beans.TrackBean.action(TrackBean.java:35)
com.tnt.integration.bean.AbstractServiceBean.invokeService(AbstractServiceBean.java:259)
com.tnt.integration.bean.AbstractServiceBean.onMessage(AbstractServiceBean.java:157)
com.rex.ih2.beans.TrackBean.onMessage(TrackBean.java)

This is a question much bigger than the two methods you have provided. It's more of a design question rather than a code question, IMHO. I guess the question I'd ask here is, where is this information coming from, and where is it going? You mention JMS and MDBs; is it possible to handle the information in smaller chunks at a time? — dcsohl, May 09 '16 at 14:36
This is a get API which receives request message from message broker which has few parameters based on which a search has to be made in database.That search populates the 2 entities mentioned in the first method. From these entities we have to populate a DTO which also has a list of another DTO as a member variable. This DTO is then set in the Response Object. — Neel, May 09 '16 at 14:50
So can you apply pagination or something? How big are `baseEntityList` and `dataEntityList` anyway? Maybe you just need a larger heap size. — dcsohl, May 09 '16 at 14:54
No of records fetched are not more than 5 for most of the searches, so Nopagination has not been put in. The baseEntityList and dataEntityList have 15 and 6 variables respectively. In the Collection Leaks in HP Diagnostics it is showing collection growth rate as 3848, collection size as 122312 and maximum size 143291. I am not sure why the list is growing so much. Is it possible that objects are still references even after the response is sent. Is the list available of garbage collection as soon as the response is sent? — Neel, May 09 '16 at 15:03

score 0 · Answer 1 · answered May 09 '16 at 14:52

0

I agree with dcsohi. This is actually a design problem. You may want to look at below approaches:-

1) Size of the object being added in the list. If it can be optimized. 2) Handle data in chunks instead of adding all at once in the list. 3) Optimizing JVM arguments to increase head size so that it can handle more objects.

You can try to simulate this by increasing number of test objects and reducing heap size in dev environment or maybe taking production dump and running with the same volume.

answered May 09 '16 at 14:52

Jitesh Shivnani

363
1
9

1) I am not sure how can I reduce the size of the object as it only has essential varibales. 2) Amount of data fetched per request is quite low, issue arises when no of requests are high. 3)This option I can use only if there is no other option to tune the code itself. – Neel May 09 '16 at 15:21
Check the size of your list. Looking at what you mentioned in comment it doesnt look too much. Try taking a heap dump as well. – Jitesh Shivnani May 09 '16 at 15:33

score 0 · Answer 2 · answered May 09 '16 at 15:01

Ok, it looks to me like you only care about DataEntity objects and BaseEntity objects where their "consignment IDs" match. You really should do this sort of thing in the database query. The use of "entity" objects makes it seem like your DB interactions are via JPA/Hibernate, in which case you may want to create a DB view that joins the two tables by consignment ID, and provides the necessary information for your output. Next, create a custom read-only entity that matches this view. Then you can apply pagination to your query of this view (if it's still necessary) and retrieve the information in smaller batches.

I am not making any queries to the database directly as the queries are very complex. I am calling a stored procedure which is returning me 2 refcursors each containing the list of entities mentioned in the description. That is why I have to loop through the list and see if the second list has same consignment id. — Neel, May 09 '16 at 15:17

Facing OutOfMemory issue in Java

2 Answers2