2

My application is deployed on a cluster environment. Recently the server went down with the following stacktrace. It doesn't seem to be coming from the code. It was running all right until recently when this error pop up. No major changes were made to the server. Can someone advise?

java.lang.OutOfMemoryError: Java heap space
    at java.lang.AbstractStringBuilder.<init>(AbstractStringBuilder.java:44)
    at java.lang.StringBuilder.<init>(StringBuilder.java:69)
    at java.io.ObjectStreamClass$FieldReflectorKey.<init>(ObjectStreamClass.java:2106)
    at java.io.ObjectStreamClass.getReflector(ObjectStreamClass.java:2039)
    at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:586)
    at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1552)
    at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1466)
    at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1591)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1299)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:348)
    at weblogic.utils.io.ChunkedObjectInputStream.readObject(ChunkedObjectInputStream.java:195)
    at weblogic.rjvm.MsgAbbrevInputStream.readObject(MsgAbbrevInputStream.java:565)
    at weblogic.utils.io.ChunkedObjectInputStream.readObject(ChunkedObjectInputStream.java:191)
    at weblogic.rmi.internal.dgc.DGCServerImpl_WLSkel.invoke(Unknown Source)
    at weblogic.rmi.internal.BasicServerRef.invoke(BasicServerRef.java:589)
    at weblogic.rmi.internal.BasicServerRef$1.run(BasicServerRef.java:479)
    at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:363)
    at weblogic.security.service.SecurityManager.runAs(Unknown Source)
    at weblogic.rmi.internal.BasicServerRef.handleRequest(BasicServerRef.java:475)
    at weblogic.rmi.internal.BasicServerRef.access$300(BasicServerRef.java:59)
    at weblogic.rmi.internal.BasicServerRef$BasicExecuteRequest.run(BasicServerRef.java:1016)
    at weblogic.work.SelfTuningWorkManagerImpl.schedule(SelfTuningWorkManagerImpl.java:126)
    at weblogic.rmi.internal.BasicServerRef.dispatch(BasicServerRef.java:321)
    at weblogic.rmi.internal.BasicServerRef.dispatch(BasicServerRef.java:918)
    at weblogic.rjvm.RJVMImpl.dispatchRequest(RJVMImpl.java:1084)
    at weblogic.rjvm.RJVMImpl.dispatch(RJVMImpl.java:1001)
    at weblogic.rjvm.ConnectionManagerServer.handleRJVM(ConnectionManagerServer.java:240)
    at weblogic.rjvm.ConnectionManager.dispatch(ConnectionManager.java:877)
    at weblogic.rjvm.MsgAbbrevJVMConnection.dispatch(MsgAbbrevJVMConnection.java:446)
    at weblogic.rjvm.t3.MuxableSocketT3.dispatch(MuxableSocketT3.java:368)
    at weblogic.socket.AbstractMuxableSocket.dispatch(AbstractMuxableSocket.java:383)
    at weblogic.socket.SocketMuxer.readReadySocketOnce(SocketMuxer.java:872)
musiKk
  • 14,751
  • 4
  • 55
  • 82
Dave
  • 147
  • 2
  • 3
  • 6

5 Answers5

8

You are running out of memory, which suggests one of the following:

  • you need to give your process more memory (with the -Xmx java command line option); or
  • you have a memory leak

Without more information, it's hard to say which is the case. The stack trace for an OutOfMemoryError is rarely useful, as it only shows the point at which heap was exhausted; it doesn't show you why your heap is being filled up.

Simon Nickerson
  • 42,159
  • 20
  • 102
  • 127
  • Further to Simon's points, I think that your best bet is to monitor the heap usage ( possibly by using verbose garbage collection ) and then, when the heap shows signs of irreversiible growth, use a tool like 'jmap' and the eclipse memory analyser tool to see what object it is that is dominating your heap. – DaveH May 19 '11 at 10:59
1

The answer by Simon Nickerson is correct

Just to add, your stack trace begins from weblogic.socket.SocketMuxer.readReadySocketOnce which is the internal weblogic class that accepts incoming requests. So this means the server is not having enough memory to accept requests also.

shinynewbike
  • 2,334
  • 5
  • 28
  • 42
1

Are you using the JRockit JVM? If you are you can use JRockit Mission Control and monitor the Java heap usage. You can also use the JRockit Flight Recorder to record JVM events for offline analysis. There is an Oracle webcast on this here: http://www.vimeo.com/22109838. You can skip to 4:54 which is where the overview of JRockit, WLDF and JRF starts.

Keep in mind that when the heap is full it is the NEXT operation that fails with the OutOfMemory Exception, and therefore this stack trace may not indicate any cause of the failure. This simply indicates that when this code ran there wasn't enough heap, not that this code caused the heap to fill up.

** Edits...

Clearly the server is out of memory - at the time of this specific operation. The question is... why? This stack trace doesn't tell you -why- it just indicates that whatever was happening at the time could not complete because there was not enough memory available at that time. This does not mean that it is the cause of the problem.

Sure, you can add more memory but that may not fix the problem - it may only take longer for it to appear.

Jeff West
  • 1,563
  • 9
  • 11
0

It means that the JVM has run out of all the memory that has been allocated to it. You can change the amount of memory allocated for use by your JVM using the -Xms and -Xmx command line parameters.check the root cause here

OutOfMemoryError in Java is a subclass of java.lang.VirtualMachineError and JVM throws java.lang.OutOfMemoryError when it ran out of memory in heap. OutOfMemoryError in Java can come any time in heap mostly while you try to create an object and there is not enough space in heap to allocate that object

user2664373
  • 1
0

set catalina.sh/bat find set JAVA_OPTS=%JAVA_OPTS% what ever your RAM - adjust but don't give above half of RAM set JAVA_OPTS=%JAVA_OPTS% %LOGGING_CONFIG% -server -Xms512M -Xmx512M -XX:MaxPermSize=256M

Ravi Parekh
  • 5,253
  • 9
  • 46
  • 58