How to fix java heap space on websphere when sending invalid xml

Question

when I send invalid xml to my web services on websphere server, I get java heap space:

<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:soap-enc="http://schemas.xmlsoap.org/soap/encoding/">
   <soap:Header/>
   <soap:Body>
       <MyElementsomething</MyElement>

   </soap:Body>
</soap:Envelope>

with the element

   <MyElement>something</MyElement>

Everything works ok

Exception:

java.lang.OutOfMemoryError: Java heap space
        at com.ibm.ws.webservices.utils.BAIS.create(BAIS.java:74) ~[com.ibm.ws.runtime.jar:?]
        at com.ibm.ws.webservices.utils.WebServicesInputSource.getBAISContent(WebServicesInputSource.java:744) ~[com.ibm.ws.runtime.jar:?]
        at com.ibm.ws.webservices.utils.WebServicesInputSource.getByteArrayContent(WebServicesInputSource.java:710) ~[com.ibm.ws.runtime.jar:?]
        at com.ibm.ws.webservices.utils.WebServicesInputSource.getStringContent(WebServicesInputSource.java:674) ~[com.ibm.ws.runtime.jar:?]
        at com.ibm.ws.webservices.utils.WebServicesInputSource.getString(WebServicesInputSource.java:425) ~[com.ibm.ws.runtime.jar:?]
        at com.ibm.ws.webservices.engine.SOAPPart._getSOAPEnvelope(SOAPPart.java:1090) ~[com.ibm.ws.runtime.jar:?]
        at com.ibm.ws.webservices.engine.SOAPPart.getAsSOAPEnvelope(SOAPPart.java:628) ~[com.ibm.ws.runtime.jar:?]
        at com.ibm.ws.webservices.engine.SOAPPart.getEnvelope(SOAPPart.java:656) ~[com.ibm.ws.runtime.jar:?]
        at org.springframework.ws.soap.saaj.support.SaajUtils.getSaajVersion(SaajUtils.java:155) ~[spring-ws-core-2.1.0.RELEASE.jar:?]
        at org.springframework.ws.soap.saaj.SaajSoapMessage.getSaajVersion(SaajSoapMessage.java:257) ~[spring-ws-core-2.1.0.RELEASE.jar:?]
        at org.springframework.ws.soap.saaj.SaajSoapMessage.getImplementation(SaajSoapMessage.java:342) ~[spring-ws-core-2.1.0.RELEASE.jar:?]
        at org.springframework.ws.soap.saaj.SaajSoapMessage.<init>(SaajSoapMessage.java:117) ~[spring-ws-core-2.1.0.RELEASE.jar:?]
        at org.springframework.ws.soap.saaj.SaajSoapMessageFactory.createWebServiceMessage(SaajSoapMessageFactory.java:186) ~[spring-ws-core-2.1.0.RELEASE.jar:?]
        at org.springframework.ws.soap.saaj.SaajSoapMessageFactory.createWebServiceMessage(SaajSoapMessageFactory.java:60) ~[spring-ws-core-2.1.0.RELEASE.jar:?]
        at org.springframework.ws.transport.AbstractWebServiceConnection.receive(AbstractWebServiceConnection.java:90) ~[spring-ws-core-2.1.0.RELEASE.jar:?]
        at org.springframework.ws.transport.support.WebServiceMessageReceiverObjectSupport.handleConnection(WebServiceMessageReceiverObjectSupport.java:86) ~[spring-ws-core-2.1.0.RELEASE.jar:?]

EDIT:

nobody know ?

PS: spring validation doesnt help because it is after websphere stuff. When I send invalid xml agains xsd (for example string instead of number) then it works:

2015-02-18 10:12:46.128 [WebContainer : 8] WARN  cz.isvs.ais3.ws.interceptor.ServerPayloadValidatingInterceptor - XML validation error on request: cvc-datatype-valid.1.2.1: 'b' is not a valid value for 'integer'.
2015-02-18 10:12:46.128 [WebContainer : 8] WARN  cz.isvs.ais3.ws.interceptor.ServerPayloadValidatingInterceptor - XML validation error on request: cvc-type.3.1.3: The value 'b' of element 'urn1:Ais' is not valid.

because of WebServiceMessageReceiverObjectSupport.class:

86            WebServiceMessage request = connection.receive(getMessageFactory()); // here occurs out of memory
87            MessageContext messageContext = new DefaultMessageContext(request, getMessageFactory());
88            receiver.receive(messageContext); // and in this method are interceptor which validate request

PS2:

when I tried to increase heap space from 2gb to 4gb there is another exception on the same spot:

java.lang.IndexOutOfBoundsException
        at java.io.PushbackInputStream.read(PushbackInputStream.java:181) ~[?:1.7.0]
        at java.io.FilterInputStream.read(FilterInputStream.java:133) ~[?:1.7.0]
        at com.ibm.ws.webservices.utils.WebServicesInputSource$FilterIS.read(WebServicesInputSource.java:805) ~[com.ibm.ws.runtime.jar:?]
        at com.ibm.ws.webservices.utils.BAIS.create(BAIS.java:77) ~[com.ibm.ws.runtime.jar:?]

PS3:

this is correct behavior with sun java:

Caused by: org.xml.sax.SAXParseException: The element type "urn:ZadostInfo" must be terminated by the matching end-tag "</urn:ZadostInfo>".
        at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source) ~[?:?]
        at org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source) ~[?:?]
        at org.xml.sax.helpers.XMLFilterImpl.parse(Unknown Source) ~[?:?]
        at org.apache.xalan.transformer.TransformerIdentityImpl.transform(Unknown Source) ~[?:?]
        at com.sun.xml.internal.messaging.saaj.util.transform.EfficientStreamingTransformer.transform(EfficientStreamingTransformer.java:411) ~[?:1.7.0]
        at com.sun.xml.internal.messaging.saaj.soap.EnvelopeFactory.createEnvelope(EnvelopeFactory.java:115) ~[?:1.7.0]
        at com.sun.xml.internal.messaging.saaj.soap.ver1_1.SOAPPart1_1Impl.createEnvelopeFromSource(SOAPPart1_1Impl.java:80) ~[?:1.7.0]
        at com.sun.xml.internal.messaging.saaj.soap.SOAPPartImpl.getEnvelope(SOAPPartImpl.java:140) ~[?:1.7.0]
        at org.springframework.ws.soap.saaj.support.SaajUtils.getSaajVersion(SaajUtils.java:155) ~[spring-ws-core-2.1.2.RELEASE.jar:?]

Have you already tried to extend the heap sace for the JVM settings? — ZeusNet, Feb 17 '15 at 08:48
not yet because I see no reason when it works when I send valid xml message — hudi, Feb 17 '15 at 08:53
Which size is your max heap size in your websphere configuration? The heap error normally says that there are to many objects initalized at this time. So i thougt if you would increase the heap size it should fix the issue — ZeusNet, Feb 18 '15 at 15:08
I increase max heap size to 4gb as you can see in "PS2:" but now there is another exception. I think there is bug in ibm java because when I use sun then it work ok with 2 gb — hudi, Feb 18 '15 at 15:44
Sounds very interessting, i could be possible that its an bug in the IBM implementation. But im not very familiar with these implemenation — ZeusNet, Feb 18 '15 at 18:44
This looks like a WebSphere bug to me, so I'd be raising a support incident with IBM — Steve C, Feb 23 '15 at 12:30
hm and how ? I am not partner of IBM. How should I report this bug — hudi, Feb 23 '15 at 12:45
WebSphere is a licensed product. It comes with support. However, sometimes you can get help at the [WebSphere Application Server forums](https://www.ibm.com/developerworks/community/forums/html/forum?id=11111111-0000-0000-0000-000000000266) — Steve C, Feb 24 '15 at 02:19
I already add this topic to their forum: https://www.ibm.com/developerworks/community/forums/html/topic?id=ad09828d-e2d4-4b77-89cf-e182b211b23b but still with no success — hudi, Feb 24 '15 at 08:33

score 3 · Answer 1 · 2015-06-14T11:51:12.117

This looks like a Websphere bug. Using Spring-WS I got the some error:

java.lang.OutOfMemoryError: Java heap space at com.ibm.ws.webservices.utils.BAIS.create(BAIS.java:74)

I managed to solved this way: put in /AppServer/lib/ext this library: axis2-saaj-1.6.2.jar (worked for me on Websphere v8.5 and Spring-WS v2.2.0, but you may have to change the version).

Explanation

When Spring-WS asks for a javax.xml.soap.MessageFactory implementation it gets a com.ibm.ws.webservices.engine.soap.MessageFactoryImpl instance (from Websphere runtime) instead of a com.sun.xml.internal.messaging.saaj.soap.MessageFactoryImpl instance (from JVM), which leads to the mentioned error. The axis2-saaj library also contains a javax.xml.soap.MessageFactory implementation: org.apache.axis2.saaj.MessageFactoryImpl. This one is now used when axis2-saaj is in lib/ext folder and error is gone (I also configured PARENT LAST classpath, don't know if it matters here)

Update

@hudi your solution is correct, I just tested it. Sadly, cannot upvote you.

score 2 · Answer 2 · answered Feb 17 '15 at 09:51

2

the best way is to use spring payloadinterceptor to validate the incoming xml against the XML schema before it allows to be parsed further... Sample code:

bean id="validatingInterceptor"
    class="org.springframework.ws.soap.server.endpoint.interceptor.PayloadValidatingInterceptor">
<property name="schema" value="/WEB-INF/conf/MyElement.xsd"/>
<property name="validateRequest" value="true"/>
<property name="validateResponse" value="false"/>

For more details, read the spring documentation

answered Feb 17 '15 at 09:51

ag112

5,537
2
23
42

I am validation xml but this websphere stuff are before this validation – hudi Feb 17 '15 at 13:28
can u describe how u r validating? – ag112 Feb 17 '15 at 18:37
what you want to describe ? I have validation similar that you posted – hudi Feb 18 '15 at 08:46
if you have defined your validating interceptor, then this should be called before websphere based library is invoked for actually parsing of message....so in case invalid xml, the processing should stops there with SOAP fault to consumer.....do one thing, define user defined custom interceptor and see whether interceptor is really being invoked... – ag112 Feb 18 '15 at 09:47
sorry but you are wrong see my updated answer. interceptors are called AFTER webshere stuff about saaj – hudi Feb 18 '15 at 10:05
this sounds does not right to me...there is no point of validation post message parsing......could you please specify bean definition of webserviceTemplate you are using..... – ag112 Feb 23 '15 at 07:10
why ? your validation above validate just xml against xsd. But validation of well formed xml must be before this this. – hudi Feb 23 '15 at 09:52
XML well formedness check is performed separately and independently only if there are multiple XML processing nodes it your system...then it make sense to check XML well formedness...otherwise XML schema validation and well-formedness are combined function.....please publish bean definition of webserviceTemplate...I should be able to solve your issue.... – ag112 Feb 24 '15 at 07:13

score 2 · Answer 3 · answered Feb 23 '15 at 20:12

You just found a bug. Please report the bug. Since this is related to the parsing. You can do two things.

First of all you can take a memory dump. Just dump the memory (or profile this using visualvm and make a memory profile run). This way you understand what polutes the memory.

Second you can correct the issue you found. I do it all the way with third party code. So just before they step into parsing the snippet you do another step evaluating if the xml is correct without checking for shema. Just use a parser run with no shema involved.

In order to provide a modified implementation use the sourcecode or if not available, decompile it or rename the class and provide a new implementation class extending from this. If it is a private method or feature you need to access just modify the byte code.

In order to inject your version just add your class to your class path (or the classpath of the websphere) and put it in front of it. Usually your code is preceeded for any library code. This way you can override every implementation detail of any library as long as you control the classpath (usually by setting it within the java.exe.

So just add an additional test for correctness before this bugged version is issued. And report the bug and also blog about it.

If you have questions or need a tool advice just ask. I do this on a regular basis. With open source and non-open source software. Also great for other things like extending features.

hm and how you extending non-open source software ? I tried to check well formed xml with spring-ws before websphere do it so I hope it is ok. My example also work on another servers (I tried tomcat) but it is not necessary — hudi, Feb 24 '15 at 08:53
Non-open source software is just software. Bytecode can be decompiled and if you havent done this try it with your own code. It is amazing how it looks. Just like the sources. If they obfusicate just rename the class (byte code manipulation) and provide your own code extending from this renamed one. Its that simple. Try it yourself it is a great asset in your toolbox. — Martin Kersten, Feb 24 '15 at 22:21

score 2 · Accepted Answer · answered Feb 24 '15 at 08:44

I tried to fix this by my own. I examined how spring-ws validate xml and tried to do it before websphere. Here is code. I test it and it is working for me.

public class SaajSoapMessageValidatorFactory extends SaajSoapMessageFactory {

    @Override
    public SaajSoapMessage createWebServiceMessage(InputStream inputStream) throws IOException {
        InputStream original;

        try {
            ByteArrayOutputStream baos = new ByteArrayOutputStream();
            IOUtils.copy(inputStream, baos);

            original = new ByteArrayInputStream(baos.toByteArray());
            InputStream validate = new ByteArrayInputStream(baos.toByteArray());

            try {
                ValidatorResources validator = new XmlValidatorResources(validate);
                validator.process();
            } catch (Exception parseException) {
                throw new InvalidXmlException("Could not parse XML", parseException);
            } finally {
                IOUtils.closeQuietly(validate);
            }
        } finally {
            IOUtils.closeQuietly(inputStream);
        }

        return super.createWebServiceMessage(original);
    }
}



public class XmlValidatorResources extends ValidatorResources {

    private static final long serialVersionUID = 1L;

    public XmlValidatorResources(InputStream in) throws IOException, SAXException {
        Digester digester = new Digester();
        digester.push(this);
        digester.parse(in);
    }

}

and in spring context I add:

<bean id="messageFactory" class="cz.isvs.ais3.ws.validator.SaajSoapMessageValidatorFactory" />

score 2 · Answer 5 · answered Feb 25 '15 at 15:38

After Martin advise I try to decompile ibm code and I think problem is in this procedure:

private SOAPEnvelope _getSOAPEnvelope()
    throws WebServicesFault
{
    SOAPEnvelope soapEnvelope = _getCachedSOAPEnvelope();
    if(soapEnvelope == null)
        if(currentForm == 4)
        {
            soapEnvelope = (SOAPEnvelope)getCurrentMessage();
        } else
        {
            if(log.isDebugEnabled())
                log.debug("Converting " + formNames[currentForm] + " to " + formNames[4]);
            MessageContext mc = getMessage().getMessageContext();
            if(mc != null)
                if(mc.isHighFidelity())
                {
                    if(originalContent == null)
                        originalContent = _getCachedWebServicesInputSource();
                } else
                {
                    originalContent = null;
                }
            WebServicesInputSource wsis = _getWebServicesInputSource();
            if(log.isDebugEnabled())
                log.debug("Parsing inputSource " + wsis.dumpStats());
            String encoding = getMessage().getCharacterEncoding();
            if(encoding != null)
                wsis.setEncoding(encoding);
            DeserializationContext dser = new DeserializationContextImpl(wsis, getMessage().getMessageContext(), getMessage().getMessageType(), getMessage().getSOAPConstants(), Utils._isSAAJ13Enabled());
            try
            {
                dser.parse();
            }
            catch(SAXException e)
            {
                FFDCFilter.processException(e, "com.ibm.ws.webservices.engine.SOAPPart.getSOAPEnvelope", "964", this);
                Exception real = e.getException();
                wsis.reset();
                if(real instanceof WebServicesFault)
                    throw (WebServicesFault)real;
                if(originalContent == null)
                    originalContent = wsis;

problem section:

                if(originalContent == null)// I am not sure but I think this will never happend. So to fix it I will delete this condition and  throw this 
exception not the second one where it want to display message
                        throw WebServicesFault.makeFault("To see the message containing the parsing error in the log, either enable web service engine tracing or set MessageContext.setHighFidelity(true).", ((Throwable) (real == null ? ((Throwable) (e)) : ((Throwable) (real)))));
                    else
                        throw WebServicesFault.makeFault("Message being parsed: " + originalContent.getString(), ((Throwable) (real == null ? ((Throwable) (e)) : ((Throwable) (real)))));
                }
                if(log.isDebugEnabled())
                    log.debug("Done parsing inputSource.");
                originalParseLength = wsis.getContentLength();
                soapEnvelope = dser.getEnvelope();
                setCurrentForm(soapEnvelope, 4);
                String messageProtocol = getMessage().getSOAPConstants().getSOAPProtocol();
                String factoryProtocol = soapEnvelope.getSOAPFactory().getSOAPConstants().getSOAPProtocol();
                String envelopeProtocol = "SOAP 1.1 Protocol";
                String ns = soapEnvelope.getNamespaceURI();
                if(soapEnvelope.getNamespaceURI().equals("http://schemas.xmlsoap.org/soap/envelope/"))
                    envelopeProtocol = "SOAP 1.2 Protocol";
                if(messageProtocol.equals("Dynamic Protocol"))
                {
                    getMessage().setSOAPConstants(soapEnvelope.getSOAPFactory().getSOAPConstants());
                    messageProtocol = getMessage().getSOAPConstants().getSOAPProtocol();
                }
                if(envelopeProtocol.equals(factoryProtocol) && factoryProtocol.equals(messageProtocol))
                {
                    if(log.isDebugEnabled())
                        log.debug("The protocol is: " + envelopeProtocol);
                } else
                if(log.isDebugEnabled())
                {
                    log.debug("The SOAPEnvelope protocol is: " + envelopeProtocol);
                    log.debug("The SOAPFactory protocol is: " + factoryProtocol);
                    log.debug("The SOAPMessage protocol is: " + messageProtocol);
                }
            }
        return soapEnvelope;
    }

score 2 · Answer 6 · answered Jul 29 '16 at 18:08

2

Refer to IBM developer wroks post

https://developer.ibm.com/answers/questions/291621/how-to-resolve-outofmemoryerror-when-sending-inval.html

answered Jul 29 '16 at 18:08

Krishna Jaladhi

66
3

score 1 · Answer 7 · edited Jul 30 '16 at 04:16

Customer using WebSphere 8.5.5.x with SDK 1.7 and using SAAJ with Spring frmae work may encounter OutOfMemoryError when sending SOAP request that that has invalid request. This is due to bug fix went into SDK 1.7 and later versions.

IBM has fixed this issue through IBM SDK APAR IV73614.

Details of the bug fix went in later releases

JAVA.IO.PACKEDINPUTSTREAM.AVAILABLE() RETURNS INTEGER.MAX_VALUE I.E., 2147483647 ON EMPTY STREAM IN JAVA 7

From javacore stack indicate following 4XESTACKTRACE at com/ibm/ws/webservices/utils/BAIS.create(BAIS.java:74) 4XESTACKTRACE at com/ibm/ws/webservices/utils/WebServicesInputSource.getBAISContent(WebServicesInputSource.java:744) 4XESTACKTRACE at com/ibm/ws/webservices/utils/WebServicesInputSource.getByteArrayContent(WebServicesInputSource.java:710) 4XESTACKTRACE at com/ibm/ws/webservices/utils/WebServicesInputSource.getStringContent(WebServicesInputSource.java:674) 4XESTACKTRACE at com/ibm/ws/webservices/utils/WebServicesInputSource.getString(WebServicesInputSource.java:425(Compiled Code)) 4XESTACKTRACE at com/ibm/ws/webservices/engine/SOAPPart._getSOAPEnvelope(SOAPPart.java:1090(Compiled Code)) 4XESTACKTRACE at com/ibm/ws/webservices/engine/SOAPPart.getAsSOAPEnvelope(SOAPPart.java:628(Compiled Code)) 4XESTACKTRACE at com/ibm/ws/webservices/engine/SOAPPart.getEnvelope(SOAPPart.java:656(Compiled Code)) 4XESTACKTRACE at org/springframework/ws/soap/saaj/support/SaajUtils.getSaajVersion(SaajUtils.java:155) 4XESTACKTRACE at org/springframework/ws/soap/saaj/SaajSoapMessage.getSaajVersion(SaajSoapMessage.java:257) 4XESTACKTRACE at org/springframework/ws/soap/saaj/SaajSoapMessage.getImplementation(SaajSoapMessage.java:342(Compiled Code))

How to fix java heap space on websphere when sending invalid xml

7 Answers7