No SOAP message exchange between Metro client and WCF service on 64Bit Win7 and Windows 2008 Server

Question

I have a running web service connection between a Java client program with the Metro {2.2.1-1} web service stack and a WCF {.NET 4.0 v30319} wsHttpBinding web service on Windows XP SP3.

If I move the exact same setup to a Windows 7 {with some enterprise setup} or Windows 2008 R2 Server SP1 {from MS DVD}, I get hanging requests from the Java client to the WCF service. I.e. there are no symptoms of any data exchange between the two partners {I have -Dcom.sun.xml.ws.transport.http.client.HttpTransportPipe.dump=true on the client side and diagnostic print messages on the server side -- both without output}. Networkologically, a TCP connection is open ("netstat -a") until a timeout occurs after 200+-5 seconds with the following stack trace:

Jul 19, 2013 12:13:00 PM ch.xxxxxxxx.xxxxx.balance.client.start.Start main
SEVERE: null
com.sun.xml.ws.streaming.XMLStreamReaderException: XML reader error: com.ctc.wstx.exc.WstxIOException: Connection reset
    at com.sun.xml.ws.streaming.XMLStreamReaderUtil.wrapException(XMLStreamReaderUtil.java:326)
    at com.sun.xml.ws.streaming.XMLStreamReaderUtil.next(XMLStreamReaderUtil.java:99)
    at com.sun.xml.ws.streaming.XMLStreamReaderUtil.nextContent(XMLStreamReaderUtil.java:169)
    at com.sun.xml.ws.streaming.XMLStreamReaderUtil.nextElementContent(XMLStreamReaderUtil.java:104)
    at com.sun.xml.ws.wsdl.parser.RuntimeWSDLParser.parseBinding(RuntimeWSDLParser.java:584)
    at com.sun.xml.ws.wsdl.parser.RuntimeWSDLParser.parseWSDL(RuntimeWSDLParser.java:470)
    at com.sun.xml.ws.wsdl.parser.RuntimeWSDLParser.parseImport(RuntimeWSDLParser.java:427)
    at com.sun.xml.ws.wsdl.parser.RuntimeWSDLParser.parseImport(RuntimeWSDLParser.java:835)
    at com.sun.xml.ws.wsdl.parser.RuntimeWSDLParser.parseWSDL(RuntimeWSDLParser.java:464)
    at com.sun.xml.ws.wsdl.parser.RuntimeWSDLParser.parse(RuntimeWSDLParser.java:232)
    at com.sun.xml.ws.wsdl.parser.RuntimeWSDLParser.parse(RuntimeWSDLParser.java:192)
    at com.sun.xml.ws.wsdl.parser.RuntimeWSDLParser.parse(RuntimeWSDLParser.java:161)
    at com.sun.xml.ws.client.WSServiceDelegate.parseWSDL(WSServiceDelegate.java:328)
    at com.sun.xml.ws.client.WSServiceDelegate.<init>(WSServiceDelegate.java:290)
    at com.sun.xml.ws.client.WSServiceDelegate.<init>(WSServiceDelegate.java:217)
    at com.sun.xml.ws.client.WSServiceDelegate.<init>(WSServiceDelegate.java:199)
    at com.sun.xml.ws.client.WSServiceDelegate.<init>(WSServiceDelegate.java:195)
    at com.sun.xml.ws.spi.ProviderImpl.createServiceDelegate(ProviderImpl.java:112)
    at javax.xml.ws.Service.<init>(Unknown Source)
    at ch.xxxxxxxx.xxxxx.balance.client.SystemIntegrationServiceBridge.<init>(SystemIntegrationServiceBridge.java:50)
    at ch.xxxxxxxx.xxxxx.balance.client.start.Start.main(Start.java:37)
Caused by: com.ctc.wstx.exc.WstxIOException: Connection reset
    at com.ctc.wstx.sr.StreamScanner.constructFromIOE(StreamScanner.java:625)
    at com.ctc.wstx.sr.StreamScanner.loadMoreFromCurrent(StreamScanner.java:1049)
    at com.ctc.wstx.sr.StreamScanner.parseLocalName2(StreamScanner.java:1857)
    at com.ctc.wstx.sr.StreamScanner.parseLocalName(StreamScanner.java:1817)
    at com.ctc.wstx.sr.BasicStreamReader.handleStartElem(BasicStreamReader.java:2925)
    at com.ctc.wstx.sr.BasicStreamReader.nextFromTree(BasicStreamReader.java:2817)
    at com.ctc.wstx.sr.BasicStreamReader.next(BasicStreamReader.java:1065)
    at com.sun.xml.ws.util.xml.XMLStreamReaderFilter.next(XMLStreamReaderFilter.java:96)
    at com.sun.xml.ws.streaming.XMLStreamReaderUtil.next(XMLStreamReaderUtil.java:80)
    ... 19 more
Caused by: java.net.SocketException: Connection reset
    at java.net.SocketInputStream.read(Unknown Source)
    at java.net.SocketInputStream.read(Unknown Source)
    at java.io.BufferedInputStream.fill(Unknown Source)
    at java.io.BufferedInputStream.read1(Unknown Source)
    at java.io.BufferedInputStream.read(Unknown Source)
    at sun.net.www.MeteredStream.read(Unknown Source)
    at java.io.FilterInputStream.read(Unknown Source)
    at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(Unknown Source)
    at com.ctc.wstx.io.BaseReader.readBytes(BaseReader.java:155)
    at com.ctc.wstx.io.UTF8Reader.loadMore(UTF8Reader.java:368)
    at com.ctc.wstx.io.UTF8Reader.read(UTF8Reader.java:111)
    at com.ctc.wstx.io.ReaderSource.readInto(ReaderSource.java:87)
    at com.ctc.wstx.io.BranchingReaderSource.readInto(BranchingReaderSource.java:57)
    at com.ctc.wstx.sr.StreamScanner.loadMoreFromCurrent(StreamScanner.java:1046)
    ... 26 more

If I terminate the service during this period, the client stops waiting with a similar stack trace immediately, and, as expected, no TCP connection is existing anymore.
Activating WCF logging (see http://msdn.microsoft.com/en-us/library/ms730064.aspx) shows a log according to which WCF thinks, it has delivered the three parts of the WSDL (/?wsdl, /?wsdl=wsdl0, /?wsdl=wsdl1) completely and successfully.
All is running as Administrator, UAC is switched off, it doesn't matter, if I have the firewall off or on or if I have IPv6 off or on. I tried JRE 7_u17 32bit, 7_u25 32bit and 7_u25 64bit.

SoapUI 4.5.1 perfectly communicates with the service locally on the win7-oid platform.
A WCF client communicates with the service locally and without proxy.
A small Java web service client using the Metro libraries, calling a corresponding WCF web service locally, works fine.
If I install the http proxy Fiddler on the win7-oid platforms and reconfigure my Java client app to use the proxy {-Dhttp.proxyHost=localhost -Dhttp.proxyPort=8888 -Dhttp.nonProxyHosts=""}, all is fine.
If the client side is on the XP box and the service on the W7oid one or the other way around, remotely accessing the WCF service works fine without any tricks.

Running out of imagination, what may be the cause for this strange behaviour, I'd like to ask the following questions:
- is there anybody out there, who experienced a similar problem
- any propositions, which Windows mechanism could interfere in the described manner
- any propositions, which other experiments I could take on to get nearer to a solution
- which additional diagnostic measures should I take to boil it down to the very problem [and the solution, I hope]

score 0 · Answer 1 · answered Apr 28 '14 at 13:11

The symptoms above show, that the problem

arises during retrieval of [parts of] the WSDL
occurs, when directly using a W7oid network connection .

This gave rise of a further analysis of what happened on the TCP level. First of all, it was not possible to see any local IP traffic of my web service connection with Wireshark 1.10.2. Even the IP traffic via the physical IP interface was not visible. Also, most of the other at least somehow freely available network anaysers did not seem to support analysis of IP traffic on the loopback interface of W7 and Windows 2008 Server.

Finally, I was able to get a bit of more insight with the trial versions of CommView and LocalNetworkMonitor:

also IP traffic, configured to use the physical IP interface is [secretly] routed via the local network stack as if it were a loopback connection.
such loopback-like IP traffic seems to work on a technically different IP stack on W7oid platforms. Otherwise, tools would not distinguish between the two situations.
on the TCP level, the client begins with opening a http-TCP connection to retrieve "?wsdl". ("?wsdl" contains references to the two large wsdl files "?wsdl=wsdl0" and "?wsdl=wsdl1") Then, almost simultanously, the client requests "?wsdl=wsdl0" on the same first TCP connection (probably due to keep-alive) and a newly opened second TCP connection. One of these two connections is then hanging around unread and unterminated. Only after a timeout of around 200 seconds, some additional chunk of data gets on the way and the TCP connections are both closed (immediately followed by the client stack trace above).

Conclusions:
At least with HTTP-keep-alive, which is the standard setting for WCF and Metro, the way, W7 and W2k8 server handle HTTP-TCP traffic via the loopback interface and the way, WCF and Metro expect it, do not fit together.
In the given scope, it was not possible to boil it down to the very reason. However, it seems the W7 loopback network stack is very picky. Even the network monitoring tools have severe problems to offer correctly working solutions.

The workaround...
... was to configure the web service channel to the physical IP interface AND set a high priority IP route on the W7/W2k8 box, which directs traffic to this interface via the IP gateway. Only then could W2k8 have been forced to refrain from loopbacking and really deliver IP traffic to the physical network interface via its "usual" IP stack -- on which everything, ie. web service and tools, works fine.

score -1 · Answer 2 · answered Jul 25 '13 at 03:18

Are you sure that all connections are being closed properly? The symptoms you describe are often seen in the .NET world when a .Close() was forgotten on a prior response stream.

On later requests, the client ends up blocking while waiting for a connection to become available (due to the HTTP connection limit). http://fiddler2.com/blog/blog/2013/02/28/help!-running-fiddler-fixes-my-app-

No SOAP message exchange between Metro client and WCF service on 64Bit Win7 and Windows 2008 Server

2 Answers2