1

I am running into a very strange problem and I am not sure what is going on. I have tried many permutations and I cannot resolve this. Any insight would be very appreciated. Here is my environment:

I have set up an archiva and an artifactory repo in a staging area that can only be accessed over a Cisco VPN. I do not need both caches, but I have experimented with both trying to solve the problem.

I build my software using gradle, but it does not seem to be the problem. When my project builds, it happily goes along for quite a while downloading POMs and JARs. Every time however, it gets "stuck" on a jar. For that particular local snapshot of the downloads, I can run the build N times, and it will hang in the same spot. If I delete my local cache, it may hang on a different jar.

The VPN transfer rate is not great (100-200K/s) and right before it hangs, it slows down significantly, all the way to 0. Also, if I build in a staging area server, the transfers are over a 10G network and everything works just fine.

I have used wget to download the jars and the output generally looks like this:

/tmp > wget -vS http://git-jenkins01.aus.sf:8082/artifactory/ecomm-snapshot-plus-remote/org/codehaus/groovy/groovy/1.8.6/groovy-1.8.6.jar
--2012-10-11 09:51:41--  http://git-jenkins01.aus.sf:8082/artifactory/ecomm-snapshot-plus-remote/org/codehaus/groovy/groovy/1.8.6/groovy-1.8.6.jar
Resolving git-jenkins01.aus.sf (git-jenkins01.aus.sf)... 10.57.10.226
Connecting to git-jenkins01.aus.sf (git-jenkins01.aus.sf)|10.57.10.226|:8082... connected.
HTTP request sent, awaiting response... 
  HTTP/1.1 200 OK
  Server: Artifactory/2.6.4
  X-Artifactory-Id: 583c10bfdbd326ba:71d02e01:13a5059d090:-8000
  Last-Modified: Thu, 09 Feb 2012 17:57:32 GMT
  ETag: 553ca93e0407c94c89b058c482a404427ac7fc72
  X-Checksum-Sha1: 553ca93e0407c94c89b058c482a404427ac7fc72
  X-Checksum-Md5: e7ddf15d2f343537549dbbfd860c5f5b
  Content-Disposition: attachment; filename=groovy-1.8.6.jar
  X-Artifactory-Filename: groovy-1.8.6.jar
  Content-Type: application/java-archive
  Content-Length: 5546084
  Date: Thu, 11 Oct 2012 15:51:47 GMT
Length: 5546084 (5.3M) [application/java-archive]
Saving to: `groovy-1.8.6.jar.7'

95% [======================================================================================================================================>       ] 5,278,650   --.-K/s  eta 3s

It always stops at ~90% or more, but it will not finish. The logs give no indication of an error. I have tried tomcat 6, tomcat 7, jetty, java 6, java 7, a squid proxy in the artifactory server to attempt to trick it into a local download.

I inspected the traffic with Charles and looked at the logs, and as best as I can tell, the server thinks it sent all the data and the client gets stuck waiting for the last few bytes. I am completely out of ideas. Any thoughts?

Julio Garcia
  • 1,087
  • 3
  • 14
  • 22

1 Answers1

2

I saw this happening due to Jumbo packets getting converted to standard TCP/IP packet. What happened is that Tomcat is sending big jumbo packet by default (buffer size -MTU- of 9000 bytes), which get cut into smaller packets by the network routers since the network does not support Jumbo packets. So, at some points Tomcat send the "end of packets" block which get received "before" all the other sub packets arrived. Conclusion, it hangs or get disconnected before all data are received.

The workaround is to change the buffer size in the tomcat server.xml to something around 1500, or force the TCP/IP card configuration of the server to 1500 MTU (annoying for 1G local network).

Hope this helps.

Fred Simon
  • 567
  • 3
  • 10