we are using jgroups to synchronize several java objects through our cluster. At the moment we only have two servers and we use the TCP protocol. We configured a single tcp port (42812) for synchronization.
We discovered the following problem: When the servers are running a while, 1 hour to 3 hours, then the memory usage is raising a lot. Thats why we took a heap dump and on both servers we find the following leak suspect:
The thread java.lang.Thread @ 0x5ff35fdd8 Connection.Receiver [11.63.99.51:42813
- 11.63.99.51:42813],userNotificationCounterSync,hjad-gts4-de16-38376 keeps local
variables with total size 1.650.814.560 (38,04%) bytes.
The memory is accumulated in one instance of "byte[]" loaded by "".
The stacktrace of this Thread is available. See stacktrace.
When we made a tcp dump we only saw syn ack packages sent over 42813 port, but this was after the memory had raised. So it seems to us, that the servers are receiving messages over a not configured port and do not process them. The memory usage never raises over 1.650.814.560 bytes and the synchronized containers are synchron, so we do not have problems with synchronization. We configure the JChannel in our spring xml configuration with a port_range of 0:
TCP(bind_port=12812;
port_range=0;
recv_buf_size=130k;
send_buf_size=130k;
tcp_nodelay=true;
enable_bundling=false;)
:TCPPING(initial_hosts=server1[42812],server2[42812])
:MERGE2:FD_SOCK:VERIFY_SUSPECT:pbcast.NAKACK(use_mcast_xmit=false;retransmit_timeout=300,600,1200,2400,4800;discard_delivered_msgs=true;)
:UNICAST:pbcast.STABLE(stability_delay=1000;desired_avg_gossip=50000;max_bytes=400000;)
:FRAG:pbcast.GMS(print_local_addr=true;join_timeout=3000;view_bundling=true;):pbcast.STATE
In our opinion with this configuration the nodes should not communicate through the port 42813.
How is ist possible that Connection.Receiver is receiving so much data over this port, and why is it not processed?
