I have a java application which uses a few MulticastSocket instances to listen
to a few UDP multicast feeds. Each such socket is handled by a dedicated thread.
The thread reads each Datagram, parses it's content and writes to log (log4j) the packet's sequence id (long) and the timestamp the Datagram was received.
When I try to run 2 instances of the same application on a Windows Server 2008 R2, with 2 * 6 cores and compare the 2 logs created by the 2 applications, I notice that quite frequently the timing of the packets isn't the same.
Most packets are received by the 2 apps at the same time (milis), but frequently there's a difference of about 1-7ms diff between the reception time of the same packet by the 2 apps.
I tried allocating more buffers in the NIC, and also made the socket read buffer bigger. In addition I tried minimizing GC runs and I also use -verbose:gc and can see that GC times and the problematic timing diff does not occur at the same time. This allows me to assume that my problem isn't GC related.
No drop packets problem was observed, and a bandwidth problem is not likely.
Ideas / Opinions are welcome. Thanks.