(Trying this forum after the question was deemed off-topic for Stack Overflow & Network Engineering)
I am investigating an issue where we see occasional 200ms+ spikes in a simple tcp client/server application. These spikes are always in 200ms - 220ms range indicating RTO based retransmission is being employed for such cases. For e.g. here is an example from tcpdump:
1. 2022-08-30 05:42:12.354834 Client > Server: Flags [P.], cksum 0x3aa4 (incorrect -> 0xbd21), seq 1337931499:1337964267, ack 71199642, win 54, options [nop,nop,TS val 3783770243 ecr 1428361383], length 32768
2. 2022-08-30 05:42:12.358580 Client > Server: Flags [P.], cksum 0xd09b (incorrect -> 0x3c8b), seq 1337958643:1337964267, ack 71199642, win 54, options [nop,nop,TS val 3783770247 ecr 1428361383], length 5624
3. 2022-08-30 05:42:12.358615 Server > Client: Flags [.], cksum 0x8759 (correct), ack 1337931499, win 986, options [nop,nop,TS val 1428361888 ecr 3783769742,nop,nop,sack 1 {1337958643:1337964267}], length 0
# 200ms wait here
4. 2022-08-30 05:42:12.563601 Client > Server: Flags [.], cksum 0xddfb (incorrect -> 0xce54), seq 1337931499:1337940547, ack 71199642, win 54, options [nop,nop,TS val 3783770452 ecr 1428361888], length 9048
- Client sends a 32KB request to server. Our MTU is configured to be 9100.
- Client does not receive any acks from server in next 3ms. So, some sort of early retransmission logic kicks in. Client retransmits the last segment (last 5624 bytes).
- Server sends an ack back indicating receipt of those 5624 bytes (
sack 1 {1337958643:1337964267}). Overall ack number is not advanced as expected. - Client waits 200ms+ before it retransmits other segments.
This is just one example. There are other cases too for e.g. where after client sends 32KB request, server sends an ack back indicating only 3 segments were received. So, client retransmits the last segment but after waiting 200ms.
My question here is - why is Client waiting 200ms before it retransmits the lost segments even though Server has already sent an ack for the received segments? Why is it not retransmitting right away as soon as it has received ack from Server, or employing other fast retransmission mechanisms?
Has it something to do with the win window length (986 in step 3) in Server's acks?
Any insights appreciated!
