Packet loss with TP-Link switch and Linux

Question

I’ve set up four Linux servers. They’ve all been imaged the exact same. I got a cheap TP Link switch which I plugged them all into. When all turned on and plugged into the switch the machines seem to experience crazy packet loss and I can’t connect to them, and the machines can’t connect to each other.

The connect issues extend to the servers trying to ping each other. I had each server get an IP and then I disconnected the switch from the router. When I have any two servers plugged in to the switch they are able to ping each other, but as soon as a plug in a third they are unable to continue pinging. Given this happens even without being plugged into the router it has to be something on the machine or the switch.

I tried taking a packet capture on one of the machines, but didn’t notice anything other than crazy amount of tcp retransmission which I think is consistent with packet loss. I was wondering if I would seem some crazy amount of multicast traffic or something, but not really. I also tried killing anything on the machines that might be doing network traffic. The adapter settings on the machines all look pretty normal to me, but I’m not sure what could cause this.

I also tried swapping the switch out with another of the same make and model, but it had the same problem. I’m wondering if there is something strange happening with the switch, but I have no insight into it.

Not sure what to try next in terms of debugging. I feel like there is something simple I’m overlooking.

Start with the essentials: IP address and netmask or prefix. How have you actually configured these machines? — Michael Hampton, Sep 12 '20 at 00:31
What is the model number of the switch and the server NICs? IPv4 or IPv6? What cabling? Cat3, Cat5 or what? — fpmurphy, Sep 12 '20 at 14:33

score 0 · Answer 1 · answered Sep 12 '20 at 03:20

Easy stuff first... make sure you don't have any duplicate IP addresses assigned and that your netmasks on all hosts are the same.

The next thing I would suggest is that you manually set the interface speed and duplex on all three hosts to 100/Full. Basically, turn off auto-negotiation.

Lots of cheap switches out there just don't auto-negotiate well. And, even if the switch does, there are cheap NICs that don't handle it well either. By manually setting each host to a fixed speed and duplex, you eliminate this as one of the factors.

score 0 · Accepted Answer · answered Nov 24 '20 at 08:37

0

Posting here in case this happens to help someone else who runs across this. It turns out LACP was enabled on the interfaces, and the switch I was using doesn't support it.

answered Nov 24 '20 at 08:37

Bob Bobbio

101
1

Packet loss with TP-Link switch and Linux

2 Answers2