0

For a driver test I'm working on, I need to connect 2 Linux machines through a third Linux machine which acts as a router. Each machine has a proprietary network device with 2 ports so that one port is under subnet 11.x.x.x, and the second under subnet 12.x.x.x. The service port through which I remotely connect is under 10.x.x.x subnet. The test is done only on the proprietary devices in the 11 and 12 subnets.

An illusatration of the machines configuration:

A(11.0.0.1) <-> B(11.0.0.2) ||- C(11.0.0.3)

A(12.0.0.1) -|| B(12.0.0.2) <-> C(12.0.0.3)

(Literal description of the illustration: A's fist port is connected to B's first port, B's second port is connected to C's second port, and A's second port, and C's first port are disconnected.)

I've configured B (the routing machine) to forward IP requests. Then I configured A and C using the ip route shell command like so:

A

ip route add "12.0.0.0/16" via 11.0.0.2

B

ip route add "11.0.0.0/16" via 12.0.0.2

This works. I ping 12.0.0.3 from A with any packet size and it works, and vice versa. The problem is that my TCP code doesn't work properly between A and C. It only works with adjacent machines like A+B and B+C.

A simple python script that sends an "Hello World!" string over TCP works, but when the same script sends message larger than 1450 bytes, nothing goes through. The connection is established between the 2 hosts, but the information doesn't come through. It's important to mention here again that ping with packets larger than 1450 bytes works.

I think I'm probably doing something wrong here with the configuration of the routing machine or incomplete configuration in the ip route.

What could cause such a problem?

Avihu Turzion
  • 103
  • 2
  • Strange your 'ip route add' commands do not cover the network ranges you discuss. –  Jul 02 '09 at 08:46
  • Oops... the IP addresses were incompatible, but I was just a bit confused writing this question. I fixed it, and this incompatibility was only in the question. I thought of asking this on serverfault.com, but wasn't sure. Maybe I should really try. –  Jul 02 '09 at 10:50
  • How can this be a routing problem if message of one size routes. Have you tried running wireshark on the three nodes (A,B & C) –  Jul 06 '09 at 18:02

5 Answers5

2

Just an idea, the MTU (Maximum Transmission Unit) is usually 1500 bytes. Check with ifconfig. 1450 sounds very close to 1500 - IP+TCP headers. If you raise the MTU on both interfaces on all machines, are you then able to transmit larger TCP packets?

ifconfig <device> mtu 2000

I dunno about the fact that ping packets above 1450 bytes works.

bjarkef
  • 111
  • 1
  • 1
  • 7
  • The 1450 in the original question is a dead giveaway that this is MTU rather than routing. I think the solution is to configure the MTU down rather than up, so the senders fragment packets. For some reason path MTU discovery isn't working. – pjc50 Jul 24 '09 at 09:04
1

  1. Ensure that there is no firewall on router (11.0.0.2 / 12.0.0.2). Use iptables -F.

  2. Make sure ip forwarding got enabled. It requires reboot if you just change in /etc/sysctl.conf. Use sysctl net.ipv4.ip_forward to see whether ip forwarding is enabled or not.

  3. If above things dont work then run wireshark or at least tcpdump on all three nodes and then run your program.

Saurabh Barjatiya
  • 4,703
  • 2
  • 30
  • 34
0

Wow, I've neglected this question. I've switched a workplace since then.

But the answer was simple - turns out it didn't work because of a bug in the driver, which is exactly what we were testing. It was just such a basic component in the driver that failed, that we didn't think the described problem can be caused by a failure in the driver.

Thanks for the help though :)

Avihu Turzion
  • 103
  • 2
0

Do you have network filtering (like iptables) running on the router?

If you filter by state and allow NEW, you should also allow RELATED and ESTABLISHED packets too. Otherwise only the first packet will be allowed, which looks like what you have here.

0

Just a tiny remark: you specify the above networks as /16, but these are normally /8 networks you want to interconnect. Are the target hosts for testing on 11.0.. and 12.0.. and are the routers set to work with the correct networks?

dadver
  • 183
  • 1
  • 12