0

[Similar to Right way to set the MTU of an IPsec Client (Linux/Racoon), but different in that there is no router on the responder side]

I have a setup where machines in a local network need to talk to a Linux server in a datacenter. The router for the local network has a static external IP address, so I've configured a policy on both the router and the server to use IPsec in transport mode to speak to each other.

This works fine for small packets, however the server cannot accurately determine the MTU for outgoing packets, leading to connection hangs.

What is the best way to avoid these issues?

Ideas so far:

  1. limit the MTU in the routing table. This requires a static route on the server and basically works, but breaks for mobile ("roadwarrior") clients when I introduce them in two weeks.

  2. use iptables to modify the TCPMSS setting on incoming packets. This appears to have no real effect, and would not work for UDP.

Community
  • 1
Simon Richter
  • 3,317
  • 19
  • 19

2 Answers2

1

This is a bug in the Linux kernel IPsec. It fails to account for the size of the transport-mode ESP encapsulation when deciding whether to fragment the outgoing packet; it's then dropped on output as it exceeds the interface MTU. I don't know whether this has been fixed in newer kernels.

Thor Simon
  • 11
  • 1
0

1) There's no MTU for packets. MTU is for an interface. Packets size is compared to the interface MTU. 2) Router always knows MTU of its interface. This knowledge can be erroneous, but stil it does exist. 3) TCP MSS can be modified on incoming packets. And this HAS real effect on the transmission.

As about your setup - the maximum size that your IPSec channel can incapsulate without fragmentation is determined bu the IPSec headers size, thus this is dependant on the transform set and encryption ciphers. Without knowing one, it's impossible to calculate.

Still, if the sum of the payload + headers is more that the MTU of the outgoing interface, it doesn't mean that the packet will be dropped - the fragmentation may occur. Even if the encapsulating packet has the DF bit set - it may or it may not be copied to the outer header (if we speak about tunnels), and this depends on the configuration of your security endpoint.

I don't quite understand what your setup is. I guess you are using IPsec transport mode with some intermediate encapsulation, like gre or ipip.

drookie
  • 8,625
  • 1
  • 19
  • 29
  • No, I'm using transport mode directly, without another tunnel protocol. I can see that in principle this should lead to two encrypted fragments (which is suboptimal but would work), however I see that the packet is not sent at all. – Simon Richter Dec 12 '14 at 02:48
  • does it work at all ? because transport mode is for encryption between two endpoints, and that's all. It doesn't add one more IP header, thus a set of machines cannot talk to whatever is on the other side - only one machine can. – drookie Dec 12 '14 at 03:59
  • Yes, it works fine. The router does NAT regularly, then wraps IPsec around it – Simon Richter Dec 12 '14 at 19:52