Win32 sockets - Forcing ip packets to leave physical interfaces when sending to other local interfaces

Question

Summary: I'm trying to create sockets to pass data between two physical interfaces that exist on the same machine, and Win32 sockets always forwards the traffic directly in the kernel instead of pushing through the physical interfaces. Is there any way to disable this behavior, perhaps through device settings, registry tweaks, routing table shenanigans, or socket options? We're using Windows XP SP3.

Some background. I'm attempting to build some completely automated IP tests to exercise our custom IPv4 equipment. We have a large lab of Windows XP machines, and individual physical ethernet interfaces for each device we're connecting to. Our devices are effectively ethernet routers each with their own IPs.

We need to send data out our lab machines, through our devices, then back into the same computer. We will be sending Unicast and Multicast UDP, TCP, and broadcast IP traffic through the devices.

We want (and likely need) the traffic to originate on the same machine it is destined to. To do this, we configure two separate NICs each with their own IP on their own subnet, for instance NIC #1 with 10.0.0.1/24 and NIC #2 with 10.0.1.1/24. Our devices then act like simple passthrough routers, and have two interfaces, one on the 10.0.0.0/24 subnet, one on the 10.0.1.0/24 subnet, which they just forward packets back and forth from.

To generate our data, we'd like to be able to use Win32 sockets, since it is well-understood, well-supported, what our customers are using, and would probably be the most rapid approach. Packet injection is probably feasible for UDP and broadcast IP, but very likely not so for TCP. I'd entertain ideas that used packet injection, but would strongly prefer standard Win32 sockets.

As stated in the summary, the packets never leave the machine. I've googled like a madman and I've not found much. Any ideas?

Answer 1

Use Windows' command-line ROUTE utility. You can configure it so any IP packet sent to a specific IP address on a specific Subnet gets sent to another IP/device. For example:

route ADD <NIC_1_IP> MASK <NIC_1_SUBNET> <DEVICE_IP_CONNECTED_TO_NIC_2> METRIC 1
route ADD <NIC_2_IP> MASK <NIC_2_SUBNET> <DEVICE_IP_CONNECTED_TO_NIC_1> METRIC 1

Alternatively, if you know the index numbers of the NIC interfaces, you can specify them instead:

route ADD <NIC_1_IP> MASK <NIC_1_SUBNET> METRIC 1 IF <NIC_2_INTF>
route ADD <NIC_2_IP> MASK <NIC_2_SUBNET> METRIC 1 IF <NIC_1_INTF>

This way, whenever a packet is sent to NIC #1's IP, the packet goes to the device connected to NIC #2, which will then pass it on to NIC #1. And vice versa for packets sent to NIC #2's IP.

For instance, this is a useful technique for allowing WireShark to capture local IP traffic if the PC is connected to a network with a router. Packets from one local IP/Port to another local IP/Port can be bounced off the router back to the PC so they travel through physical interfaces that WireShark can monitor (WireShark will see duplicate copies of each local packet - one outbound and one inbound - but you can filter out the duplicates).

Answer 2

Winsock is always going to bring the packet data up into the kernel space and deal with it there. Thats the whole point to a generic API is that any device is dealt with at the same "layer". If you want to stick with Winsock, I don't believe you can (or would want to) work around this behavior.

You can remove some of the buffer copying with TransmitPackets or TransmitFile, but not between two device interfaces.

That being said, are you having a performance issue with the additional buffer coping that Winsock performs? Security concerns?

Answer 3

How about running the endpoints of your tester inside of distinct virtual machines? Then you need only a single piece of hardware, but you'll have separate TCP/IP stacks that don't know each other are local (and most VM solutions pass the packet straight through the host unchanged, I don't think the host is going to grab the packet and send it straight to another VM unless you configure bridging between VMs... but you'll bind each VM to a different physical network adapter).