IP_TRANSPARENT SYN not getting SYN+ACK response on localhost

Question

I am writing the following program (though I don't think that this is where the problem is):

#include <arpa/inet.h>
#include <netinet/ip.h>
#include <sys/types.h>
#include <sys/socket.h>

#include <stdio.h>
#include <stdlib.h>

int main() {
    int sock=socket(AF_INET, SOCK_STREAM, 0);
    if( sock<0 ) {
        perror("socket creation failed");
        return 1;
    }

    int ip_transparent_enabled = 1;
    if( setsockopt(sock, IPPROTO_IP, IP_TRANSPARENT, &ip_transparent_enabled, sizeof(ip_transparent_enabled))<0 ) {
        perror("Setting IP_TRANSPARENT failed");
        return 1;
    }

    struct sockaddr_in bind_addr = { AF_INET, htons(31337) };
    inet_aton("93.184.216.34", &bind_addr.sin_addr); // example.com

    if( bind(sock, (const struct sockaddr *)&bind_addr, sizeof(bind_addr))<0 ) {
        perror("bind failed");
        return 1;
    }

    struct sockaddr_in dest = { AF_INET, htons(7007) };
    inet_aton("127.0.0.1", &dest.sin_addr);

    if( connect(sock, (const struct sockaddr *)&dest, sizeof(dest))<0 ) {
        perror("Connect failed");
        return 1;
    }
}

Port 7007 is running an echo server, but it's not important, because the program there never receives the connection.

When I run tcpdump, I see that the SYN is sent with the correct (made up) source address, but SYN+ACK is issued on neither loopback nor eth0:

$ sudo tcpdump -i any port 31337                                                                                                                                                       
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode                                                                                                                                        
listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes                                                                                                                                   
22:22:41.475942 IP 93.184.216.34.31337 > localhost.afs3-bos: Flags [S], seq 2953286612, win 65495, options [mss 65495,sackOK,TS val 443673031 ecr 0,nop,wscale 7], length 0                                       
22:22:42.478172 IP 93.184.216.34.31337 > localhost.afs3-bos: Flags [S], seq 2953286612, win 65495, options [mss 65495,sackOK,TS val 443674033 ecr 0,nop,wscale 7], length 0                                       
22:22:44.494174 IP 93.184.216.34.31337 > localhost.afs3-bos: Flags [S], seq 2953286612, win 65495, options [mss 65495,sackOK,TS val 443676049 ecr 0,nop,wscale 7], length 0                                       
22:22:48.590423 IP 93.184.216.34.31337 > localhost.afs3-bos: Flags [S], seq 2953286612, win 65495, options [mss 65495,sackOK,TS val 443680145 ecr 0,nop,wscale 7], length 0

If I comment out the bind, then everything works as expected. There is no firewall port that seems relevant, and rp_filter is set to 0 on loopback.

Why is the connection not accepted? Is this an error in my code, or is this configuration related?

score 2 · Accepted Answer · answered Oct 14 '19 at 18:53

The missing SYN+ACK should have had the following information:

127.0.0.1:7007 -> 93.184.216.34:31337

While traversing the routing table, the destination IP would match the default route, and would be scheduled to go out on eth0 (as expected).

However, eth0 has (as do all other interfaces in the system) Martian packet protection enabled. Since the source address is 127.0.0.1, the packet never goes out, and is never reported by tcpdump.

One solution is to turn off Martian packet protection for eth0:

# echo 0 > /proc/sys/net/ipv4/conf/eth0/route_localnet

This, however, disables a protection that is quite useful. A much better solution is to direct this traffic to the loopback interface to begin with using a source route:

# ip rule add from 127.0.0.1 lookup 100
# ip route add local default dev lo table 100

With this in place, actual connection is established between the transparent proxy and the local service, using the outside IP address and port.

score 1 · Answer 2 · answered Oct 13 '19 at 19:49

1

You are trying to create a TCP connection from some external address (93.184.216.34) to 127.0.0.1. The SYN+ACK therefore must be created with source 127.0.0.1 and the external address as destination. Only, there is no route using the loopback interface (127.0.0.1) to this external address and thus the SYN+ACK cannot be sent.

answered Oct 13 '19 at 19:49

Steffen Ullrich

114,247
10
131
172

Why would the source address matter for that case? There is no source routing defined on the machine? – Shachar Shemesh Oct 13 '19 at 20:01
@ShacharShemesh: the source IP of SYN+ACK must match the destination of the SYN. The source IP 127.0.0.1 is bound to loopback so the SYN+ACK must leave from the loopback interface. But there is no route to the external address which involves the loopback interface. – Steffen Ullrich Oct 13 '19 at 20:03
Changing the address to something other than 127.0.0.1 indeed causes the SYN+ACK to issue, so you are, at the very least, partially correct. What I do not understand is why. I was taught (many many years ago) that the source address does not affect the routing decision. What command would add the missing route to loopback? – Shachar Shemesh Oct 13 '19 at 20:41
@ShacharShemesh: I'm not sure what you are trying to achieve in the first place with your setup. – Steffen Ullrich Oct 13 '19 at 21:20
I have actually found a solution to my specific problem. What I don't understand is why that solution is necessary. As such, give me a command line *without ip rules* that would add _any_ route to packets from 127.0.0.1 (lo, eth0, whatever). When I go over the `ip rule` man page, it clearly says "Classic routing algorithms used in the Internet make routing decisions based only on the destination address of packets". Seems like here this is not true despite not having any ip rules. I don't understand why. – Shachar Shemesh Oct 14 '19 at 04:38
@ShacharShemesh: The rule to the target would probably go through eth0. Only, it cannot use this route since 127.0.0.1 is not bound to eth0 but lo. – Steffen Ullrich Oct 14 '19 at 06:41
No other IP is bound to a specific interface. Even when you use SO_BIND on a socket, it only affects incoming traffic. I have actually found why that is, I'll write a proper answer when I get to it. – Shachar Shemesh Oct 14 '19 at 10:17
Please see my answer. As you can see, binding has nothing to do with this, nor does source address routing decision. It's all about a protection mechanism specific to this particular IP address. – Shachar Shemesh Oct 14 '19 at 18:55

IP_TRANSPARENT SYN not getting SYN+ACK response on localhost

2 Answers2