5

In an attempt to get some insights into the Kubernetes networking model, I started following this doc. I am trying to intercept the packets to a SERVICE_IP from the userspace by writing this code(from the blog shared above).

func main() {
    clusterIP := "10.7.111.132"
    podIP := "10.5.41.204"
    port := 80
    proto := "tcp"

    addRedirectRules(clusterIP, port, proto)
    createProxy(podIP, port, proto)
}

func addRedirectRules(clusterIP string, port int, proto string) error {
    p := strconv.Itoa(port)
    cmd := exec.Command("iptables", "-t", "nat", "-A", "OUTPUT", "-p", "tcp",
        "-d", clusterIP, "--dport", p, "-j", "REDIRECT", "--to-port", p)
    return cmd.Run()
}

func createProxy(podIP string, port int, proto string) {
    host := ""
    listener, err := net.Listen(proto, net.JoinHostPort(host, strconv.Itoa(port)))

    for {
        inConn, err := listener.Accept()
        outConn, err := net.Dial(proto, net.JoinHostPort(podIP, strconv.Itoa(port)))

        go func(in, out *net.TCPConn) {
            var wg sync.WaitGroup
            wg.Add(2)
            fmt.Printf("Proxying %v <-> %v <-> %v <-> %v\n",
                in.RemoteAddr(), in.LocalAddr(), out.LocalAddr(), out.RemoteAddr())
            go copyBytes(in, out, &wg)
            go copyBytes(out, in, &wg)
            wg.Wait()
        }(inConn.(*net.TCPConn), outConn.(*net.TCPConn))
    }

    listener.Close()
}

func copyBytes(dst, src *net.TCPConn, wg *sync.WaitGroup) {
    defer wg.Done()
    if _, err := io.Copy(dst, src); err != nil {
        if !strings.HasSuffix(err.Error(), "use of closed network connection") {
            fmt.Printf("io.Copy error: %v", err)
        }
    }
    dst.Close()
    src.Close()
}

After running this command I see the output chain being configured like this:

Chain OUTPUT (policy ACCEPT)
num  target     prot opt source               destination
1    KUBE-SERVICES  all  --  0.0.0.0/0            0.0.0.0/0            /* kubernetes service portals */
2    DOCKER     all  --  0.0.0.0/0           !127.0.0.0/8          ADDRTYPE match dst-type LOCAL
3    REDIRECT   tcp  --  0.0.0.0/0            100.65.248.49        tcp dpt:80 redir ports 80

But I don't see the tcp server I created intercept the traffic. Being a networking noob, I am not quite able to reason about it. Any help would be appreciated

Always_Beginner
  • 2,546
  • 6
  • 25
  • 33
  • the code you posted does not check for errors (error returned by `addRedirectRules()`, errors returned by various calls in `createProxy()`). Did you just remove that code before posting on SO, or does your original code silently ignore errors ? – LeGEC Oct 26 '20 at 09:03
  • I see it is a copy/paste from the code in the linked article, so I guess your code ignores the errors as well : at the very least, print errors on the console if they occur – LeGEC Oct 26 '20 at 09:06
  • 1
    Hi @LeGEC, unlike the example. I did try debugging with logging the errors. There weren't any – Always_Beginner Oct 26 '20 at 11:34
  • ok. If you log `fmt.Println("accepted conn from", inConn.RemoteAddr())` right after `listener.Accept()`, do you see these lines displayed ? – LeGEC Oct 26 '20 at 14:48
  • @Always_Beginner, 1. what is the `SERVER_IP` in your code (a `clusterIP`)? 2. `OUTPUT` chain in `nat` table will process only packets generated by **local** process (your code have to run on a server which generate packets). 3. `REDIRECT` rule will never be hit as `KUBE-SERVICES` rule masks it. Use `iptables -t nat -I OUTPUT 1 ...` in your code to fix it. 4. `createProxy` is your `tcp server`. This code (i assume, as have no knowledge in `Go` yet :-)) should run a proxy the communication between [process]<->[your_code]<->[`SERVICE_IP`]. – Tom Lime Oct 27 '20 at 08:48

0 Answers0