UDP response issues

Question

I'm having an issue with port forwarding UDP traffic in kubernetes. I am running a coreos baremetal setup and in the past just used fleet to run my containers, so these containers to work and my network and port forwarding is setup correctly. I can manually run the container and port forward to it and things work as expected, so it seems something is going on with kubernetes and UDP. I have multiple services that are not working properly, but the easiest one is this mumble server. Here is the setup for it starting with replication controller.

apiVersion: v1
kind: ReplicationController
metadata:
  name: mumble-v0
  labels:
    app: mumble
    version: v0
spec:
  replicas: 1
  selector:
    app: mumble
    version: v0
  template:
    metadata:
      labels:
        app: mumble
        version: v0
    spec:
      containers:
      - name: mumble
        image: coppit/mumble-server
        imagePullPolicy: Always
        resources:
          limits:
            cpu: 0.5
            memory: 500Mi
        ports:
        - containerPort: 64738
          name: mumble
        - containerPort: 64738
          name: mudp
          protocol: UDP

And the service itself:

apiVersion: v1
kind: Service
metadata:
  name: mumble
  labels:
    app: mumble
    kubernetes.io/name: "mumble"
spec:
  selector:
    app: mumble
  ports:
  - name: mumble
    port: 64738
  - name: mumble-udp
    port: 64738
    protocol: UDP
  externalIPs: ["10.0.1.19"]

Clients are able to connect to the service and see the server, but no voice traffic flows properly. This voice traffic is sent and received via UDP. Looking around, I see other reports of UDP issues that seem to be related Responses from kubernetes containers getting lost and Problems on running a SIP application (UDP) on Kubernetes.

Does anyone know what could be going wrong here or the fix?

what does tcpdump show? First thing to confirm is whether the traffic makes it back to the machine or not. — Tim Hockin, Jan 25 '16 at 01:24
Also, what exact kubernetes version? Also, is this with the userspace proxy (default) or the iptables one (needs to be enabled)? — Tim Hockin, Jan 25 '16 at 01:25
I'm running CoreOS beta (899.5.0), which has kubelet version Kubernetes v1.1.2+3085895. I'm pulling hyperkube:v1.1.2 to run the proxy. I've followed the CoreOS setup instructions and I assume with the proxy container I'm running proxy, but I do see iptables rules for forwarding. http://stackoverflow.com/questions/34368093/responses-from-kubernetes-containers-getting-lost has a tcpdump, but I'll try to snag one as well. — Mike Wilson, Jan 25 '16 at 07:40
@TimHockin Sorry it has taken me so long to get back to you. I've been digging through tcpdump and investigating and have found that the problem with mumble seems to be if it happens to land on my master. I have my master setup to run jobs as well, but I'm thinking that is either a bad idea or I just did it incorrectly. I still can't get the homeworld game server to work properly if launched in kubernetes, but I see traffic entering and leaving on both the node and inside the container. I'm confused as to the problem there, but it doesn't seem to be packets not making it out. — Mike Wilson, Jan 26 '16 at 17:08

Wernight · Answer 1 · 2016-08-17T11:27:49.770

If you kubectl get svc you should see that it either opened the UDP or the TCP port. Kubernetes supporting both UDP and TCP on the same port seems to be an ongoing issue.

If you changed port number, you'd get an error "cannot create an external load balancer with mix protocols" (see related issue).

What you can do is create two services with the same static IP (should be reserved):

apiVersion: v1
kind: Service
metadata:
  name: mumble-tcp
  labels:
    project: mumble
spec:
  type: LoadBalancer
  loadBalancerIP: 10.10.10.10
  ports:
    - port: 64738
  selector:
    name: mumble
    project: mumble
---
apiVersion: v1
kind: Service
metadata:
  name: mumble-udp
  labels:
    project: mumble
spec:
  type: LoadBalancer
  loadBalancerIP: 10.10.10.10
  ports:
    - port: 64738
      protocol: UDP
  selector:
    name: mumble
    project: mumble

UDP response issues

1 Answers1