kubectl logs displays only 'API server listening at: [::]:40000' when remote debugging with dlv is enabled - How do I get my logs back?

Question

I currently have a go app that uses a lot fmt.printf. Whenever that app would run in a pod I was able to get the logs back by doing

kubectl logs podname

However I also needed to integrate remote debugging. I need to use dlv to allow my ide(GoLand) to remotely connect to the pod. It connects to the Pod at port 40000. Also when the Pods image runs it exposes port 40000 i.e the docker file has this in it 40000

I also have a service that looks like this in my minikube

apiVersion: v1
kind: Service
metadata:
  name: mydebug
spec:
  type: ClusterIP
  selector:
    app: fooapp
  ports:
  - protocol: TCP
    port: 40000
    targetPort: 40000
    name: delve

Now when I do kubectl logs podname I only get this back

API server listening at: [::]:40000
2022-10-30T21:18:57Z warning layer=rpc Listening for remote connections (connections are not authenticated nor encrypted)

Is there anyway to get my logs back ? Ho

Just to make this more clear; you are wanting to connect to your pod from OUTSIDE the cluster correct? — Mike, Nov 02 '22 at 10:15
@MikeKilic yes I am able to do that but when I do that I do not see any of my fmt.println produced from my app when I do a `kubectl logs` — Rajeshwar, Nov 02 '22 at 17:48
Could you share your dockerfile or dlv command? Did you try appending `--log` option to your `dlv` command? — YwH, Nov 03 '22 at 02:57
@Rajeshwar: Is it necessary that your go app prints to STDOUT? I suggest your app should write to a log file instead using Go's `log` module. Then, if you properly mount/map the file system between your pod and the host you can just do a `tail -f` on that filename. — Hechi, Nov 04 '22 at 14:02

chresse · Accepted Answer · 2022-11-09T08:03:59.017

You can use the --continue exec flag, to continue the debugged process on the start, which then will lead to continued logs.

So start delve e.g. with:

dlv --listen=:2345 --headless exec your/app --continue

Without the --continue flag, delve will wait for remote connections and halt your application. With the --continue flag, the application instead will start already.

From dlv help exec:

...
Usage:
  dlv exec <path/to/binary> [flags]

Flags:
      --continue     Continue the debugged process on start.
...

kubectl logs displays only 'API server listening at: [::]:40000' when remote debugging with dlv is enabled - How do I get my logs back?

1 Answers1