0

While trying to debug a panic in kubelet, I have been trying unsuccessfully to attach delve to kubelet executable:

me@mycomputer:~$ sudo dlv attach 23796
could not attach to pid 23796: could not open debug info

file and objdump --syms reveal no debug info is included in kubelet:

me@mycomputer:~$ file _output/bin/kubelet 
_output/bin/kubelet: ELF 64-bit LSB executable, x86-64, version 1 
(SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, 
BuildID[sha1]=c7a1645940f91048eea9086bd0500deb7ec75b38, for 
GNU/Linux 3.2.0, stripped

me@mycomputer:~$ objdump --syms _output/bin/kubelet 

_output/bin/kubelet:     file format elf64-x86-64
SYMBOL TABLE:
no symbols

I have tried different options while building kubelet:

make WHAT=cmd/kubelet GOGCFLAGS=-dwarf

or:

make WHAT=cmd/kubelet GOFLAGS=-dwarf

but still no debug info is included in the executable. It seems to me that -w option is included by default in go tool link, which strips the DWARF symbol table to save on final executable size (see here). But I was not able to reverse that.

Any thoughts on how I should build kubelet to include debug info?

Dani
  • 19
  • 3
  • These are just go packages, what happens when you call `go build` manually? I cloned the repo and it works fine here. – JimB Apr 29 '19 at 21:29
  • @JimB you mean calling `go build kubelet`? I tried that both at `kubernetes' root directory and at `pkg\kubelet`; both issued `go: error loading module requirements` error. – Dani Apr 30 '19 at 01:37
  • @JimB `make all WHAT=cmd/kubelet` is what `Makefile` recommends. but my main question is how to build `kubelet` so debug info is included, in order to use `delve`. BTW, yesterday [issue #77174](https://github.com/kubernetes/kubernetes/issues/77174) was opened about `kubelet` panic. – Dani Apr 30 '19 at 01:44
  • Their Makefile says that `GOGCFLAGS="-N -l"` should suffice. Make sure you're executing the binary you just built, or build it manually from the correct directory (`go build kubelet` isn't a valid command, and the kubelet main package is `github.com/kubernetes/kubernetes/cmd/kubelet`) – JimB Apr 30 '19 at 15:18

1 Answers1

1

As JimB advised, go build does the trick:

me@mycomputer:~/kubernetes/cmd/kubelet$ sudo go build kubelet.go
me@mycomputer:~/kubernetes/cmd/kubelet$ file ./kubelet
kubelet: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically 
linked, interpreter /lib64/ld-linux-x86-64.so.2, 
BuildID[sha1]=a1377345bfab4ca57b28af60eed92f23c81d7ae8, for GNU/Linux 3.2.0, 
not stripped

As can be seen, the debug info is not stripped anymore. Now you can copy the output to _output/bin and debug it using delve.

Dani
  • 19
  • 3