2

Idea is to use argdist to measure latency duration of napi_poll() which returns number of packet processed (called work). Ratio of execution latency of napi_poll() to number of packets processed would give me average amount of time it took to process each packet in form of histogram.

I am using following command

argdist -H 'r:c:napi_poll():u64:$latency/$retval#avg time per packet (ns)' which end up giving me error Failed to attach BPF to kprobe and in dmesg I get message like Could not insert probe at napi_poll+0: -2

I am just curios why I can not attach kretprobes to napi_poll() when similar trick works with net_rx_action() ?

Qeole
  • 8,284
  • 1
  • 24
  • 52
Valmik Roy
  • 57
  • 1
  • 7
  • Could you try with a simpler tool? For example, `funccount napi_poll`. On my system, `napi_poll` doesn't work but `net_rx_action` does. Probably just because one is inlined and not the other. – pchaigno Dec 24 '17 at 20:39
  • I think you are right about inlined-ness of function as I tried every other way and napi_poll never worked. – Valmik Roy Dec 26 '17 at 21:12
  • I'll write a full answer with suggestions, but unfortunately, there isn't much you can do if you can't intercept the function. – pchaigno Dec 26 '17 at 21:17

1 Answers1

5

Most of the time the Failed to attach BPF to kprobe error is caused by an inlined function. As explained in the Kprobes documentation (section Kprobes Features and Limitations), Kprobes will fail to attach if the target function was inlined. Since napi_poll is static, it might have been inlined at compile time.

You can check in kernel symbols if napi_poll was inlined or not:

$ cat /boot/System.map-`uname -r` | grep " napi_poll"
$
$ cat /boot/System.map-`uname -r` | grep " net_rx_action"
ffffffff817d8110 t net_rx_action

On my system, napi_poll is inlined while net_rx_action is not.

There are several workarounds for this problem, depending on your goal.

  1. If you don't mind recompiling your kernel, you can use the Linux inline attribute to ensure napi_poll is not inlined.
  2. If you can't change your kernel, the usual workaround is to find a calling function of napi_poll that provides the same information. A function called by napi_poll can also work if it provides enough information and is not inlined itself.
pchaigno
  • 11,313
  • 2
  • 29
  • 54