bpf/bcc reports error when trying to access `struct rq`

Question

This is my bpf program to profile a kernel function pick_next_task_fiar.

#include <uapi/linux/ptrace.h>
#include <linux/sched.h>
#include <linux/nsproxy.h>
#include <linux/pid_namespace.h>


struct rq; // forward declaration

struct val_t {
   pid_t pid;
   u64 vruntime;
   int type;       // Note, 0 for previous task, 1 for next task.
};

BPF_PERF_OUTPUT(events);

int kprobe_pick_next_fair(struct pt_regs *ctx, struct rq *rq, 
struct task_struct *prev)
{


    int cpu = rq->cpu;
    struct val_t data = {};
    data.pid = prev->pid;
    data.vruntime = prev->se.vruntime;
    data.type = 0;
    events.perf_submit(ctx, &data, sizeof(data));    

    return 0;
};

It reports error as follow:

    int cpu = rq->cpu;
              ~~^
/virtual/main.c:8:8: note: forward declaration of 'struct rq'
struct rq; // forward declaration
       ^
1 error generated.
Traceback (most recent call last):
  File "picknextfair__back.py", line 73, in <module>
    b = BPF(text=bpf_text)
  File "/usr/lib/python2.7/dist-packages/bcc/__init__.py", line 297, in __init__
    raise Exception("Failed to compile BPF text:\n%s" % text)
Exception: Failed to compile BPF text:

My question why the bpf can not recognise the struct rq since I have already included the # include <linux/sched.h>. However, it does recognise the struct task_struct. These two structs are in the same head file.

Kernel version: 4.4.0-141-generic on ubuntu 16.04

pchaigno · Accepted Answer · 2019-02-08T17:17:26.930

1

struct rq is actually not part of the kernel headers, as you can see on Bootlin.

You can either:

retrieve the offset to rq->cpu from the rq pointer and hardcode it in your BPF program, but I wouldn't recommend it;
or find some other way to retrieve the cpu number, maybe through prev or the current task (e.g., prev->cpu).

edited Feb 08 '19 at 17:17

answered Feb 08 '19 at 06:57

pchaigno

11,313
2
29
54

hi, Can you elaborate a bit more? I see both `task_stuct` and `rq` are defined in the same file (`include/linux/sched.h`). I can definitely retrieve the cpu number by `prev`. – Chen Wei Feb 08 '19 at 16:36
Sorry, the link in my answer was incorrect; I fixed it. `rq` is defined in the kernel sources ([`kernel/sched/sched.h`](https://elixir.bootlin.com/linux/v4.20.7/source/kernel/sched/sched.h#L782)), but not in the kernel headers ([`include/linux/sched.h`](https://elixir.bootlin.com/linux/v4.20.7/source/include/linux/sched.h#L590)). Those are two different files. – pchaigno Feb 08 '19 at 17:43
Thanks, I got it! It is really helpful – Chen Wei Feb 08 '19 at 21:04
I futher notcice it works fine if it is `int cpu_id = task_cpu(prev)`. However it reports the following error when adding : `if (cpu_id != 0) return;` ```bpf: Failed to load program: Permission denied 0: (bf) r6 = r1 1: (79) r7 = *(u64 *)(r6 +104) 2: (79) r1 = *(u64 *)(r7 +8) R7 invalid mem access 'inv' HINT: The invalid mem access 'inv' error can happen if you try to dereference memory without first using bpf_probe_read() to copy it to the BPF stack. Sometimes the bpf_probe_read is automatic by the bcc rewriter, other times you'll need to be explicit.``` – Chen Wei Feb 08 '19 at 21:18
I have tried the prev->cpu. Looks like it does not work. – Chen Wei Feb 08 '19 at 21:42
I think it is due to my kernel version is 4.4.0. I will upgrade my kernel. Thanks! – Chen Wei Feb 08 '19 at 21:56
Do you want me to change something to accept my answer? What's the error message with prev->cpu? – pchaigno Feb 09 '19 at 07:26

bpf/bcc reports error when trying to access `struct rq`

1 Answers1