Questions tagged [ftrace]

Ftrace is an internal tracing utility built directly into the Linux kernel. It can be used for debugging or analyzing latencies and performance issues that take place outside of user-space.

Ftrace is an internal tracer designed to help out developers and designers of systems to find what is going on inside the Linux kernel. Ftrace was derived from two preceding tools: the "latency tracer" by Ingo Molnar and the "logdev" utility by Steven Rostedt. It can be used for debugging or analyzing latencies and performance issues that take place outside of user-space.

Ftrace also includes a plugin infrastructure that allows for other types of tracing. Tracers available in ftrace through plugins allow tracing such events as:

  • context switches
  • the time it takes for a high priority task to run after it was woken up
  • the amount of time for which interrupts are disabled

123 questions
2
votes
1 answer

How can I use ftrace to get the in-kernel call graph called by a system call?

I've tried these, but all of them produce "empty" output files: trace-cmd record -p function_graph -g munmap -F ls trace-cmd record -p function_graph -g sys_enter_munmap -F ls trace-cmd record -p function_graph -g sys_enter -F ls
Robin Green
  • 32,079
  • 16
  • 104
  • 187
2
votes
1 answer

Change format of syscall event trace output to ftrace

I enabled ftrace event tracing for sys_enter_openat syscall. The respective output format given at events/syscalls/sys_enter_openat/format is print fmt: "dfd: 0x%08lx, filename: 0x%08lx, flags: 0x%08lx, mode: 0x%08lx", ((unsigned long)(REC->dfd)),…
Madhur Rawat
  • 468
  • 1
  • 7
  • 18
2
votes
1 answer

kprobe_events fetch arguments doesn't work on ARM

The following kprobe_based event example works well: $ echo 'p:myprobe do_sys_open' > /sys/kernel/debug/tracing/kprobe_events But, adding fetching arguments doesn't work: $ echo 'p:myprobe do_sys_open dfd=%ax filename=%dx flags=%cx mode=+4($stack)'…
bestn
  • 35
  • 8
2
votes
1 answer

Is there a way to output function durations with trace-cmd that is exclusive of nested function durations?

I am tracing a load generator with trace-cmd. Here is a snippet of output from my trace-cmd report: loadgen-2682 [003] 105.979473: funcgraph_entry: | select_task_rq_fair() { loadgen-2682 [003] …
buratino
  • 1,408
  • 2
  • 17
  • 40
2
votes
2 answers

Using FTRACE for loadable linux driver module

I am trying to use FTRACE for tracing and getting function_graph on a loadable driver module. But, somehow, I did not see any functions in that module being traced. Here is what I did (I had the kernel configured to have FTRACE already in the…
electro
  • 911
  • 4
  • 12
  • 28
2
votes
0 answers

difference between ftrace function statistics and oprofile

May anyone please compare ftrace statistics and oprofile? We observed the following phenomenons: with oprofile enabled during the network performance test, we see little impact to the performance test (compare to the case where oprofile is not…
Dien Nguyen
  • 2,019
  • 6
  • 30
  • 37
1
vote
0 answers

Using ftrace on Samsung smartphone

I'm trying to read saved_tgids using adb: cat sys/kernel/debug/tracing/saved_tgids in samsung A5 with Android 8,but i get "permission denied". Instead if i try to execute the command using adb: cat sys/kernel/debug/tracing/saved_tgids in Pixel 3a XL…
Luca
  • 19
  • 4
1
vote
0 answers

How to use ftrace to print partial function trace?

TLDR; Does ftrace have a function similiar to trace_printk() which can print out the current function's call stack? I know how to print out the function or function graph trace using ftrace, but sometimes I want to check one place's function call…
tristone
  • 95
  • 6
1
vote
3 answers

How can I parse the output of trace-cmd report using awk?

The only thing I need to see in the output is which functions were called. Input: ibv_rc_pingpong-759367 [005] 8391981.416466: funcgraph_entry: | ib_enum_all_devs() { ibv_rc_pingpong-759367 [005] 8391981.416472: funcgraph_entry: …
Bahamas
  • 345
  • 1
  • 2
  • 13
1
vote
0 answers

trace-cmd by process ID seems not working

I am working on a SUSE linux work node and want to find out some performance issue on it. The linux kernel looks as below: 5.3.18-24.99-default #1 SMP Sun Jan 23 19:03:51 UTC 2022 (712a8e6) x86_64 x86_64 x86_64 GNU/Linux After installing trace-cmd,…
Adam Z
  • 43
  • 3
1
vote
0 answers

How to trace L2 packet in linux kernel bridge and interface?

I create a Linux bridge and add two virtual interface to this bridge. ip link add br0 address 01:02:03:04:00:00 type bridge ip link add veth0 address 01:02:03:04:00:10 type dummy ip link add veth1 address 01:02:03:04:00:20 type dummy ip link set…
1
vote
0 answers

Why `events/syscalls/sys_enter` does not support string format as output?

I have a question about events/syscalls/sys_enter* trace point. Why does not events/syscalls/sys_enter* support string format? For example, in case of sys_enter_openat outputs the filename as hex, not string. $ cd /sys/kernel/debug/tracing $ cat…
m-bat
  • 579
  • 4
  • 9
1
vote
1 answer

Ftrace. is task same as thread id?

I'm using ftrace. There is task/pid field. What is the task? Is it process's thread id? If no, is it possible to get information about threads with same pid with ftrace? I used only function and function_graph.
Excentricitet
  • 143
  • 1
  • 1
  • 6
1
vote
0 answers

Ftrace overhead comparison : trace_printk v/s custom tracepoint in linux

What's the overhead comparison of trace_printk message and adding a custom Ftrace tracepoint in the Linux kernel? Since both trace_printk and custom tracepoint logs in trace log file, I'm guessing there overheads would be same while performance…
shivank
  • 67
  • 1
  • 10
1
vote
1 answer

No such device when I use adb to show /sys/kernel/debug/tracing/available_filter_functions

Guys. I am trying to study virtual memory management on Nexus 6P(An Old Android Smartphone) Because the default kernel in AOSP didn't turn CONFIG_FUNCITON_TRACER on, and I can't use function tracer to study memory management related functions. So I…
Kevin Ding
  • 568
  • 5
  • 9
1 2
3
8 9