(2015/01/10 18:51), Namhyung Kim wrote: > Hi Jiri, > > On Fri, Jan 09, 2015 at 04:44:21PM +0100, Jiri Olsa wrote: >> On Fri, Jan 09, 2015 at 04:30:56PM +0100, Jiri Olsa wrote: >>> On Sat, Jan 10, 2015 at 12:21:13AM +0900, Namhyung Kim wrote: >>>> On Fri, Jan 09, 2015 at 03:55:39PM +0100, Jiri Olsa wrote: >>>>> hi, >>>>> I couldn't use following perf command to insert return probe: >>>>> >>>>> # perf probe -a fork_exit=do_fork%return >>>>> Added new event: >>>>> Failed to write event: Invalid argument >>>>> Error: Failed to add events. >>>>> >>>>> >>>>> I'm pretty sure I used this command before, so seems like >>>>> it's broken. I can still use debugfs tracing interface to >>>>> do that: >>>>> # echo 'r:do_fork_entry do_fork' > kprobe_events >>>>> >>>>> I used Arnaldo's latest perf/core and FC20 kernel: >>>>> >>>>> # uname -a >>>>> Linux krava 3.17.7-200.fc20.x86_64 #1 SMP Wed Dec 17 03:35:33 UTC 2014 >>>>> x86_64 x86_64 x86_64 GNU/Linux >>>>> # ./perf version >>>>> perf version 3.18.g6a7d78 >>>>> >>>> >>>> Is it just return probe? Did it work for normal kprobes? >>> >>> yep, works for normal probes >>> >>>> Maybe it's related to the below: >>>> >>>> https://lkml.org/lkml/2014/12/31/15 >>>> >>>> Have you check the acme/perf/urgent too? >>> >>> hum.. can't access lkml, I'll check, also with perf/urgent >> >> neither helped.. > > I think I've found the reason. > > The commit dfef99cd0b2c ("perf probe: Use ref_reloc_sym based address > instead of the symbol name") converts kprobes to use ref_reloc_sym > (i.e. _stext) and offset instead of using symbol's name directly. So > on my system, adding do_fork ends up with like below: > > $ sudo perf probe -v --add do_fork%return > probe-definition(0): do_fork%return > symbol:do_fork file:(null) line:0 offset:0 return:1 lazy:(null) > 0 arguments > Looking at the vmlinux_path (7 entries long) > Using /lib/modules/3.17.6-1-ARCH/build/vmlinux for symbols > Could not open debuginfo. Try to use symbols. > Opening /sys/kernel/debug/tracing/kprobe_events write=1 > Added new event: > Writing event: r:probe/do_fork _stext+456136 > Failed to write event: Invalid argument > Error: Failed to add events. Reason: Operation not permitted (Code: -1) > > > As you can see, the do_fork was translated to _stext+456136. This was > because to support (local) symbols that have same name. But the > problem is that kretprobe requires to be inserted at function start > point so it simply checks whether it's called with offset 0. And if > not, it'll return with -EINVAL. You can see it with dmesg. > > $ dmesg | tail -1 > [125621.764103] Return probe must be used without offset. > > So we need to use the symbol name instead of ref_reloc_sym in case of > return probes. During the tracking down, I found a couple of problems > in the code. I'll send fixes soon.
Oops, thank you for analyzing the problem:) I've completely forgot about return probes with KASLR. This also should be fixed in kprobe-tracer, which should convert given _stext+offset to actual symbol in kernel. Thanks, > > Thanks, > Namhyung > -- Masami HIRAMATSU Software Platform Research Dept. Linux Technology Research Center Hitachi, Ltd., Yokohama Research Laboratory E-mail: [email protected] -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
