Decode the caller address instead of the return address by default. This
also introduced -R option to provide return address decoding mode.
This changes the decode_stacktrace.sh to decode the line info 1byte before
the return address which will be the call(branch) instruction address. If
the return address is a symbol address (zero offset from it), it falls
back to decoding the return address.
This improves results especially when optimizations have changed the order
of the lines around the return address, or when the return address does
not have the actual line information.
With this change;
Call Trace:
<TASK>
dump_stack_lvl (lib/dump_stack.c:94 lib/dump_stack.c:120)
lockdep_rcu_suspicious (kernel/locking/lockdep.c:6876)
event_filter_pid_sched_process_fork (kernel/trace/trace_events.c:1057)
kernel_clone (include/trace/events/sched.h:396 include/trace/events/sched.h:396 kernel/fork.c:2664)
__x64_sys_clone (kernel/fork.c:2795 kernel/fork.c:2779 kernel/fork.c:2779)
do_syscall_64 (arch/x86/entry/syscall_64.c:63 arch/x86/entry/syscall_64.c:94)
? entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:121)
? trace_irq_disable (include/trace/events/preemptirq.h:36)
entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:121)
Without this (or give -R option);
Call Trace:
<TASK>
dump_stack_lvl (lib/dump_stack.c:122)
lockdep_rcu_suspicious (kernel/locking/lockdep.c:6877)
event_filter_pid_sched_process_fork (kernel/trace/trace_events.c:?)
kernel_clone (include/trace/events/sched.h:? include/trace/events/sched.h:396 kernel/fork.c:2664)
__x64_sys_clone (kernel/fork.c:2779)
do_syscall_64 (arch/x86/entry/syscall_64.c:?)
? entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:130)
? trace_irq_disable (include/trace/events/preemptirq.h:36)
entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:130)
[akpm@linux-foundation.org: fix spello]
Link: https://lkml.kernel.org/r/177275821652.1557019.18367881408364381866.stgit@mhiramat.tok.corp.google.com
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Reviewed-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>
Tested-by: Luca Ceresoli <luca.ceresoli@bootlin.com> [arm64]
Cc: Carlos Llamas <cmllamas@google.com>
Cc: Sasha Levin (Microsoft) <sashal@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Support for parsing PC source info in stacktraces (e.g. '(P)') was added
in commit 2bff77c665 ("scripts/decode_stacktrace.sh: fix decoding of
lines with an additional info"). However, this logic was placed after the
build ID processing. This incorrect order fails to parse lines containing
both elements, e.g.:
drm_gem_mmap_obj+0x114/0x200 [drm 03d0564e0529947d67bb2008c3548be77279fd27] (P)
This patch fixes the problem by extracting the PC source info first and
then processing the module build ID. With this change, the line above is
now properly parsed as such:
drm_gem_mmap_obj (./include/linux/mmap_lock.h:212 ./include/linux/mm.h:811 drivers/gpu/drm/drm_gem.c:1177) drm (P)
While here, also add a brief explanation the build ID section.
Link: https://lkml.kernel.org/r/20251030010347.2731925-1-cmllamas@google.com
Fixes: 2bff77c665 ("scripts/decode_stacktrace.sh: fix decoding of lines with an additional info")
Signed-off-by: Carlos Llamas <cmllamas@google.com>
Reviewed-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>
Reviewed-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Cc: Breno Leitao <leitao@debian.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Marc Rutland <mark.rutland@arm.com>
Cc: Mark Brown <broonie@kernel.org>
Cc: Matthieu Baerts <matttbe@kernel.org>
Cc: Miroslav Benes <mbenes@suse.cz>
Cc: Puranjay Mohan <puranjay@kernel.org>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
With lines having a code to decode, the alignment was not preserved for
the first line.
With this sample ...
[ 52.238089][ T55] RIP: 0010:__ip_queue_xmit+0x127c/0x1820
[ 52.238401][ T55] Code: c1 83 e0 07 48 c1 e9 03 83 c0 03 (...)
... the script was producing the following output:
[ 52.238089][ T55] RIP: 0010:__ip_queue_xmit (...)
[ 52.238401][ T55] Code: c1 83 e0 07 48 c1 e9 03 83 c0 03 (...)
That's because scripts/decodecode doesn't preserve the alignment. No need
to modify it, it is enough to give only the "Code: (...)" part to this
script, and print the prefix without modifications.
With the same sample, we now have:
[ 52.238089][ T55] RIP: 0010:__ip_queue_xmit (...)
[ 52.238401][ T55] Code: c1 83 e0 07 48 c1 e9 03 83 c0 03 (...)
Link: https://lkml.kernel.org/r/20250908-decode_strace_indent-v1-3-28e5e4758080@kernel.org
Signed-off-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>
Tested-by: Carlos Llamas <cmllamas@google.com>
Cc: Breno Leitao <leitao@debian.org>
Cc: Elliot Berman <quic_eberman@quicinc.com>
Cc: Luca Ceresoli <luca.ceresoli@bootlin.com>
Cc: Stephen Boyd <swboyd@chromium.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
With lines having a symbol to decode, the script was only trying to
preserve the alignment for the timestamps, but not the rest, nor when the
caller was set (CONFIG_PRINTK_CALLER=y).
With this sample ...
[ 52.080924] Call Trace:
[ 52.080926] <TASK>
[ 52.080931] dump_stack_lvl+0x6f/0xb0
... the script was producing the following output:
[ 52.080924] Call Trace:
[ 52.080926] <TASK>
[ 52.080931] dump_stack_lvl (arch/x86/include/asm/irqflags.h:19)
(dump_stack_lvl is no longer aligned with <TASK>: one missing space)
With this other sample ...
[ 52.080924][ T48] Call Trace:
[ 52.080926][ T48] <TASK>
[ 52.080931][ T48] dump_stack_lvl+0x6f/0xb0
... the script was producing the following output:
[ 52.080924][ T48] Call Trace:
[ 52.080926][ T48] <TASK>
[ 52.080931][ T48] dump_stack_lvl (arch/x86/include/asm/irqflags.h:19)
(the misalignment is clearer here)
That's because the script had a workaround for CONFIG_PRINTK_TIME=y only,
see the previous comment called "Format timestamps with tabs".
To always preserve spaces, they need to be recorded along the words. That
is what is now done with the new 'spaces' array.
Some notes:
- 'extglob' is needed only for this operation, and that's why it is set
in a dedicated subshell.
- 'read' is used with '-r' not to treat a <backslash> character in any
special way, e.g. when followed by a space.
- When a word is removed from the 'words' array, the corresponding space
needs to be removed from the 'spaces' array as well.
With the last sample, we now have:
[ 52.080924][ T48] Call Trace:
[ 52.080926][ T48] <TASK>
[ 52.080931][ T48] dump_stack_lvl (arch/x86/include/asm/irqflags.h:19)
(the alignment is preserved)
Link: https://lkml.kernel.org/r/20250908-decode_strace_indent-v1-2-28e5e4758080@kernel.org
Signed-off-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>
Tested-by: Carlos Llamas <cmllamas@google.com>
Cc: Breno Leitao <leitao@debian.org>
Cc: Elliot Berman <quic_eberman@quicinc.com>
Cc: Luca Ceresoli <luca.ceresoli@bootlin.com>
Cc: Stephen Boyd <swboyd@chromium.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Since commit bdf8eafbf7 ("arm64: stacktrace: report source of unwind
data") a stack trace line can contain an additional info field that was not
present before, in the form of one or more letters in parentheses. E.g.:
[ 504.517915] led_sysfs_enable+0x54/0x80 (P)
^^^
When this is present, decode_stacktrace decodes the line incorrectly:
[ 504.517915] led_sysfs_enable+0x54/0x80 P
Extend parsing to decode it correctly:
[ 504.517915] led_sysfs_enable (drivers/leds/led-core.c:455 (discriminator 7)) (P)
The regex to match such lines assumes the info can be extended in the
future to other uppercase characters, and will need to be extended in case
other characters will be used. Using a much more generic regex might incur
in false positives, so this looked like a good tradeoff.
Link: https://lkml.kernel.org/r/20241230-decode_stacktrace-fix-info-v1-1-984910659173@bootlin.com
Fixes: bdf8eafbf7 ("arm64: stacktrace: report source of unwind data")
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Mark Brown <broonie@kernel.org>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Miroslav Benes <mbenes@suse.cz>
Cc: Puranjay Mohan <puranjay@kernel.org>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Sometimes there are special characters around module names in stack
traces, such as ARM32 with BACKTRACE_VERBOSE in "(%pS)" format, such as:
[<806e4845>] (dump_stack_lvl) from [<7f806013>] (hello_init+0x13/0x1000
[test])
In this case, $module will be "[test])", the trace can be decoded by
stripping the right parenthesis first: (dump_stack_lvl) from hello_init
(/foo/test.c:10) test.
Link: https://lkml.kernel.org/r/20240524042600.14738-3-xndchn@gmail.com
Signed-off-by: Xiong Nandi <xndchn@gmail.com>
Suggested-by: Elliot Berman <quic_eberman@quicinc.com>
Cc: Bjorn Andersson <quic_bjorande@quicinc.com>
Cc: Carlos Llamas <cmllamas@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
GNU's addr2line can have problems parsing a vmlinux built with LLVM,
particularly when LTO was used. In order to decode the traces correctly
this patch adds the ability to switch to LLVM's utilities readelf and
addr2line. The same approach is followed by Will in [1].
Before:
$ scripts/decode_stacktrace.sh vmlinux < kernel.log
[17716.240635] Call trace:
[17716.240646] skb_cow_data (??:?)
[17716.240654] esp6_input (ld-temp.o:?)
[17716.240666] xfrm_input (ld-temp.o:?)
[17716.240674] xfrm6_rcv (??:?)
[...]
After:
$ LLVM=1 scripts/decode_stacktrace.sh vmlinux < kernel.log
[17716.240635] Call trace:
[17716.240646] skb_cow_data (include/linux/skbuff.h:2172 net/core/skbuff.c:4503)
[17716.240654] esp6_input (net/ipv6/esp6.c:977)
[17716.240666] xfrm_input (net/xfrm/xfrm_input.c:659)
[17716.240674] xfrm6_rcv (net/ipv6/xfrm6_input.c:172)
[...]
Note that one could set CROSS_COMPILE=llvm- instead to hack around this
issue. However, doing so can break the decodecode routine as it will
force the selection of other LLVM utilities down the line e.g. llvm-as.
[1] https://lore.kernel.org/all/20230914131225.13415-3-will@kernel.org/
Link: https://lkml.kernel.org/r/20230929034836.403735-1-cmllamas@google.com
Signed-off-by: Carlos Llamas <cmllamas@google.com>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
Reviewed-by: Elliot Berman <quic_eberman@quicinc.com>
Tested-by: Justin Stitt <justinstitt@google.com>
Cc: Will Deacon <will@kernel.org>
Cc: John Stultz <jstultz@google.com>
Cc: Masahiro Yamada <masahiroy@kernel.org>
Cc: Nathan Chancellor <nathan@kernel.org>
Cc: Tom Rix <trix@redhat.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
When the kernel log is acquired over a serial cable it is not uncommon for
the log to contain carriage return characters, in addition to the expected
line feeds.
When this output is feed into decode_stacktrace.sh, handle_line() fails to
strip the trailing ']' off the module name, which results in find_module()
not being able to find the referred to kernel module. This is reported to
the user as:
WARNING! Modules path isn't set, but is needed to parse this symbol
The solution is to reconfigure the serial port, or to strip the carriage
returns from the log, but this isn't obvious from the error reported by
the script.
Instead, make decode_stacktrace.sh more user friendly by stripping the
trailing carriage return.
Link: https://lkml.kernel.org/r/20231225-decode-stacktrace-cr-v1-1-9f306f38cdde@quicinc.com
Signed-off-by: Bjorn Andersson <quic_bjorande@quicinc.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Old bash version don't support associative array variables. Avoid to use
associative array variables to avoid error.
Without this, old bash version will report error as fellowing
[ 15.954042] Kernel panic - not syncing: sysrq triggered crash
[ 15.955252] CPU: 1 PID: 167 Comm: sh Not tainted 5.18.0-rc1-00208-gb7d075db2fd5 #4
[ 15.956472] Hardware name: Hobot J5 Virtual development board (DT)
[ 15.957856] Call trace:
./scripts/decode_stacktrace.sh: line 128: ,dump_backtrace: syntax error: operand expected (error token is ",dump_backtrace")
Link: https://lkml.kernel.org/r/20220409180331.24047-1-schspa@gmail.com
Signed-off-by: Schspa Shi <schspa@gmail.com>
Cc: Stephen Boyd <swboyd@chromium.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Try to find module in directory with vmlinux (for fresh build). Then try
standard paths where debuginfo are usually placed. Pick first file which
have elf section '.debug_line'.
Before:
$ echo 'tap_open+0x0/0x0 [tap]' |
./scripts/decode_stacktrace.sh /usr/lib/debug/boot/vmlinux-5.4.0-37-generic
WARNING! Modules path isn't set, but is needed to parse this symbol
tap_open+0x0/0x0 tap
After:
$ echo 'tap_open+0x0/0x0 [tap]' |
./scripts/decode_stacktrace.sh /usr/lib/debug/boot/vmlinux-5.4.0-37-generic
tap_open (drivers/net/tap.c:502) tap
Signed-off-by: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Cc: Sasha Levin <sashal@kernel.org>
Link: http://lkml.kernel.org/r/159282923068.248444.5461337458421616083.stgit@buzz
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
