For all CONFIG_CFI+CONFIG_CALL_PADDING configs, for C functions, the
__cfi_ symbols only cover the 5-byte kCFI type hash. After that there
also N bytes of NOP padding between the hash and the function entry
which aren't associated with any symbol.
The NOPs can be replaced with actual code at runtime. Without a symbol,
unwinders and tooling have no way of knowing where those bytes belong.
Grow the existing __cfi_* symbols to fill that gap.
Note that assembly functions with SYM_TYPED_FUNC_START() aren't affected
by this issue, their __cfi_ symbols also cover the padding.
Also, CONFIG_PREFIX_SYMBOLS has no reason to exist: CONFIG_CALL_PADDING
is what causes the compiler to emit NOP padding before function entry
(via -fpatchable-function-entry), so it's the right condition for
creating prefix symbols.
Remove CONFIG_PREFIX_SYMBOLS, as it's no longer needed. Simplify the
LONGEST_SYM_KUNIT_TEST dependency accordingly. Rework objtool's
arguments a bit to handle the variety of prefix/cfi-related cases.
Suggested-by: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Section symbols aren't grouped after their corresponding FILE symbols.
Their sym->file should really be NULL rather than whatever random FILE
happened to be last.
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Song Liu <song@kernel.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
In find_reloc_by_dest_range(), hash collisions can cause a high-offset
relocation to appear when probing a low-offset hash bucket.
Only return early when the best match found so far genuinely belongs to
the current bucket (its offset is within the bucket's stride range).
Otherwise, continue scanning later buckets which may contain
lower-offset matches.
This ensures the first reloc in the range gets returned.
Fixes: 74b873e49d ("objtool: Optimize find_rela_by_dest_range()")
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Song Liu <song@kernel.org>
Reviewed-by: Miroslav Benes <mbenes@suse.cz>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
klp-diff treats all rodata as uncorrelated, so any reference to it uses
a duplicated copy rather than using a KLP reloc.
For the contents of the data itself, a duplicated copy is fine.
However, pointer comparisons (e.g., f->f_op == &foo_ops) are broken.
Fix it by correlating non-anonymous rodata objects.
Also, use a new find_symbol_containing_inclusive() helper for matching
the end of a symbol so bounds calculations don't get broken, for the
case where an array or other symbol's ending address is used as part of
a bounds calculation.
While these are really two distinct changes, they need to be done in the
same patch so as to avoid introducing bisection regressions.
Acked-by: Song Liu <song@kernel.org>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Move the sec->rodata marking from check.c to elf.c so it's set during
ELF reading rather than during the check pipeline. This makes the
rodata flag available to all objtool users, including klp-diff which
reads ELF files directly without running check().
Add an is_rodata_sec() helper to elf.h for consistency with
is_text_sec() and is_string_sec().
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Song Liu <song@kernel.org>
Reviewed-by: Miroslav Benes <mbenes@suse.cz>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Convert the callback-based iterate_sym_by_demangled_name() with a new
for_each_sym_by_demangled_name() macro. This eliminates the callback
struct/function and makes the code more compact and readable.
Acked-by: Song Liu <song@kernel.org>
Reviewed-by: Miroslav Benes <mbenes@suse.cz>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Pull objtool updates from Ingo Molnar:
- KLP support updates and fixes (Song Liu)
- KLP-build script updates and fixes (Joe Lawrence)
- Support Clang RAX DRAP sequence, to address clang false positive
(Josh Poimboeuf)
- Reorder ORC register numbering to match regular x86 register
numbering (Josh Poimboeuf)
- Misc cleanups (Wentong Tian, Song Liu)
* tag 'objtool-core-2026-04-13' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
objtool/x86: Reorder ORC register numbering
objtool: Support Clang RAX DRAP sequence
livepatch/klp-build: report patch validation fuzz
livepatch/klp-build: add terminal color output
livepatch/klp-build: provide friendlier error messages
livepatch/klp-build: improve short-circuit validation
livepatch/klp-build: fix shellcheck complaints
livepatch/klp-build: add Makefile with check target
livepatch/klp-build: add grep-override function
livepatch/klp-build: switch to GNU patch and recountdiff
livepatch/klp-build: support patches that add/remove files
objtool/klp: Correlate locals to globals
objtool/klp: Match symbols based on demangled_name for global variables
objtool/klp: Remove .llvm suffix in demangle_name()
objtool/klp: Also demangle global objects
objtool/klp: Use sym->demangled_name for symbol_name hash
objtool/klp: Remove trailing '_' in demangle_name()
objtool/klp: Remove redundant strcmp() in correlate_symbols()
objtool: Use section/symbol type helpers
The elf_create_file() function fails with EINVAL when the build directory
path is long enough to truncate the "XXXXXX" suffix in the 256-byte
tmp_name buffer.
Simplify the code to remove the unnecessary dirname()/basename() split
and concatenation. Instead, allocate the exact number of bytes needed for
the path.
Acked-by: Song Liu <song@kernel.org>
Signed-off-by: Joe Lawrence <joe.lawrence@redhat.com>
Link: https://patch.msgid.link/20260310203751.1479229-3-joe.lawrence@redhat.com
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
correlate_symbols() will always try to match full name first. If there is
no match, try match only demangled_name.
In very rare cases, it is possible to have multiple foo.llvm.<hash> in
the same kernel. Whenever there is ambiguity like this, fail the klp diff.
Signed-off-by: Song Liu <song@kernel.org>
Link: https://patch.msgid.link/20260305231531.3847295-7-song@kernel.org
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
For klp-build with LTO, it is necessary to correlate demangled symbols,
e.g., correlate foo.llvm.<num 1> and foo.llvm.<num 2>. However, these two
symbols do not have the same str_hash(name). To be able to correlate the
two symbols, calculate hash based on demanged_name, so that these two
symbols have the same hash.
No functional changes intended.
Signed-off-by: Song Liu <song@kernel.org>
Link: https://patch.msgid.link/20260305231531.3847295-4-song@kernel.org
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
With CONFIG_LTO_CLANG_THIN, it is possible to have nested __UNIQUE_ID_,
such as:
__UNIQUE_ID_addressable___UNIQUE_ID_pci_invalid_bar_694_695
To remove both trailing numbers, also remove trailing '_'.
Also add comments to demangle_name().
Signed-off-by: Song Liu <song@kernel.org>
Link: https://patch.msgid.link/20260305231531.3847295-3-song@kernel.org
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
The source file tools/objtool/elf.c defines the macros ALIGN_UP(),
ALIGN_UP_POW2() and MAX(). These macros unnecessarily duplicate
functionality already available under tools/include/, specifically ALIGN(),
roundup_pow_of_two() and max().
More importantly, the definition of ALIGN_UP_POW2() is incorrect when the
input is 1, as it results in a call to __builtin_clz(0), which produces an
undefined result. This issue impacts the function elf_alloc_reloc(). When
adding the first relocation to a section, the function allocates an
undefined number of relocations.
Replace the custom macros with the shared functionality to resolve these
issues.
Fixes: 2c05ca0262 ("objtool: Add elf_create_reloc() and elf_init_reloc()")
Signed-off-by: Petr Pavlu <petr.pavlu@suse.com>
Link: https://patch.msgid.link/20260126151356.3924887-1-petr.pavlu@suse.com
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
The .cold function parent/child correlation logic has two passes: one in
read_symbols() and one in add_jump_destinations().
The second pass was added with commit cd77849a69 ("objtool: Fix GCC 8
cold subfunction detection for aliased functions") to ensure that if the
parent symbol had aliases then the canonical symbol was chosen as the
parent.
That solution was rather clunky, not to mention incomplete due to the
existence of alternatives and switch tables. Now that we have
sym->alias, the canonical alias fix can be done much simpler in the
first pass, making the second pass obsolete.
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/bdab245a38000a5407f663a031f39e14c67a43d4.1763671318.git.jpoimboe@kernel.org
The objtool .cold child/parent correlation is done in two phases: first
in elf_add_symbol() and later in add_jump_destinations().
The first phase is rather crude and can pick the wrong parent if there
are duplicates with the same name.
The second phase usually fixes that, but only if the parent has a direct
jump to the child. It does *not* work if the only branch from the
parent to the child is an alternative or jump table entry.
Make the first phase more robust by looking for the parent in the same
STT_FILE as the child.
Fixes the following objtool warnings in an AutoFDO build with a large
CLANG_AUTOFDO_PROFILE profile:
vmlinux.o: warning: objtool: rdev_add_key() falls through to next function rdev_add_key.cold()
vmlinux.o: warning: objtool: rdev_set_default_key() falls through to next function rdev_set_default_key.cold()
Fixes: 13810435b9 ("objtool: Support GCC 8's cold subfunctions")
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/82c7b52e40efa75dd10e1c550cc75c1ce10ac2c9.1763671318.git.jpoimboe@kernel.org
AutoFDO enables -fsplit-machine-functions which can move the cold parts
of a function to a <func>.cold symbol in a .text.split.<func> section.
Unlike GCC, the Clang <func>.cold symbols are not marked STT_FUNC. This
confuses objtool in several ways, resulting in warnings like the
following:
vmlinux.o: warning: objtool: apply_retpolines.cold+0xfc: unsupported instruction in callable function
vmlinux.o: warning: objtool: machine_check_poll.cold+0x2e: unsupported instruction in callable function
vmlinux.o: warning: objtool: free_deferred_objects.cold+0x1f: relocation to !ENDBR: free_deferred_objects.cold+0x26
vmlinux.o: warning: objtool: rpm_idle.cold+0xe0: relocation to !ENDBR: rpm_idle.cold+0xe7
vmlinux.o: warning: objtool: tcp_rcv_state_process.cold+0x1c: relocation to !ENDBR: tcp_rcv_state_process.cold+0x23
Fix it by marking the .cold symbols as STT_FUNC.
Fixes: 2fd65f7afd ("AutoFDO: Enable machine function split optimization for AutoFDO")
Closes: https://lore.kernel.org/20251103215244.2080638-2-xur@google.com
Reported-by: Rong Xu <xur@google.com>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: xur@google.com
Tested-by: xur@google.com
Link: https://patch.msgid.link/20a67326f04b2a361c031b56d58e8a803b3c5893.1763671318.git.jpoimboe@kernel.org
In preparation for klp-build, enable "classic" objtool to work on
livepatch modules:
- Avoid duplicate symbol/section warnings for prefix symbols and the
.static_call_sites and __mcount_loc sections which may have already
been extracted by klp diff.
- Add __klp_funcs to the IBT function pointer section whitelist.
- Prevent KLP symbols from getting incorrectly classified as cold
subfunctions.
Acked-by: Petr Mladek <pmladek@suse.com>
Tested-by: Joe Lawrence <joe.lawrence@redhat.com>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
The prefix symbol creation code currently ignores all errors, presumably
because some functions don't have the leading NOPs.
Shuffle the code around a bit, improve the error handling and document
why some errors are ignored.
Acked-by: Petr Mladek <pmladek@suse.com>
Tested-by: Joe Lawrence <joe.lawrence@redhat.com>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Add a new klp diff subcommand which performs a binary diff between two
object files and extracts changed functions into a new object which can
then be linked into a livepatch module.
This builds on concepts from the longstanding out-of-tree kpatch [1]
project which began in 2012 and has been used for many years to generate
livepatch modules for production kernels. However, this is a complete
rewrite which incorporates hard-earned lessons from 12+ years of
maintaining kpatch.
Key improvements compared to kpatch-build:
- Integrated with objtool: Leverages objtool's existing control-flow
graph analysis to help detect changed functions.
- Works on vmlinux.o: Supports late-linked objects, making it
compatible with LTO, IBT, and similar.
- Simplified code base: ~3k fewer lines of code.
- Upstream: No more out-of-tree #ifdef hacks, far less cruft.
- Cleaner internals: Vastly simplified logic for symbol/section/reloc
inclusion and special section extraction.
- Robust __LINE__ macro handling: Avoids false positive binary diffs
caused by the __LINE__ macro by introducing a fix-patch-lines script
(coming in a later patch) which injects #line directives into the
source .patch to preserve the original line numbers at compile time.
Note the end result of this subcommand is not yet functionally complete.
Livepatch needs some ELF magic which linkers don't like:
- Two relocation sections (.rela*, .klp.rela*) for the same text
section.
- Use of SHN_LIVEPATCH to mark livepatch symbols.
Unfortunately linkers tend to mangle such things. To work around that,
klp diff generates a linker-compliant intermediate binary which encodes
the relevant KLP section/reloc/symbol metadata.
After module linking, a klp post-link step (coming soon) will clean up
the mess and convert the linked .ko into a fully compliant livepatch
module.
Note this subcommand requires the diffed binaries to have been compiled
with -ffunction-sections and -fdata-sections, and processed with
'objtool --checksum'. Those constraints will be handled by a klp-build
script introduced in a later patch.
Without '-ffunction-sections -fdata-sections', reliable object diffing
would be infeasible due to toolchain limitations:
- For intra-file+intra-section references, the compiler might
occasionally generated hard-coded instruction offsets instead of
relocations.
- Section-symbol-based references can be ambiguous:
- Overlapping or zero-length symbols create ambiguity as to which
symbol is being referenced.
- A reference to the end of a symbol (e.g., checking array bounds)
can be misinterpreted as a reference to the next symbol, or vice
versa.
A potential future alternative to '-ffunction-sections -fdata-sections'
would be to introduce a toolchain option that forces symbol-based
(non-section) relocations.
Acked-by: Petr Mladek <pmladek@suse.com>
Tested-by: Joe Lawrence <joe.lawrence@redhat.com>
Signed-off-by: Josh Poimboeuf <jpoimboe@kernel.org>
Josh Poimboeuf
Linus Torvalds
Joe LawrenceandJosh Poimboeuf
Song LiuandJosh Poimboeuf
Wentong TianandJosh Poimboeuf
Petr PavluandJosh Poimboeuf
Chen NiandJosh Poimboeuf