When the kallsyms relative base was introduced, per-CPU variable
references on x86_64 SMP were implemented as offsets into the respective
per-CPU region, rather than offsets relative to the location of the
variable's template in the kernel image, which is how other
architectures implement it.
This required kallsyms to reason about the difference between the two,
and the sign of the value in the kallsyms_offsets[] array was used to
distinguish them. This meant that negative offsets were not permitted
for ordinary variables, and so it was crucial that the relative base was
chosen such that all offsets were positive numbers.
This is no longer needed: instead, the offsets can simply be encoded as
values in the range -/+ 2 GiB, which is precisely what PC32 relocations
provide on most architectures. So it is possible to simplify the logic,
and just use _text as the anchor directly, and let the linker calculate
the final value based on the location of the entry itself.
Some architectures (nios2, extensa) do not support place-relative
relocations at all, but these are all 32-bit and non-relocatable, and so
there is no need for place-relative relocations in the first place, and
the actual symbol values can just be stored directly.
This makes all entries in the kallsyms_offsets[] array visible as
place-relative references in the ELF metadata, which will be important
when implementing ELF-based fg-kaslr.
Reviewed-by: Kees Cook <kees@kernel.org>
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Link: https://patch.msgid.link/20260116093359.2442297-6-ardb+git@google.com
Signed-off-by: Nathan Chancellor <nathan@kernel.org>
After commit 64e166099b ("kallsyms: get rid of code for absolute,
kallsyms"), there is only one call site for output_address(). Squash it.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
When malloc() or realloc() fails, there is not much userspace programs
can do. xmalloc() and xrealloc() are useful to bail out on a memory
allocation failure.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
The struct sym_entry uses the 'seq' and 'start_pos' fields to remember
the index in the symbol table. They serve the same purpose and are not
used simultaneously. Unify them.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Commit bea5b74504 ("kallsyms: expand symbol name into comment for
debugging") added the uncompressed type/name in the comment lines of
kallsyms_offsets.
It would be useful to do the same for kallsyms_names and
kallsyms_seqs_of_names.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
This reimplements commit 951bcae6c5 ("kallsyms: Avoid weak references
for kallsyms symbols") because I am not a big fan of PROVIDE().
As an alternative solution, this commit prepends one more kallsyms step.
KSYMS .tmp_vmlinux.kallsyms0.S # added
AS .tmp_vmlinux.kallsyms0.o # added
LD .tmp_vmlinux.btf
BTF .btf.vmlinux.bin.o
LD .tmp_vmlinux.kallsyms1
NM .tmp_vmlinux.kallsyms1.syms
KSYMS .tmp_vmlinux.kallsyms1.S
AS .tmp_vmlinux.kallsyms1.o
LD .tmp_vmlinux.kallsyms2
NM .tmp_vmlinux.kallsyms2.syms
KSYMS .tmp_vmlinux.kallsyms2.S
AS .tmp_vmlinux.kallsyms2.o
LD vmlinux
Step 0 takes /dev/null as input, and generates .tmp_vmlinux.kallsyms0.o,
which has a valid kallsyms format with the empty symbol list, and can be
linked to vmlinux. Since it is really small, the added compile-time cost
is negligible.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Acked-by: Ard Biesheuvel <ardb@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
getline() returns -1 at EOF as well as on error. It also doesn't set
errno to 0 on success, so initialize it to 0 before using errno to check
for an error condition. See the paragraph here [1]:
For some system calls and library functions (e.g., getpriority(2)),
-1 is a valid return on success. In such cases, a successful return
can be distinguished from an error return by setting errno to zero
before the call, and then, if the call returns a status that indicates
that an error may have occurred, checking to see if errno has a
nonzero value.
Bear has a bug [2] that launches processes with errno set and causes the
following build failure:
$ bear -- make LLVM=1
...
LD .tmp_vmlinux.kallsyms1
NM .tmp_vmlinux.kallsyms1.syms
KSYMS .tmp_vmlinux.kallsyms1.S
read_symbol: Invalid argument
[1]: https://linux.die.net/man/3/errno
[2]: https://github.com/rizsotto/Bear/issues/469
Fixes: 1c975da56a ("scripts/kallsyms: remove KSYM_NAME_LEN_BUFFER")
Reviewed-by: Miguel Ojeda <ojeda@kernel.org>
Signed-off-by: James Clark <james.clark@arm.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Commit 6eb4bd92c1 ("kallsyms: strip LTO suffixes from static functions")
stripped all function/variable suffixes started with '.' regardless
of whether those suffixes are generated at LTO mode or not. In fact,
as far as I know, in LTO mode, when a static function/variable is
promoted to the global scope, '.llvm.<...>' suffix is added.
The existing mechanism breaks live patch for a LTO kernel even if
no <symbol>.llvm.<...> symbols are involved. For example, for the following
kernel symbols:
$ grep bpf_verifier_vlog /proc/kallsyms
ffffffff81549f60 t bpf_verifier_vlog
ffffffff8268b430 d bpf_verifier_vlog._entry
ffffffff8282a958 d bpf_verifier_vlog._entry_ptr
ffffffff82e12a1f d bpf_verifier_vlog.__already_done
'bpf_verifier_vlog' is a static function. '_entry', '_entry_ptr' and
'__already_done' are static variables used inside 'bpf_verifier_vlog',
so llvm promotes them to file-level static with prefix 'bpf_verifier_vlog.'.
Note that the func-level to file-level static function promotion also
happens without LTO.
Given a symbol name 'bpf_verifier_vlog', with LTO kernel, current mechanism will
return 4 symbols to live patch subsystem which current live patching
subsystem cannot handle it. With non-LTO kernel, only one symbol
is returned.
In [1], we have a lengthy discussion, the suggestion is to separate two
cases:
(1). new symbols with suffix which are generated regardless of whether
LTO is enabled or not, and
(2). new symbols with suffix generated only when LTO is enabled.
The cleanup_symbol_name() should only remove suffixes for case (2).
Case (1) should not be changed so it can work uniformly with or without LTO.
This patch removed LTO-only suffix '.llvm.<...>' so live patching and
tracing should work the same way for non-LTO kernel.
The cleanup_symbol_name() in scripts/kallsyms.c is also changed to have the same
filtering pattern so both kernel and kallsyms tool have the same
expectation on the order of symbols.
[1] https://lore.kernel.org/live-patching/20230615170048.2382735-1-song@kernel.org/T/#u
Fixes: 6eb4bd92c1 ("kallsyms: strip LTO suffixes from static functions")
Reported-by: Song Liu <song@kernel.org>
Signed-off-by: Yonghong Song <yhs@fb.com>
Reviewed-by: Zhen Lei <thunder.leizhen@huawei.com>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
Acked-by: Song Liu <song@kernel.org>
Link: https://lore.kernel.org/r/20230628181926.4102448-1-yhs@fb.com
Signed-off-by: Kees Cook <keescook@chromium.org>
You do not need to decide the buffer size statically.
Use getline() to grow the line buffer as needed.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <n.schier@avm.de>
Commit 010a0aad39 ("kallsyms: Correctly sequence symbols when
CONFIG_LTO_CLANG=y") added --lto-clang, and updated the usage()
function, but not the comment. Update it in the same way.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
Currently, expand_symbol() is called many times to get the uncompressed
symbol names for sorting, and also for adding comments.
With the output order shuffled in the previous commit, the symbol data
are now written in the following order:
(1) kallsyms_num_syms
(2) kallsyms_names <-- need compressed names
(3) kallsyms_markers
(4) kallsyms_token_table
(5) kallsyms_token_index
(6) kallsyms_addressed / kallsyms_offsets <-- need uncompressed names (for commenting)
(7) kallsyms_relative_base
(8) kallsyms_seq_of_names <-- need uncompressed names (for sorting)
The compressed names are only needed by (2).
Call expand_symbol() between (2) and (3) to restore the original symbol
names. This requires just one expand_symbol() call for each symbol.
Call cleanup_symbol_name() between (7) and (8) instead of during sorting.
It is allowed to overwrite the ->sym field because (8) just outputs the
index instead of the name of each symbol. Again, this requires just one
cleanup_symbol_name() call for each symbol.
This refactoring makes it ~30% faster.
[Before]
$ time scripts/kallsyms --all-symbols --absolute-percpu --base-relative \
.tmp_vmlinux.kallsyms2.syms >/dev/null
real 0m1.027s
user 0m1.010s
sys 0m0.016s
[After]
$ time scripts/kallsyms --all-symbols --absolute-percpu --base-relative \
.tmp_vmlinux.kallsyms2.syms >/dev/null
real 0m0.717s
user 0m0.717s
sys 0m0.000s
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Currently, this tool outputs symbol data in the following order.
(1) kallsyms_addressed / kallsyms_offsets
(2) kallsyms_relative_base
(3) kallsyms_num_syms
(4) kallsyms_names
(5) kallsyms_markers
(6) kallsyms_seq_of_names
(7) kallsyms_token_table
(8) kallsyms_token_index
This commit changes the order as follows:
(1) kallsyms_num_syms
(2) kallsyms_names
(3) kallsyms_markers
(4) kallsyms_token_table
(5) kallsyms_token_index
(6) kallsyms_addressed / kallsyms_offsets
(7) kallsyms_relative_base
(8) kallsyms_seq_of_names
The motivation is to decrease the number of function calls to
expand_symbol() and cleanup_symbol_name().
The compressed names are only required for writing 'kallsyms_names'.
If you do this first, we can restore the original symbol names.
You do not need to repeat the same operation over again.
The actual refactoring will happen in the next commit.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
scripts/kallsyms.c maintains compiler-generated symbols, but we end up
with something similar in scripts/mksysmap to avoid the "Inconsistent
kallsyms data" error. For example, commit c17a253870 ("mksysmap: Fix
the mismatch of 'L0' symbols in System.map").
They were separately maintained prior to commit 94ff2f63d6 ("kbuild:
reuse mksysmap output for kallsyms").
Now that scripts/kallsyms.c parses the output of scripts/mksysmap,
it makes more sense to collect all the ignored patterns to mksysmap.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
Drop the symbols generated by scripts/kallsyms itself automatically
instead of maintaining the symbol list manually.
Pass the kallsyms object from the previous kallsyms step (if it exists)
as the third parameter of scripts/mksysmap, which will weed out the
generated symbols from the input to the next kallsyms step.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
The symbol types 'U' and 'N' are already filtered out by the following
line in scripts/mksysmap:
-e ' [aNUw] '
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
The assembler output of kallsyms.c is not meant for people to understand,
and is generally not helpful when debugging "Inconsistent kallsyms data"
warnings. I have previously struggled with these, but found it helpful
to list which symbols changed between the first and second pass in the
.tmp_vmlinux.kallsyms*.S files.
As this file is preprocessed, it's possible to add a C-style multiline
comment with the full type/name tuple.
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Ard BiesheuvelandNathan Chancellor
Brian GerstandIngo Molnar
Masahiro Yamada
Song LiuandKees Cook
Jann HornandMasahiro Yamada
Kent OverstreetandAndrew Morton
James ClarkandMasahiro Yamada
Yonghong SongandKees Cook
Arnd BergmannandMasahiro Yamada