One common cause of modpost version generation failures is a failure to
prototype exported assembly functions - the tooling requires this for
exported functions even if they are not and should not be called from C
code in order to do the version mangling for symbols. Unfortunately the
error message is currently rather abstruse, simply saying that "version
generation failed" and even diving into the code doesn't directly show
what's going on since there's several steps between the problem and it
being observed.
Provide an explicit hint as to the likely cause of a version generation
failure to help anyone who runs into this in future more readily diagnose
and fix the problem.
Signed-off-by: Mark Brown <broonie@kernel.org>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Currently, clang LTO built vmlinux won't work with pahole.
LTO introduced cross-cu dwarf tag references and broke
current pahole model which handles one cu as a time.
The solution is to merge all cu's as one pahole cu as in [1].
We would like to do this merging only if cross-cu dwarf
references happens. The LTO build mode is a pretty good
indication for that.
In earlier version of this patch ([2]), clang flag
-grecord-gcc-switches is proposed to add to compilation flags
so pahole could detect "-flto" and then merging cu's.
This will increate the binary size of 1% without LTO though.
Arnaldo suggested to use a note to indicate the vmlinux
is built with LTO. Such a cheap way to get whether the vmlinux
is built with LTO or not helps pahole but is also useful
for tracing as LTO may inline/delete/demote global functions,
promote static functions, etc.
So this patch added an elfnote with a new type LINUX_ELFNOTE_LTO_INFO.
The owner of the note is "Linux".
With gcc 8.4.1 and clang trunk, without LTO, I got
$ readelf -n vmlinux
Displaying notes found in: .notes
Owner Data size Description
...
Linux 0x00000004 func
description data: 00 00 00 00
...
With "readelf -x ".notes" vmlinux", I can verify the above "func"
with type code 0x101.
With clang thin-LTO, I got the same as above except the following:
description data: 01 00 00 00
which indicates the vmlinux is built with LTO.
[1] https://lore.kernel.org/bpf/20210325065316.3121287-1-yhs@fb.com/
[2] https://lore.kernel.org/bpf/20210331001623.2778934-1-yhs@fb.com/
Suggested-by: Arnaldo Carvalho de Melo <arnaldo.melo@gmail.com>
Signed-off-by: Yonghong Song <yhs@fb.com>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
Tested-by: Sedat Dilek <sedat.dilek@gmail.com> # LLVM/Clang v12.0.0-rc4 (x86-64)
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Nathan reports that the mips defconfig emits the following warning:
WARNING: modpost: Symbol info of vmlinux is missing. Unresolved symbol check will be entirely skipped.
This false-positive happens when CONFIG_TRIM_UNUSED_KSYMS is enabled,
but no CONFIG option is set to 'm'.
Commit a0590473c5 ("nfs: fix PNFS_FLEXFILE_LAYOUT Kconfig default")
turned the last 'm' into 'y' for the mips defconfig, and uncovered
this issue.
In this case, the module feature itself is enabled, but we have no
module to build. As a result, CONFIG_TRIM_UNUSED_KSYMS drops all the
instances of EXPORT_SYMBOL. Then, modpost wrongly assumes vmlinux is
missing because vmlinux.symvers is empty. (As another false-positive
case, you can create a module that does not use any symbol of vmlinux).
The current behavior is to entirely suppress the unresolved symbol
warnings when vmlinux is missing just because there are too many.
I found the origin of this code in the historical git tree. [1]
If this is a matter of noisiness, I think modpost can display the
first 10 warnings, and the number of suppressed warnings at the end.
You will get a bit noisier logs when you run 'make modules' without
vmlinux, but such warnings are better to show because you never know
the resulting modules are actually loadable or not.
This commit changes the following:
- If any of input *.symver files is missing, pass -w option to let
the module build keep going with warnings instead of errors.
- If there are too many (10+) unresolved symbol warnings, show only
the first 10, and also the number of suppressed warnings.
[1]: https://git.kernel.org/pub/scm/linux/kernel/git/history/history.git/commit/?id=1cc0e0529569bf6a94f6d49770aa6d4b599d2c46
Reported-by: Nathan Chancellor <nathan@kernel.org>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
The external module build shows the following warning if Module.symvers
is missing in the kernel tree.
WARNING: Symbol version dump "Module.symvers" is missing.
Modules may not have dependencies or modversions.
I think this is an important heads-up because the resulting modules may
not work as expected. This happens when you did not build the entire
kernel tree, for example, you might have prepared the minimal setups
for external modules by 'make defconfig && make modules_preapre'.
A problem is that 'make modules' creates Module.symvers even without
vmlinux. In this case, that warning is suppressed since Module.symvers
already exists in spite of its incomplete content.
The incomplete (i.e. invalid) Module.symvers should not be created.
This commit changes the second pass of modpost to dump symbols into
modules-only.symvers. The final Module.symvers is created by
concatenating vmlinux.symvers and modules-only.symvers if both exist.
Module.symvers is supposed to collect symbols from both vmlinux and
modules. It might be a bit confusing, and I am not quite sure if it
is an official interface, but presumably it is difficult to rename it
because some tools (e.g. kmod) parse it.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Pull module updates from Jessica Yu:
- Retire EXPORT_UNUSED_SYMBOL() and EXPORT_SYMBOL_GPL_FUTURE(). These
export types were introduced between 2006 - 2008. All the of the
unused symbols have been long removed and gpl future symbols were
converted to gpl quite a long time ago, and I don't believe these
export types have been used ever since. So, I think it should be safe
to retire those export types now (Christoph Hellwig)
- Refactor and clean up some aged code cruft in the module loader
(Christoph Hellwig)
- Build {,module_}kallsyms_on_each_symbol only when livepatching is
enabled, as it is the only caller (Christoph Hellwig)
- Unexport find_module() and module_mutex and fix the last module
callers to not rely on these anymore. Make module_mutex internal to
the module loader (Christoph Hellwig)
- Harden ELF checks on module load and validate ELF structures before
checking the module signature (Frank van der Linden)
- Fix undefined symbol warning for clang (Fangrui Song)
- Fix smatch warning (Dan Carpenter)
* tag 'modules-for-v5.12' of git://git.kernel.org/pub/scm/linux/kernel/git/jeyu/linux:
module: potential uninitialized return in module_kallsyms_on_each_symbol()
module: remove EXPORT_UNUSED_SYMBOL*
module: remove EXPORT_SYMBOL_GPL_FUTURE
module: move struct symsearch to module.c
module: pass struct find_symbol_args to find_symbol
module: merge each_symbol_section into find_symbol
module: remove each_symbol_in_section
module: mark module_mutex static
kallsyms: only build {,module_}kallsyms_on_each_symbol when required
kallsyms: refactor {,module_}kallsyms_on_each_symbol
module: use RCU to synchronize find_module
module: unexport find_module and module_mutex
drm: remove drm_fb_helper_modinit
powerpc/powernv: remove get_cxl_module
module: harden ELF info handling
module: Ignore _GLOBAL_OFFSET_TABLE_ when warning for undefined symbols
EXPORT_UNUSED_SYMBOL* is not actually used anywhere. Remove the
unused functionality as we generally just remove unused code anyway.
Reviewed-by: Miroslav Benes <mbenes@suse.cz>
Reviewed-by: Emil Velikov <emil.l.velikov@gmail.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Jessica Yu <jeyu@kernel.org>
As far as I can tell this has never been used at all, and certainly
not any time recently.
Reviewed-by: Miroslav Benes <mbenes@suse.cz>
Reviewed-by: Emil Velikov <emil.l.velikov@gmail.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Jessica Yu <jeyu@kernel.org>
Using EXPORT_SYMBOL*() on static functions is fundamentally wrong.
Modpost currently reports that as a warning, but clearly this is not a
pattern we should allow, and all in-tree occurences should have been
fixed by now. So, promote the warn() message to error() to make sure
this never happens again.
Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-by: Matthias Maennich <maennich@google.com>
Signed-off-by: Quentin Perret <qperret@google.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
There is code that reports static EXPORT_SYMBOL a few lines below.
It is not a good idea to bail out here.
I renamed sec_mismatch_fatal to sec_mismatch_warn_only (with logical
inversion) to match to CONFIG_SECTION_MISMATCH_WARN_ONLY.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Change fatal() to error() to continue running to report more possible
issues.
There is no difference in the fact that modpost will fail anyway.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
We have 3 log functions. fatal() is special because it lets modpost bail
out immediately. The difference between warn() and error() is the only
prefix parts ("WARNING:" vs "ERROR:").
In my understanding, the expected handling of error() is to propagate
the return code of the function to the exit code of modpost, as
check_exports() etc. already does. This is a good manner in general
because we should display as many error messages as possible in a
single run of modpost.
What is annoying about fatal() is that it kills modpost at the first
error. People would need to run Kbuild again and again until they fix
all errors.
But, unfortunately, people tend to do:
"This case should not be allowed. Let's replace warn() with fatal()."
One of the reasons is probably it is tedious to manually hoist the error
code to the main() function.
This commit refactors error() so any single call for it automatically
makes modpost return the error code.
I also added comments in modpost.h for warn(), error(), and fatal().
Please use fatal() only when you have a strong reason to do so.
For example:
- Memory shortage (i.e. malloc() etc. has failed)
- The ELF file is broken, and there is no point to continue parsing
- Something really odd has happened
For general coding errors, please use error().
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Tested-by: Quentin Perret <qperret@google.com>
The log function names, warn(), merror(), fatal() are inconsistent.
Commit 2a11665945 ("kbuild: distinguish between errors and warnings
in modpost") intentionally chose merror() to avoid the conflict with
the library function error(). See man page of error(3).
But, we are already causing the conflict with warn() because it is also
a library function. See man page of warn(3). err() would be a problem
for the same reason.
The common technique to work around name conflicts is to use macros.
For example:
/* in a header */
#define error(fmt, ...) __error(fmt, ##__VA_ARGS__)
#define warn(fmt, ...) __warn(fmt, ##__VA_ARGS__)
/* function definition */
void __error(const char *fmt, ...)
{
<our implementation>
}
void __warn(const char *fmt, ...)
{
<our implementation>
}
In this way, we can implement our own warn() and error(), still we can
include <error.h> and <err.h> with no problem.
And, commit 93c95e526a ("modpost: rework and consolidate logging
interface") already did that.
Since the log functions are all macros, we can use error() without
causing "conflicting types" errors.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Mention why we open-code strsep, so it is clear that it is intentional.
Fixes: 736bb11898 ("modpost: remove use of non-standard strsep() in HOSTCC code")
Signed-off-by: Wolfram Sang <wsa@kernel.org>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
strsep() is neither standard C nor POSIX and used outside
the kernel code here. Using it here requires that the
build host supports it out of the box which is e.g.
not true for a Darwin build host and using a cross-compiler.
This leads to:
scripts/mod/modpost.c:145:2: warning: implicit declaration of function 'strsep' [-Wimplicit-function-declaration]
return strsep(stringp, "\n");
^
and a segfault when running MODPOST.
See also: https://stackoverflow.com/a/7219504
So let's replace this by strchr() instead of using strsep().
It does not hurt kernel size or speed since this code is run
on the build host.
Fixes: ac5100f543 ("modpost: add read_text_file() and get_line() helpers")
Co-developed-by: Masahiro Yamada <masahiroy@kernel.org>
Signed-off-by: H. Nikolaus Schaller <hns@goldelico.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Pull Kbuild updates from Masahiro Yamada:
- fix warnings in 'make clean' for ARCH=um, hexagon, h8300, unicore32
- ensure to rebuild all objects when the compiler is upgraded
- exclude system headers from dependency tracking and fixdep processing
- fix potential bit-size mismatch between the kernel and BPF user-mode
helper
- add the new syntax 'userprogs' to build user-space programs for the
target architecture (the same arch as the kernel)
- compile user-space sample code under samples/ for the target arch
instead of the host arch
- make headers_install fail if a CONFIG option is leaked to user-space
- sanitize the output format of scripts/checkstack.pl
- handle ARM 'push' instruction in scripts/checkstack.pl
- error out before modpost if a module name conflict is found
- error out when multiple directories are passed to M= because this
feature is broken for a long time
- add CONFIG_DEBUG_INFO_COMPRESSED to support compressed debug info
- a lot of cleanups of modpost
- dump vmlinux symbols out into vmlinux.symvers, and reuse it in the
second pass of modpost
- do not run the second pass of modpost if nothing in modules is
updated
- install modules.builtin(.modinfo) by 'make install' as well as by
'make modules_install' because it is useful even when
CONFIG_MODULES=n
- add new command line variables, GZIP, BZIP2, LZOP, LZMA, LZ4, and XZ
to allow users to use alternatives such as pigz, pbzip2, etc.
* tag 'kbuild-v5.8' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: (96 commits)
kbuild: add variables for compression tools
Makefile: install modules.builtin even if CONFIG_MODULES=n
mksysmap: Fix the mismatch of '.L' symbols in System.map
kbuild: doc: rename LDFLAGS to KBUILD_LDFLAGS
modpost: change elf_info->size to size_t
modpost: remove is_vmlinux() helper
modpost: strip .o from modname before calling new_module()
modpost: set have_vmlinux in new_module()
modpost: remove mod->skip struct member
modpost: add mod->is_vmlinux struct member
modpost: remove is_vmlinux() call in check_for_{gpl_usage,unused}()
modpost: remove mod->is_dot_o struct member
modpost: move -d option in scripts/Makefile.modpost
modpost: remove -s option
modpost: remove get_next_text() and make {grab,release_}file static
modpost: use read_text_file() and get_line() for reading text files
modpost: avoid false-positive file open error
modpost: fix potential mmap'ed file overrun in get_src_version()
modpost: add read_text_file() and get_line() helpers
modpost: do not call get_modinfo() for vmlinux(.o)
...
Now that is_vmlinux() is called only in new_module(), we can inline
the function call.
modname is the basename with '.o' is stripped. No need to compare it
with 'vmlinux.o'.
vmlinux is always located at the current working directory. No need
to strip the directory path.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
new_module() conditionally strips the .o because the modname has .o
suffix when it is called from read_symbols(), but no .o when it is
called from read_dump().
It is clearer to strip .o in read_symbols().
I also used flexible-array for mod->name.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
The meaning of 'skip' is obscure since it does not explain
"what to skip".
mod->skip is set when it is vmlinux or the module info came from
a dump file.
So, mod->skip is equivalent to (mod->is_vmlinux || mod->from_dump).
For the check in write_namespace_deps_files(), mod->is_vmlinux is
unneeded because the -d option is not passed in the first pass of
modpost.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
is_vmlinux() is called in several places to check whether the current
module is vmlinux or not.
It is faster and clearer to check mod->is_vmlinux flag.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
