When running the sign script the kernel is within the source directory
of external modules. This caused issues when the kernel uses relative
paths, like:
make[5]: Entering directory '/build/client/devel/kernel/work/linux-2.6'
make[6]: Entering directory '/build/client/devel/addmodules/vtx/work/vtx'
INSTALL /build/client/devel/addmodules/vtx/_/lib/modules/6.13.0-devel+/extra/vtx.ko
SIGN /build/client/devel/addmodules/vtx/_/lib/modules/6.13.0-devel+/extra/vtx.ko
/bin/sh: 1: scripts/sign-file: not found
DEPMOD /build/client/devel/addmodules/vtx/_/lib/modules/6.13.0-devel+
Working around it by using absolute pathes here.
Fixes: 13b25489b6 ("kbuild: change working directory to external module directory with M=")
Signed-off-by: Torsten Hilbrich <torsten.hilbrich@secunet.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
With the previous changes, $(extmod_prefix), $(MODORDER), and
$(MODULES_NSDEPS) are constant. (empty, modules.order, and
modules.nsdeps, respectively).
Remove these variables and hard-code their values.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
$(objtree) refers to the top of the output directory of kernel builds.
This commit adds the explicit $(objtree)/ prefix to build artifacts
needed for building external modules.
This change has no immediate impact, as the top-level Makefile
currently defines:
objtree := .
This commit prepares for supporting the building of external modules
in a different directory.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
Pull module updates from Luis Chamberlain:
"There are a few fixes / cleanups from Vincent, Chunhui, and Petr, but
the most important part of this pull request is the Rust community
stepping up to help maintain both C / Rust code for future Rust module
support. We grow the set of modules maintainers by three now, and with
this hope to scale to help address what's needed to properly support
future Rust module support.
A lot of exciting stuff coming in future kernel releases.
This has been on linux-next for ~ 3 weeks now with no issues"
* tag 'modules-6.12-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/linux:
module: Refine kmemleak scanned areas
module: abort module loading when sysfs setup suffer errors
MAINTAINERS: scale modules with more reviewers
module: Clean up the description of MODULE_SIG_<type>
module: Split modules_install compression and in-kernel decompression
Parallel execution is supported by GNU Make:
$ make -j<N> modules_install
It is questionable to enable multithreading within each zstd process
by default.
If you still want to do it, you can use the environment variable:
$ ZSTD_NBTHREADS=<N> make modules_install
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
Tested-by: Sedat Dilek <sedat.dilek@gmail.com>
The kernel configuration allows specifying a module compression mode. If
one is selected then each module gets compressed during
'make modules_install' and additionally one can also enable support for
a respective direct in-kernel decompression support. This means that the
decompression support cannot be enabled without the automatic compression.
Some distributions, such as the (open)SUSE family, use a signer service for
modules. A build runs on a worker machine but signing is done by a separate
locked-down server that is in possession of the signing key. The build
invokes 'make modules_install' to create a modules tree, collects
information about the modules, asks the signer service for their signature,
appends each signature to the respective module and compresses all modules.
When using this arrangment, the 'make modules_install' step produces
unsigned+uncompressed modules and the distribution's own build recipe takes
care of signing and compression later.
The signing support can be currently enabled without automatically signing
modules during 'make modules_install'. However, the in-kernel decompression
support can be selected only after first enabling automatic compression
during this step.
To allow only enabling the in-kernel decompression support without the
automatic compression during 'make modules_install', separate the
compression options similarly to the signing options, as follows:
> Enable loadable module support
[*] Module compression
Module compression type (GZIP) --->
[*] Automatically compress all modules
[ ] Support in-kernel module decompression
* "Module compression" (MODULE_COMPRESS) is a new main switch for the
compression/decompression support. It replaces MODULE_COMPRESS_NONE.
* "Module compression type" (MODULE_COMPRESS_<type>) chooses the
compression type, one of GZ, XZ, ZSTD.
* "Automatically compress all modules" (MODULE_COMPRESS_ALL) is a new
option to enable module compression during 'make modules_install'. It
defaults to Y.
* "Support in-kernel module decompression" (MODULE_DECOMPRESS) enables
in-kernel decompression.
Signed-off-by: Petr Pavlu <petr.pavlu@suse.com>
Acked-by: Masahiro Yamada <masahiroy@kernel.org>
Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
Kmod is now (since kmod commit 09c9f8c5df04 ("libkmod: Use kernel
decompression when available")) using the kernel decompressor, when
loading compressed modules.
However, the kernel XZ decompressor is XZ Embedded, which doesn't
handle CRC64 and dictionaries larger than 1MiB.
Use CRC32 and 1MiB dictionary when XZ compressing and installing
kernel modules.
Link: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1050582
Signed-off-by: Martin Nybo Andersen <tweek@tweek.dk>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Running "make modules_install" may fail with
make[2]: execvp: /bin/sh: Argument list too long
if many modules are built and INSTALL_MOD_PATH is long. This is because
scripts/Makefile.modinst creates all directories with one mkdir command.
Use $(foreach ...) instead to prevent an excessive argument list.
Fixes: 2dfec887c0 ("kbuild: reduce the number of mkdir calls during modules_install")
Signed-off-by: Michal Kubecek <mkubecek@suse.cz>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
The modules_sign target is currently only available for in-tree modules,
but it actually works for external modules as well.
Move the modules_sign rule to the common part.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
Commit d890f510c8 ("MODSIGN: Add modules_sign make target") introduced
'make modules_sign' to manually sign modules.
Some time later, commit d9d8d7ed49 ("MODSIGN: Add option to not sign
modules during modules_install") introduced CONFIG_MODULE_SIG_ALL.
If it was disabled, mod_sign_cmd was set to no-op ('true' command).
It affected not only 'make modules_install' but also 'make modules_sign'.
With CONFIG_MODULE_SIG_ALL=n, neither modules_install nor modules_sign
is able to sign modules.
Kbuild has kept that behavior, and nobody has complained about it, but
I think it is weird.
CONFIG_MODULE_SIG_ALL=n should turn off signing only for modules_install.
If users want to sign modules manually, modules_sign should be offered.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
Move more relevant code to scripts/Makefile.modinst.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
Calling 'mkdir' for every module results in redundant syscalls.
Use $(sort ...) to drop the duplicated directories.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
depmod is a part of the module installation.
scripts/Makefile.modinst is a better place to run it.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
Pull modules updates from Luis Chamberlain:
"Nothing exciting at all for modules for v6.3.
The biggest change is just the change of INSTALL_MOD_DIR from "extra"
to "updates" which I found lingered for ages for no good reason while
testing the CXL mock driver [0].
The CXL mock driver has no kconfig integration and requires building
an external module... and re-building the *rest* of the production
drivers. This mock driver when loaded but not the production ones will
crash.
All this can obviously be fixed by integrating kconfig semantics into
such test module, however that's not desirable by the maintainer, and
so sensible defaults must be used to ensure a default "make
modules_install" will suffice for most distros which do not have a
file like /etc/depmod.d/dist.conf with something like `search updates
extra built-in`.
Since most distros rely on kmod and since its inception the "updates"
directory is always in the search path it makes more sense to use that
than the "extra" which only *some* RH based systems rely on.
All this stuff has been on linux-next for a while"
[0] https://lkml.kernel.org/r/20221209062919.1096779-1-mcgrof@kernel.org
* tag 'modules-6.3-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/linux:
Documentation: livepatch: module-elf-format: Remove local klp_modinfo definition
module.h: Document klp_modinfo struct using kdoc
module: Use kstrtobool() instead of strtobool()
kernel/params.c: Use kstrtobool() instead of strtobool()
test_kmod: stop kernel-doc warnings
kbuild: Modify default INSTALL_MOD_DIR from extra to updates
When CONFIG_MODULE_SIG_KEY is PKCS#11 URI (pkcs11:*), signing of modules
fails:
scripts/sign-file sha256 /.../linux/pkcs11:token=foo;object=bar;pin-value=1111 certs/signing_key.x509 /.../kernel/crypto/tcrypt.ko
Usage: scripts/sign-file [-dp] <hash algo> <key> <x509> <module> [<dest>]
scripts/sign-file -s <raw sig> <hash algo> <x509> <module> [<dest>]
First, we need to avoid adding the $(srctree)/ prefix to the URL.
Second, since the kconfig string values no longer include quotes, we need to add
them again when passing a PKCS#11 URI to sign-file. This avoids
splitting by the shell if the URI contains semicolons.
Fixes: 4db9c2e3d0 ("kbuild: stop using config_filename in scripts/Makefile.modsign")
Fixes: 129ab0d2d9 ("kbuild: do not quote string values in include/config/auto.conf")
Signed-off-by: Jan Luebbe <jlu@pengutronix.de>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
The default INSTALL_MOD_DIR of using the /lib/modules/$(uname -r)/extra
directory for external modules assumes distributions will have something
like /etc/depmod.d/dist.conf with:
search updates extra built-in
However, only some Red Hat releases have and use the "extra" stuff for
years now. Meanwhile, the depmod.c tool in kmod has *forever* used
the "updates" directory as part of the search path by default *if*
your distribution does not have any depmod.d configuration.
If you compile and install an external module today, even upstream
kernel mock drivers (tools/testing/cxl) the modules_install target
will pick up the new drivers but will not allow override of drivers
from updates to override built-in ones.
Since module-init-tools was deprecated over 11 years ago and now kmod
has since its inception used the "updates" directory as part of its
default search path to allow overrides, and since the "extra" stuff
was in practice only used by Red Hat stuff, use the more distro
agnostic override path "updates" to allow external modules to
also override proper production kernel modules.
This would allow mocking drivers tools to not have to muck with
depmod.d config files or assume that your distro will have extra
on a configuration file over built-in.
With today's default you end up actually *crashing* Linux when
trying to load cxl_test with the default "extra" [0] directory being
used. This fixes that and allows other mocking drivers to do
less work.
[0] https://lkml.kernel.org/r/20221209062919.1096779-1-mcgrof@kernel.org
Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
Acked-by: Dan Williams <dan.j.williams@intel.com>
Acked-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
scripts/Makefile.build replaces the suffix .o with .ko, then
scripts/Makefile.modpost calls the sed command to change .ko back
to the original .o suffix.
Instead of converting the suffixes back-and-forth, store the .o paths
in modules.order, and replace it with .ko in 'make modules_install'.
This avoids the unneeded sed command.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
Since GNU Make 4.2, $(file ...) supports the read operater '<', which
is useful to read a file without forking a new process. No warning is
shown even if the input file is missing.
For older Make versions, it falls back to the cat command.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
Reviewed-by: Alexander Lobakin <alexandr.lobakin@intel.com>
Tested-by: Alexander Lobakin <alexandr.lobakin@intel.com>
modules.order lists modules in the deterministic order (that is why
"modules order"), and there is no duplication in the list.
$(sort ) is pointless.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
If the directory path given to INSTALL_MOD_PATH contains % or :,
the module_install fails.
% is used in pattern rules, and : as the separator of dependencies.
Bail out with a clearer error message.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <n.schier@avm.de>
Commit 65ce9c3832 ("kbuild: move module strip/compression code into
scripts/Makefile.modinst") added this unused code.
Perhaps, I thought cmd_none was useful for CONFIG_MODULE_COMPRESS_NONE,
but I did not use it after all.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
The previous commit fixed up all shell scripts to not include
include/config/auto.conf.
Now that include/config/auto.conf is only included by Makefiles,
we can change it into a more Make-friendly form.
Previously, Kconfig output string values enclosed with double-quotes
(both in the .config and include/config/auto.conf):
CONFIG_X="foo bar"
Unlike shell, Make handles double-quotes (and single-quotes as well)
verbatim. We must rip them off when used.
There are some patterns:
[1] $(patsubst "%",%,$(CONFIG_X))
[2] $(CONFIG_X:"%"=%)
[3] $(subst ",,$(CONFIG_X))
[4] $(shell echo $(CONFIG_X))
These are not only ugly, but also fragile.
[1] and [2] do not work if the value contains spaces, like
CONFIG_X=" foo bar "
[3] does not work correctly if the value contains double-quotes like
CONFIG_X="foo\"bar"
[4] seems to work better, but has a cost of forking a process.
Anyway, quoted strings were always PITA for our Makefiles.
This commit changes Kconfig to stop quoting in include/config/auto.conf.
These are the string type symbols referenced in Makefiles or scripts:
ACPI_CUSTOM_DSDT_FILE
ARC_BUILTIN_DTB_NAME
ARC_TUNE_MCPU
BUILTIN_DTB_SOURCE
CC_IMPLICIT_FALLTHROUGH
CC_VERSION_TEXT
CFG80211_EXTRA_REGDB_KEYDIR
EXTRA_FIRMWARE
EXTRA_FIRMWARE_DIR
EXTRA_TARGETS
H8300_BUILTIN_DTB
INITRAMFS_SOURCE
LOCALVERSION
MODULE_SIG_HASH
MODULE_SIG_KEY
NDS32_BUILTIN_DTB
NIOS2_DTB_SOURCE
OPENRISC_BUILTIN_DTB
SOC_CANAAN_K210_DTB_SOURCE
SYSTEM_BLACKLIST_HASH_LIST
SYSTEM_REVOCATION_KEYS
SYSTEM_TRUSTED_KEYS
TARGET_CPU
UNUSED_KSYMS_WHITELIST
XILINX_MICROBLAZE0_FAMILY
XILINX_MICROBLAZE0_HW_VER
XTENSA_VARIANT_NAME
I checked them one by one, and fixed up the code where necessary.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Toward the goal of removing the config_filename macro, drop
the double-quotes and add $(srctree)/ prefix in an ad hoc way.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <n.schier@avm.de>
