Kmod is now (since kmod commit 09c9f8c5df04 ("libkmod: Use kernel
decompression when available")) using the kernel decompressor, when
loading compressed modules.
However, the kernel XZ decompressor is XZ Embedded, which doesn't
handle CRC64 and dictionaries larger than 1MiB.
Use CRC32 and 1MiB dictionary when XZ compressing and installing
kernel modules.
Link: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1050582
Signed-off-by: Martin Nybo Andersen <tweek@tweek.dk>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Running "make modules_install" may fail with
make[2]: execvp: /bin/sh: Argument list too long
if many modules are built and INSTALL_MOD_PATH is long. This is because
scripts/Makefile.modinst creates all directories with one mkdir command.
Use $(foreach ...) instead to prevent an excessive argument list.
Fixes: 2dfec887c0 ("kbuild: reduce the number of mkdir calls during modules_install")
Signed-off-by: Michal Kubecek <mkubecek@suse.cz>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
The modules_sign target is currently only available for in-tree modules,
but it actually works for external modules as well.
Move the modules_sign rule to the common part.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
Commit d890f510c8 ("MODSIGN: Add modules_sign make target") introduced
'make modules_sign' to manually sign modules.
Some time later, commit d9d8d7ed49 ("MODSIGN: Add option to not sign
modules during modules_install") introduced CONFIG_MODULE_SIG_ALL.
If it was disabled, mod_sign_cmd was set to no-op ('true' command).
It affected not only 'make modules_install' but also 'make modules_sign'.
With CONFIG_MODULE_SIG_ALL=n, neither modules_install nor modules_sign
is able to sign modules.
Kbuild has kept that behavior, and nobody has complained about it, but
I think it is weird.
CONFIG_MODULE_SIG_ALL=n should turn off signing only for modules_install.
If users want to sign modules manually, modules_sign should be offered.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
Move more relevant code to scripts/Makefile.modinst.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
Calling 'mkdir' for every module results in redundant syscalls.
Use $(sort ...) to drop the duplicated directories.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
depmod is a part of the module installation.
scripts/Makefile.modinst is a better place to run it.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
Pull modules updates from Luis Chamberlain:
"Nothing exciting at all for modules for v6.3.
The biggest change is just the change of INSTALL_MOD_DIR from "extra"
to "updates" which I found lingered for ages for no good reason while
testing the CXL mock driver [0].
The CXL mock driver has no kconfig integration and requires building
an external module... and re-building the *rest* of the production
drivers. This mock driver when loaded but not the production ones will
crash.
All this can obviously be fixed by integrating kconfig semantics into
such test module, however that's not desirable by the maintainer, and
so sensible defaults must be used to ensure a default "make
modules_install" will suffice for most distros which do not have a
file like /etc/depmod.d/dist.conf with something like `search updates
extra built-in`.
Since most distros rely on kmod and since its inception the "updates"
directory is always in the search path it makes more sense to use that
than the "extra" which only *some* RH based systems rely on.
All this stuff has been on linux-next for a while"
[0] https://lkml.kernel.org/r/20221209062919.1096779-1-mcgrof@kernel.org
* tag 'modules-6.3-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/linux:
Documentation: livepatch: module-elf-format: Remove local klp_modinfo definition
module.h: Document klp_modinfo struct using kdoc
module: Use kstrtobool() instead of strtobool()
kernel/params.c: Use kstrtobool() instead of strtobool()
test_kmod: stop kernel-doc warnings
kbuild: Modify default INSTALL_MOD_DIR from extra to updates
When CONFIG_MODULE_SIG_KEY is PKCS#11 URI (pkcs11:*), signing of modules
fails:
scripts/sign-file sha256 /.../linux/pkcs11:token=foo;object=bar;pin-value=1111 certs/signing_key.x509 /.../kernel/crypto/tcrypt.ko
Usage: scripts/sign-file [-dp] <hash algo> <key> <x509> <module> [<dest>]
scripts/sign-file -s <raw sig> <hash algo> <x509> <module> [<dest>]
First, we need to avoid adding the $(srctree)/ prefix to the URL.
Second, since the kconfig string values no longer include quotes, we need to add
them again when passing a PKCS#11 URI to sign-file. This avoids
splitting by the shell if the URI contains semicolons.
Fixes: 4db9c2e3d0 ("kbuild: stop using config_filename in scripts/Makefile.modsign")
Fixes: 129ab0d2d9 ("kbuild: do not quote string values in include/config/auto.conf")
Signed-off-by: Jan Luebbe <jlu@pengutronix.de>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
The default INSTALL_MOD_DIR of using the /lib/modules/$(uname -r)/extra
directory for external modules assumes distributions will have something
like /etc/depmod.d/dist.conf with:
search updates extra built-in
However, only some Red Hat releases have and use the "extra" stuff for
years now. Meanwhile, the depmod.c tool in kmod has *forever* used
the "updates" directory as part of the search path by default *if*
your distribution does not have any depmod.d configuration.
If you compile and install an external module today, even upstream
kernel mock drivers (tools/testing/cxl) the modules_install target
will pick up the new drivers but will not allow override of drivers
from updates to override built-in ones.
Since module-init-tools was deprecated over 11 years ago and now kmod
has since its inception used the "updates" directory as part of its
default search path to allow overrides, and since the "extra" stuff
was in practice only used by Red Hat stuff, use the more distro
agnostic override path "updates" to allow external modules to
also override proper production kernel modules.
This would allow mocking drivers tools to not have to muck with
depmod.d config files or assume that your distro will have extra
on a configuration file over built-in.
With today's default you end up actually *crashing* Linux when
trying to load cxl_test with the default "extra" [0] directory being
used. This fixes that and allows other mocking drivers to do
less work.
[0] https://lkml.kernel.org/r/20221209062919.1096779-1-mcgrof@kernel.org
Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
Acked-by: Dan Williams <dan.j.williams@intel.com>
Acked-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
scripts/Makefile.build replaces the suffix .o with .ko, then
scripts/Makefile.modpost calls the sed command to change .ko back
to the original .o suffix.
Instead of converting the suffixes back-and-forth, store the .o paths
in modules.order, and replace it with .ko in 'make modules_install'.
This avoids the unneeded sed command.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
Since GNU Make 4.2, $(file ...) supports the read operater '<', which
is useful to read a file without forking a new process. No warning is
shown even if the input file is missing.
For older Make versions, it falls back to the cat command.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
Reviewed-by: Alexander Lobakin <alexandr.lobakin@intel.com>
Tested-by: Alexander Lobakin <alexandr.lobakin@intel.com>
modules.order lists modules in the deterministic order (that is why
"modules order"), and there is no duplication in the list.
$(sort ) is pointless.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
If the directory path given to INSTALL_MOD_PATH contains % or :,
the module_install fails.
% is used in pattern rules, and : as the separator of dependencies.
Bail out with a clearer error message.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <n.schier@avm.de>
Commit 65ce9c3832 ("kbuild: move module strip/compression code into
scripts/Makefile.modinst") added this unused code.
Perhaps, I thought cmd_none was useful for CONFIG_MODULE_COMPRESS_NONE,
but I did not use it after all.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
The previous commit fixed up all shell scripts to not include
include/config/auto.conf.
Now that include/config/auto.conf is only included by Makefiles,
we can change it into a more Make-friendly form.
Previously, Kconfig output string values enclosed with double-quotes
(both in the .config and include/config/auto.conf):
CONFIG_X="foo bar"
Unlike shell, Make handles double-quotes (and single-quotes as well)
verbatim. We must rip them off when used.
There are some patterns:
[1] $(patsubst "%",%,$(CONFIG_X))
[2] $(CONFIG_X:"%"=%)
[3] $(subst ",,$(CONFIG_X))
[4] $(shell echo $(CONFIG_X))
These are not only ugly, but also fragile.
[1] and [2] do not work if the value contains spaces, like
CONFIG_X=" foo bar "
[3] does not work correctly if the value contains double-quotes like
CONFIG_X="foo\"bar"
[4] seems to work better, but has a cost of forking a process.
Anyway, quoted strings were always PITA for our Makefiles.
This commit changes Kconfig to stop quoting in include/config/auto.conf.
These are the string type symbols referenced in Makefiles or scripts:
ACPI_CUSTOM_DSDT_FILE
ARC_BUILTIN_DTB_NAME
ARC_TUNE_MCPU
BUILTIN_DTB_SOURCE
CC_IMPLICIT_FALLTHROUGH
CC_VERSION_TEXT
CFG80211_EXTRA_REGDB_KEYDIR
EXTRA_FIRMWARE
EXTRA_FIRMWARE_DIR
EXTRA_TARGETS
H8300_BUILTIN_DTB
INITRAMFS_SOURCE
LOCALVERSION
MODULE_SIG_HASH
MODULE_SIG_KEY
NDS32_BUILTIN_DTB
NIOS2_DTB_SOURCE
OPENRISC_BUILTIN_DTB
SOC_CANAAN_K210_DTB_SOURCE
SYSTEM_BLACKLIST_HASH_LIST
SYSTEM_REVOCATION_KEYS
SYSTEM_TRUSTED_KEYS
TARGET_CPU
UNUSED_KSYMS_WHITELIST
XILINX_MICROBLAZE0_FAMILY
XILINX_MICROBLAZE0_HW_VER
XTENSA_VARIANT_NAME
I checked them one by one, and fixed up the code where necessary.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Toward the goal of removing the config_filename macro, drop
the double-quotes and add $(srctree)/ prefix in an ad hoc way.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <n.schier@avm.de>
scripts/Makefile.modsign is a subset of scripts/Makefile.modinst,
and duplicates the code. Let's merge them.
By the way, you do not need to run 'make modules_sign' explicitly
because modules are signed as a part of 'make modules_install' when
CONFIG_MODULE_SIG_ALL=y. If CONFIG_MODULE_SIG_ALL=n, mod_sign_cmd is
set to 'true', so 'make modules_sign' is not functional.
In my understanding, the reason of still keeping this is to handle
corner cases like commit 64178cb62c ("builddeb: fix stripped module
signatures if CONFIG_DEBUG_INFO and CONFIG_MODULE_SIG_ALL are set").
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Both mod_strip_cmd and mod_compress_cmd are only used in
scripts/Makefile.modinst, hence there is no good reason to define them
in the top Makefile. Move the relevant code to scripts/Makefile.modinst.
Also, show separate log messages for each of install, strip, sign, and
compress.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
scripts/Makefile.modinst is ugly and weird in multiple ways; it
specifies real files $(modules) as phony, makes directory manipulation
needlessly too complicated.
Clean up the Makefile code, and show the full path of installed modules
in the log.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
VPATH is used in Kbuild to make pattern rules search for prerequisites
in both $(objtree) and $(srctree). Some of *.c, *.S files are not real
sources, but generated by tools such as flex, bison, perl.
In contrast, I doubt the benefit of --include-dir=$(abs_srctree) because
it is always clear which Makefiles are real sources, and which are not.
So, my hope is to add $(srctree)/ prefix to all check-in Makefiles,
then remove --include-dir=$(abs_srctree) flag in the future.
I am touching only some Kbuild core parts for now. Treewide fixes will
be needed to achieve this goal.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Towards the goal of removing MODVERDIR, read out modules.order to get
the list of modules to be installed. This is simpler than parsing *.mod
files in $(MODVERDIR).
For external modules, $(KBUILD_EXTMOD)/modules.order should be read.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
The comment is the same as in the top-level Makefile.
Also, the comments contain typos:
- the .PHONY variable -> the PHONY variable
- se we can ... -> so we can ...
Instead of fixing the typos, just remove the duplicated comments.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
