Pull Rust updates from Miguel Ojeda:
"Toolchain and infrastructure:
- Support 'MITIGATION_{RETHUNK,RETPOLINE,SLS}' (which cleans up
objtool warnings), teach objtool about 'noreturn' Rust symbols and
mimic '___ADDRESSABLE()' for 'module_{init,exit}'. With that, we
should be objtool-warning-free, so enable it to run for all Rust
object files.
- KASAN (no 'SW_TAGS'), KCFI and shadow call sanitizer support.
- Support 'RUSTC_VERSION', including re-config and re-build on
change.
- Split helpers file into several files in a folder, to avoid
conflicts in it. Eventually those files will be moved to the right
places with the new build system. In addition, remove the need to
manually export the symbols defined there, reusing existing
machinery for that.
- Relax restriction on configurations with Rust + GCC plugins to just
the RANDSTRUCT plugin.
'kernel' crate:
- New 'list' module: doubly-linked linked list for use with reference
counted values, which is heavily used by the upcoming Rust Binder.
This includes 'ListArc' (a wrapper around 'Arc' that is guaranteed
unique for the given ID), 'AtomicTracker' (tracks whether a
'ListArc' exists using an atomic), 'ListLinks' (the prev/next
pointers for an item in a linked list), 'List' (the linked list
itself), 'Iter' (an iterator over a 'List'), 'Cursor' (a cursor
into a 'List' that allows to remove elements), 'ListArcField' (a
field exclusively owned by a 'ListArc'), as well as support for
heterogeneous lists.
- New 'rbtree' module: red-black tree abstractions used by the
upcoming Rust Binder.
This includes 'RBTree' (the red-black tree itself), 'RBTreeNode' (a
node), 'RBTreeNodeReservation' (a memory reservation for a node),
'Iter' and 'IterMut' (immutable and mutable iterators), 'Cursor'
(bidirectional cursor that allows to remove elements), as well as
an entry API similar to the Rust standard library one.
- 'init' module: add 'write_[pin_]init' methods and the
'InPlaceWrite' trait. Add the 'assert_pinned!' macro.
- 'sync' module: implement the 'InPlaceInit' trait for 'Arc' by
introducing an associated type in the trait.
- 'alloc' module: add 'drop_contents' method to 'BoxExt'.
- 'types' module: implement the 'ForeignOwnable' trait for
'Pin<Box<T>>' and improve the trait's documentation. In addition,
add the 'into_raw' method to the 'ARef' type.
- 'error' module: in preparation for the upcoming Rust support for
32-bit architectures, like arm, locally allow Clippy lint for
those.
Documentation:
- https://rust.docs.kernel.org has been announced, so link to it.
- Enable rustdoc's "jump to definition" feature, making its output a
bit closer to the experience in a cross-referencer.
- Debian Testing now also provides recent Rust releases (outside of
the freeze period), so add it to the list.
MAINTAINERS:
- Trevor is joining as reviewer of the "RUST" entry.
And a few other small bits"
* tag 'rust-6.12' of https://github.com/Rust-for-Linux/linux: (54 commits)
kasan: rust: Add KASAN smoke test via UAF
kbuild: rust: Enable KASAN support
rust: kasan: Rust does not support KHWASAN
kbuild: rust: Define probing macros for rustc
kasan: simplify and clarify Makefile
rust: cfi: add support for CFI_CLANG with Rust
cfi: add CONFIG_CFI_ICALL_NORMALIZE_INTEGERS
rust: support for shadow call stack sanitizer
docs: rust: include other expressions in conditional compilation section
kbuild: rust: replace proc macros dependency on `core.o` with the version text
kbuild: rust: rebuild if the version text changes
kbuild: rust: re-run Kconfig if the version text changes
kbuild: rust: add `CONFIG_RUSTC_VERSION`
rust: avoid `box_uninit_write` feature
MAINTAINERS: add Trevor Gross as Rust reviewer
rust: rbtree: add `RBTree::entry`
rust: rbtree: add cursor
rust: rbtree: add mutable iterator
rust: rbtree: add iterator
rust: rbtree: add red-black tree implementation backed by the C version
...
scripts/Makefile.lib is included not only from scripts/Makefile.build
but also from scripts/Makefile.{modfinal,package,vmlinux,vmlinux_o},
where DT build rules are not required.
Split the DT build rules out to scripts/Makefile.dtbs, and include it
only when necessary.
While I was here, I added $(DT_TMP_SCHEMA) as a prerequisite of
$(multi-dtb-y).
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Rob Herring (Arm) <robh@kernel.org>
Flex and Bison are used only for host programs. Move their intermediate
target processing from scripts/Makefile.build to scripts/Makefile.host.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
This typo in scripts/Makefile.build has been present for more than 20
years. It was accidentally copy-pasted to other scripts/Makefile.* files.
Fix them all.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nathan Chancellor <nathan@kernel.org>
Pull Kbuild updates from Masahiro Yamada:
- Avoid 'constexpr', which is a keyword in C23
- Allow 'dtbs_check' and 'dt_compatible_check' run independently of
'dt_binding_check'
- Fix weak references to avoid GOT entries in position-independent code
generation
- Convert the last use of 'optional' property in arch/sh/Kconfig
- Remove support for the 'optional' property in Kconfig
- Remove support for Clang's ThinLTO caching, which does not work with
the .incbin directive
- Change the semantics of $(src) so it always points to the source
directory, which fixes Makefile inconsistencies between upstream and
downstream
- Fix 'make tar-pkg' for RISC-V to produce a consistent package
- Provide reasonable default coverage for objtool, sanitizers, and
profilers
- Remove redundant OBJECT_FILES_NON_STANDARD, KASAN_SANITIZE, etc.
- Remove the last use of tristate choice in drivers/rapidio/Kconfig
- Various cleanups and fixes in Kconfig
* tag 'kbuild-v6.10' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: (46 commits)
kconfig: use sym_get_choice_menu() in sym_check_prop()
rapidio: remove choice for enumeration
kconfig: lxdialog: remove initialization with A_NORMAL
kconfig: m/nconf: merge two item_add_str() calls
kconfig: m/nconf: remove dead code to display value of bool choice
kconfig: m/nconf: remove dead code to display children of choice members
kconfig: gconf: show checkbox for choice correctly
kbuild: use GCOV_PROFILE and KCSAN_SANITIZE in scripts/Makefile.modfinal
Makefile: remove redundant tool coverage variables
kbuild: provide reasonable defaults for tool coverage
modules: Drop the .export_symbol section from the final modules
kconfig: use menu_list_for_each_sym() in sym_check_choice_deps()
kconfig: use sym_get_choice_menu() in conf_write_defconfig()
kconfig: add sym_get_choice_menu() helper
kconfig: turn defaults and additional prompt for choice members into error
kconfig: turn missing prompt for choice members into error
kconfig: turn conf_choice() into void function
kconfig: use linked list in sym_set_changed()
kconfig: gconf: use MENU_CHANGED instead of SYMBOL_CHANGED
kconfig: gconf: remove debug code
...
The objtool, sanitizers (KASAN, UBSAN, etc.), and profilers (GCOV, etc.)
are intended only for kernel space objects.
For instance, the following are not kernel objects, and therefore should
opt out of coverage:
- vDSO
- purgatory
- bootloader (arch/*/boot/)
However, to exclude these from coverage, you need to explicitly set
OBJECT_FILES_NON_STNDARD=y, KASAN_SANITIZE=n, etc.
Kbuild can achieve this without relying on such variables because
objects not directly linked to vmlinux or modules are considered
"non-standard objects".
Detecting standard objects is straightforward:
- objects added to obj-y or lib-y are linked to vmlinux
- objects added to obj-m are linked to modules
There are some exceptional Makefiles (e.g., arch/s390/boot/Makefile,
arch/xtensa/boot/lib/Makefile) that use obj-y or lib-y for non-kernel
space objects, but they can be fixed later if necessary.
Going forward, objects that are not listed in obj-y, lib-y, or obj-m
will opt out of objtool, sanitizers, and profilers by default.
You can still override the Kbuild decision by explicitly specifying
OBJECT_FILES_NON_STANDARD, KASAN_SANITIZE, etc. but most of such Make
variables can be removed.
The next commit will clean up redundant variables.
Note:
This commit changes the coverage for some objects:
- exclude .vmlinux.export.o from UBSAN, KCOV
- exclude arch/csky/kernel/vdso/vgettimeofday.o from UBSAN
- exclude arch/parisc/kernel/vdso32/vdso32.so from UBSAN
- exclude arch/parisc/kernel/vdso64/vdso64.so from UBSAN
- exclude arch/x86/um/vdso/um_vdso.o from UBSAN
- exclude drivers/misc/lkdtm/rodata.o from UBSAN, KCOV
- exclude init/version-timestamp.o from UBSAN, KCOV
- exclude lib/test_fortify/*.o from all santizers and profilers
I believe these are positive effects.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Kees Cook <keescook@chromium.org>
Tested-by: Roberto Sassu <roberto.sassu@huawei.com>
Pull Rust updates from Miguel Ojeda:
"The most notable change is the drop of the 'alloc' in-tree fork. This
is nicely reflected in the diffstat as a ~10k lines drop. In turn,
this makes the version upgrades way simpler and smaller in the future,
e.g. the latest one in commit 56f64b3706 ("rust: upgrade to Rust
1.78.0").
More importantly, this increases the chances that a newer compiler
version just works, which in turn means supporting several compiler
versions is easier now. Thus we will look into finally setting a
minimum version in the near future.
Toolchain and infrastructure:
- Upgrade to Rust 1.78.0
This time around, due to how the kernel and Rust schedules have
aligned, there are two upgrades in fact. These allow us to remove
one more unstable feature ('offset_of') from the list, among other
improvements
- Drop 'alloc' in-tree fork of the standard library crate, which
means all the unstable features used by 'alloc' (~30 language ones,
~60 library ones) are not a concern anymore
- Support DWARFv5 via the '-Zdwarf-version' flag
- Support zlib and zstd debuginfo compression via the
'-Zdebuginfo-compression' flag
'kernel' crate:
- Support allocation flags ('GFP_*'), particularly in 'Box' (via
'BoxExt'), 'Vec' (via 'VecExt'), 'Arc' and 'UniqueArc', as well as
in the 'init' module APIs
- Remove usage of the 'allocator_api' unstable feature
- Remove 'try_' prefix in allocation APIs' names
- Add 'VecExt' (an extension trait) to be able to drop the 'alloc'
fork
- Add the '{make,to}_{upper,lower}case()' methods to 'CStr'/'CString'
- Add the 'as_ptr' method to 'ThisModule'
- Add the 'from_raw' method to 'ArcBorrow'
- Add the 'into_unique_or_drop' method to 'Arc'
- Display column number in the 'dbg!' macro output by applying the
equivalent change done to the standard library one
- Migrate 'Work' to '#[pin_data]' thanks to the changes in the
'macros' crate, which allows to remove an unsafe call in its 'new'
associated function
- Prevent namespacing issues when using the '[try_][pin_]init!'
macros by changing the generated name of guard variables
- Make the 'get' method in 'Opaque' const
- Implement the 'Default' trait for 'LockClassKey'
- Remove unneeded 'kernel::prelude' imports from doctests
- Remove redundant imports
'macros' crate:
- Add 'decl_generics' to 'parse_generics()' to support default
values, and use that to allow them in '#[pin_data]'
Helpers:
- Trivial English grammar fix
Documentation:
- Add section on Rust Kselftests to the 'Testing' document
- Expand the 'Abstractions vs. bindings' section of the 'General
Information' document"
* tag 'rust-6.10' of https://github.com/Rust-for-Linux/linux: (31 commits)
rust: alloc: fix dangling pointer in VecExt<T>::reserve()
rust: upgrade to Rust 1.78.0
rust: kernel: remove redundant imports
rust: sync: implement `Default` for `LockClassKey`
docs: rust: extend abstraction and binding documentation
docs: rust: Add instructions for the Rust kselftest
rust: remove unneeded `kernel::prelude` imports from doctests
rust: update `dbg!()` to format column number
rust: helpers: Fix grammar in comment
rust: init: change the generated name of guard variables
rust: sync: add `Arc::into_unique_or_drop`
rust: sync: add `ArcBorrow::from_raw`
rust: types: Make Opaque::get const
rust: kernel: remove usage of `allocator_api` unstable feature
rust: init: update `init` module to take allocation flags
rust: sync: update `Arc` and `UniqueArc` to take allocation flags
rust: alloc: update `VecExt` to take allocation flags
rust: alloc: introduce the `BoxExt` trait
rust: alloc: introduce allocation flags
rust: alloc: remove our fork of the `alloc` crate
...
Currently, Kbuild produces inconsistent results in some cases.
You can do an interesting experiment using the --shuffle option, which
is supported by GNU Make 4.4 or later.
Set CONFIG_KVM_INTEL=y and CONFIG_KVM_AMD=m (or vice versa), and repeat
incremental builds w/wo --shuffle=reverse.
$ make
[ snip ]
CC arch/x86/kvm/kvm-asm-offsets.s
$ make --shuffle=reverse
[ snip ]
CC [M] arch/x86/kvm/kvm-asm-offsets.s
$ make
[ snip ]
CC arch/x86/kvm/kvm-asm-offsets.s
arch/x86/kvm/kvm-asm-offsets.s is rebuilt every time w/wo the [M] marker.
arch/x86/kvm/kvm-asm-offsets.s is built as built-in when it is built as
a prerequisite of arch/x86/kvm/kvm-intel.o, which is built-in.
arch/x86/kvm/kvm-asm-offsets.s is built as modular when it is built as
a prerequisite of arch/x86/kvm/kvm-amd.o, which is a module.
Another odd example is single target builds.
When CONFIG_LKDTM=m, drivers/misc/lkdtm/rodata.o can be built as
built-in or modular, depending on how it is built.
$ make drivers/misc/lkdtm/lkdtm.o
[ snip ]
CC [M] drivers/misc/lkdtm/rodata.o
$ make drivers/misc/lkdtm/rodata.o
[ snip ]
CC drivers/misc/lkdtm/rodata.o
drivers/misc/lkdtm/rodata.o is built as modular when it is built as a
prerequisite of another, but built as built-in when it is a final
target.
The same thing happens to drivers/memory/emif-asm-offsets.s when
CONFIG_TI_EMIF_SRAM=m.
$ make drivers/memory/ti-emif-sram.o
[ snip ]
CC [M] drivers/memory/emif-asm-offsets.s
$ make drivers/memory/emif-asm-offsets.s
[ snip ]
CC drivers/memory/emif-asm-offsets.s
This is because the part-of-module=y flag defined for the modules is
inherited by its prerequisites.
Target-specific variables are likely intended only for local use.
This commit adds 'private' to them.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <n.schier@avm.de>
Kbuild conventionally uses $(obj)/ for generated files, and $(src)/ for
checked-in source files. It is merely a convention without any functional
difference. In fact, $(obj) and $(src) are exactly the same, as defined
in scripts/Makefile.build:
src := $(obj)
When the kernel is built in a separate output directory, $(src) does
not accurately reflect the source directory location. While Kbuild
resolves this discrepancy by specifying VPATH=$(srctree) to search for
source files, it does not cover all cases. For example, when adding a
header search path for local headers, -I$(srctree)/$(src) is typically
passed to the compiler.
This introduces inconsistency between upstream and downstream Makefiles
because $(src) is used instead of $(srctree)/$(src) for the latter.
To address this inconsistency, this commit changes the semantics of
$(src) so that it always points to the directory in the source tree.
Going forward, the variables used in Makefiles will have the following
meanings:
$(obj) - directory in the object tree
$(src) - directory in the source tree (changed by this commit)
$(objtree) - the top of the kernel object tree
$(srctree) - the top of the kernel source tree
Consequently, $(srctree)/$(src) in upstream Makefiles need to be replaced
with $(src).
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
Kbuild conventionally uses $(obj)/ for generated files, and $(src)/ for
checked-in source files. It is merely a convention without any functional
difference. In fact, $(obj) and $(src) are exactly the same, as defined
in scripts/Makefile.build:
src := $(obj)
Before changing the semantics of $(src) in the next commit, this commit
replaces $(obj)/ with $(src)/ in pattern rules where the prerequisite
might be a generated file.
C, assembly, Rust, and DTS files are sometimes generated by tools, so
they could be either generated files or real sources. The $(obj)/ prefix
works for both cases with the help of VPATH.
As mentioned above, $(obj) and $(src) are the same at this point, hence
this commit has no functional change.
I did not modify scripts/Makefile.userprogs because there is no use
case where userspace C files are generated.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
If one attempts to build an essentially empty file somewhere in the
kernel tree, it leads to a build error because the compiler does not
recognize the `new_uninit` unstable feature:
error[E0635]: unknown feature `new_uninit`
--> <crate attribute>:1:9
|
1 | feature(new_uninit)
| ^^^^^^^^^^
The reason is that we pass `-Zcrate-attr='feature(new_uninit)'` (together
with `-Zallow-features=new_uninit`) to let non-`rust/` code use that
unstable feature.
However, the compiler only recognizes the feature if the `alloc` crate
is resolved (the feature is an `alloc` one). `--extern alloc`, which we
pass, is not enough to resolve the crate.
Introducing a reference like `use alloc;` or `extern crate alloc;`
solves the issue, thus this is not seen in normal files. For instance,
`use`ing the `kernel` prelude introduces such a reference, since `alloc`
is used inside.
While normal use of the build system is not impacted by this, it can still
be fairly confusing for kernel developers [1], thus use the unstable
`force` option of `--extern` [2] (added in Rust 1.71 [3]) to force the
compiler to resolve `alloc`.
This new unstable feature is only needed meanwhile we use the other
unstable feature, since then we will not need `-Zcrate-attr`.
Cc: stable@vger.kernel.org # v6.6+
Reported-by: Daniel Almeida <daniel.almeida@collabora.com>
Reported-by: Julian Stecklina <julian.stecklina@cyberus-technology.de>
Closes: https://rust-for-linux.zulipchat.com/#narrow/stream/288089-General/topic/x/near/424096982 [1]
Fixes: 2f7ab1267d ("Kbuild: add Rust support")
Link: https://github.com/rust-lang/rust/issues/111302 [2]
Link: https://github.com/rust-lang/rust/pull/109421 [3]
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Reviewed-by: Gary Guo <gary@garyguo.net>
Link: https://lore.kernel.org/r/20240422090644.525520-1-ojeda@kernel.org
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
This is the next upgrade to the Rust toolchain, from 1.76.0 to 1.77.1
(i.e. the latest) [1].
See the upgrade policy [2] and the comments on the first upgrade in
commit 3ed03f4da0 ("rust: upgrade to Rust 1.68.2").
# Unstable features
The `offset_of` feature (single-field `offset_of!`) that we were using
got stabilized in Rust 1.77.0 [3].
Therefore, now the only unstable features allowed to be used outside the
`kernel` crate is `new_uninit`, though other code to be upstreamed may
increase the list.
Please see [4] for details.
# Required changes
Rust 1.77.0 merged the `unused_tuple_struct_fields` lint into `dead_code`,
thus upgrading it from `allow` to `warn` [5]. In turn, this made `rustc`
complain about the `ThisModule`'s pointer field being never read, but
the previous patch adds the `as_ptr` method to it, needed by Binder [6],
so that we do not need to locally `allow` it.
# Other changes
Rust 1.77.0 introduces the `--check-cfg` feature [7], for which there
is a Call for Testing going on [8]. We were requested to test it and
we found it useful [9] -- we will likely enable it in the future.
# `alloc` upgrade and reviewing
The vast majority of changes are due to our `alloc` fork being upgraded
at once.
There are two kinds of changes to be aware of: the ones coming from
upstream, which we should follow as closely as possible, and the updates
needed in our added fallible APIs to keep them matching the newer
infallible APIs coming from upstream.
Instead of taking a look at the diff of this patch, an alternative
approach is reviewing a diff of the changes between upstream `alloc` and
the kernel's. This allows to easily inspect the kernel additions only,
especially to check if the fallible methods we already have still match
the infallible ones in the new version coming from upstream.
Another approach is reviewing the changes introduced in the additions in
the kernel fork between the two versions. This is useful to spot
potentially unintended changes to our additions.
To apply these approaches, one may follow steps similar to the following
to generate a pair of patches that show the differences between upstream
Rust and the kernel (for the subset of `alloc` we use) before and after
applying this patch:
# Get the difference with respect to the old version.
git -C rust checkout $(linux/scripts/min-tool-version.sh rustc)
git -C linux ls-tree -r --name-only HEAD -- rust/alloc |
cut -d/ -f3- |
grep -Fv README.md |
xargs -IPATH cp rust/library/alloc/src/PATH linux/rust/alloc/PATH
git -C linux diff --patch-with-stat --summary -R > old.patch
git -C linux restore rust/alloc
# Apply this patch.
git -C linux am rust-upgrade.patch
# Get the difference with respect to the new version.
git -C rust checkout $(linux/scripts/min-tool-version.sh rustc)
git -C linux ls-tree -r --name-only HEAD -- rust/alloc |
cut -d/ -f3- |
grep -Fv README.md |
xargs -IPATH cp rust/library/alloc/src/PATH linux/rust/alloc/PATH
git -C linux diff --patch-with-stat --summary -R > new.patch
git -C linux restore rust/alloc
Now one may check the `new.patch` to take a look at the additions (first
approach) or at the difference between those two patches (second
approach). For the latter, a side-by-side tool is recommended.
Link: https://github.com/rust-lang/rust/blob/stable/RELEASES.md#version-1770-2024-03-21 [1]
Link: https://rust-for-linux.com/rust-version-policy [2]
Link: https://github.com/rust-lang/rust/pull/118799 [3]
Link: https://github.com/Rust-for-Linux/linux/issues/2 [4]
Link: https://github.com/rust-lang/rust/pull/118297 [5]
Link: https://lore.kernel.org/rust-for-linux/20231101-rust-binder-v1-2-08ba9197f637@google.com/#Z31rust:kernel:lib.rs [6]
Link: https://doc.rust-lang.org/nightly/unstable-book/compiler-flags/check-cfg.html [7]
Link: https://github.com/rust-lang/rfcs/pull/3013#issuecomment-1936648479 [8]
Link: https://github.com/rust-lang/rust/issues/82450#issuecomment-1947462977 [9]
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Tested-by: Boqun Feng <boqun.feng@gmail.com>
Link: https://lore.kernel.org/r/20240217002717.57507-1-ojeda@kernel.org
[ Upgraded to 1.77.1. Removed `allow(dead_code)` thanks to the previous
patch. Reworded accordingly. No changes to `alloc` during the beta. ]
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
Pull Kbuild updates from Masahiro Yamada:
- Generate a list of built DTB files (arch/*/boot/dts/dtbs-list)
- Use more threads when building Debian packages in parallel
- Fix warnings shown during the RPM kernel package uninstallation
- Change OBJECT_FILES_NON_STANDARD_*.o etc. to take a relative path to
Makefile
- Support GCC's -fmin-function-alignment flag
- Fix a null pointer dereference bug in modpost
- Add the DTB support to the RPM package
- Various fixes and cleanups in Kconfig
* tag 'kbuild-v6.9' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: (67 commits)
kconfig: tests: test dependency after shuffling choices
kconfig: tests: add a test for randconfig with dependent choices
kconfig: tests: support KCONFIG_SEED for the randconfig runner
kbuild: rpm-pkg: add dtb files in kernel rpm
kconfig: remove unneeded menu_is_visible() call in conf_write_defconfig()
kconfig: check prompt for choice while parsing
kconfig: lxdialog: remove unused dialog colors
kconfig: lxdialog: fix button color for blackbg theme
modpost: fix null pointer dereference
kbuild: remove GCC's default -Wpacked-bitfield-compat flag
kbuild: unexport abs_srctree and abs_objtree
kbuild: Move -Wenum-{compare-conditional,enum-conversion} into W=1
kconfig: remove named choice support
kconfig: use linked list in get_symbol_str() to iterate over menus
kconfig: link menus to a symbol
kbuild: fix inconsistent indentation in top Makefile
kbuild: Use -fmin-function-alignment when available
alpha: merge two entries for CONFIG_ALPHA_GAMMA
alpha: merge two entries for CONFIG_ALPHA_EV4
kbuild: change DTC_FLAGS_<basetarget>.o to take the path relative to $(obj)
...
`rustc` (like Cargo) may take advantage of the jobserver at any time
(e.g. for backend parallelism, or eventually frontend too). In the kernel,
we call `rustc` with `-Ccodegen-units=1` (and `-Zthreads` is 1 so far),
so we do not expect parallelism. However, in the upcoming Rust 1.76.0, a
warning is emitted by `rustc` [1] when it cannot connect to the jobserver
it was passed (in many cases, but not all: compiling and `--print sysroot`
do, but `--version` does not). And given GNU Make always passes
the jobserver in the environment variable (even when a line is deemed
non-recursive), `rustc` will end up complaining about it (in particular
in Make 4.3 where there is only the simple pipe jobserver style).
One solution is to remove the jobserver from `MAKEFLAGS`. However, we
can mark the lines with calls to `rustc` (and Cargo) as recursive, which
looks simpler. This is being documented as a recommendation in `rustc`
[2] and allows us to be ready for the time we may use parallelism inside
`rustc` (potentially now, if a user passes `-Zthreads`). Thus do so.
Similarly, do the same for `rustdoc` and `cargo` calls.
Finally, there is one case that the solution does not cover, which is the
`$(shell ...)` call we have. Thus, for that one, set an empty `MAKEFLAGS`
environment variable.
Link: https://github.com/rust-lang/rust/issues/120515 [1]
Acked-by: Masahiro Yamada <masahiroy@kernel.org>
Link: https://github.com/rust-lang/rust/pull/121564 [2]
Link: https://lore.kernel.org/r/20240217002638.57373-1-ojeda@kernel.org
[ Reworded to add link to PR documenting the recommendation. ]
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
Commit 54b8ae66ae ("kbuild: change *FLAGS_<basetarget>.o to take the
path relative to $(obj)") changed the syntax of per-file compiler flags.
The situation is the same for the following variables:
OBJECT_FILES_NON_STANDARD_<basetarget>.o
GCOV_PROFILE_<basetarget>.o
KASAN_SANITIZE_<basetarget>.o
KMSAN_SANITIZE_<basetarget>.o
KMSAN_ENABLE_CHECKS_<basetarget>.o
UBSAN_SANITIZE_<basetarget>.o
KCOV_INSTRUMENT_<basetarget>.o
KCSAN_SANITIZE_<basetarget>.o
KCSAN_INSTRUMENT_BARRIERS_<basetarget>.o
The <basetarget> is the filename of the target with its directory and
suffix stripped.
This syntax comes into a trouble when two files with the same basename
appear in one Makefile, for example:
obj-y += dir1/foo.o
obj-y += dir2/foo.o
OBJECT_FILES_NON_STANDARD_foo.o := y
OBJECT_FILES_NON_STANDARD_foo.o is applied to both dir1/foo.o and
dir2/foo.o. This syntax is not flexbile enough to handle cases where
one of them is a standard object, but the other is not.
It is more sensible to use the relative path to the Makefile, like this:
obj-y += dir1/foo.o
OBJECT_FILES_NON_STANDARD_dir1/foo.o := y
obj-y += dir2/foo.o
OBJECT_FILES_NON_STANDARD_dir2/foo.o := y
To maintain the current behavior, I made adjustments to the following two
Makefiles:
- arch/x86/entry/vdso/Makefile, which compiles vclock_gettime.o, vgetcpu.o,
and their vdso32 variants.
- arch/x86/kvm/Makefile, which compiles vmx/vmenter.o and svm/vmenter.o
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
Acked-by: Sean Christopherson <seanjc@google.com>
It is useful to have a list of all *.dtb and *.dtbo files generated
from the current build.
With this commit, 'make dtbs' creates arch/*/boot/dts/dtbs-list, which
lists the dtb(o) files created in the current build. It maintains the
order of the dtb-y additions in Makefiles although the order is not
important for DTBs. It is a (good) side effect through the reuse of the
modules.order rule.
Please note this list only includes the files directly added to dtb-y.
For example, consider this case:
foo-dtbs := foo_base.dtb foo_overlay.dtbo
dtb-y := foo.dtb
In this example, the list will include foo.dtb, but not foo_base.dtb
or foo_overlay.dtbo.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Pull Kbuild updates from Masahiro Yamada:
- Implement the binary search in modpost for faster symbol lookup
- Respect HOSTCC when linking host programs written in Rust
- Change the binrpm-pkg target to generate kernel-devel RPM package
- Fix endianness issues for tee and ishtp MODULE_DEVICE_TABLE
- Unify vdso_install rules
- Remove unused __memexit* annotations
- Eliminate stale whitelisting for __devinit/__devexit from modpost
- Enable dummy-tools to handle the -fpatchable-function-entry flag
- Add 'userldlibs' syntax
* tag 'kbuild-v6.7' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: (30 commits)
kbuild: support 'userldlibs' syntax
kbuild: dummy-tools: pretend we understand -fpatchable-function-entry
kbuild: Correct missing architecture-specific hyphens
modpost: squash ALL_{INIT,EXIT}_TEXT_SECTIONS to ALL_TEXT_SECTIONS
modpost: merge sectioncheck table entries regarding init/exit sections
modpost: use ALL_INIT_SECTIONS for the section check from DATA_SECTIONS
modpost: disallow the combination of EXPORT_SYMBOL and __meminit*
modpost: remove EXIT_SECTIONS macro
modpost: remove MEM_INIT_SECTIONS macro
modpost: remove more symbol patterns from the section check whitelist
modpost: disallow *driver to reference .meminit* sections
linux/init: remove __memexit* annotations
modpost: remove ALL_EXIT_DATA_SECTIONS macro
kbuild: simplify cmd_ld_multi_m
kbuild: avoid too many execution of scripts/pahole-flags.sh
kbuild: remove ARCH_POSTLINK from module builds
kbuild: unify no-compiler-targets and no-sync-config-targets
kbuild: unify vdso_install rules
docs: kbuild: add INSTALL_DTBS_PATH
UML: remove unused cmd_vdso_install
...
The main challenge with defining `work_struct` fields is making sure
that the function pointer stored in the `work_struct` is appropriate for
the work item type it is embedded in. It needs to know the offset of the
`work_struct` field being used (even if there are several!) so that it
can do a `container_of`, and it needs to know the type of the work item
so that it can call into the right user-provided code. All of this needs
to happen in a way that provides a safe API to the user, so that users
of the workqueue cannot mix up the function pointers.
There are three important pieces that are relevant when doing this:
* The pointer type.
* The work item struct. This is what the pointer points at.
* The `work_struct` field. This is a field of the work item struct.
This patch introduces a separate trait for each piece. The pointer type
is given a `WorkItemPointer` trait, which pointer types need to
implement to be usable with the workqueue. This trait will be
implemented for `Arc` and `Box` in a later patch in this patchset.
Implementing this trait is unsafe because this is where the
`container_of` operation happens, but user-code will not need to
implement it themselves.
The work item struct should then implement the `WorkItem` trait. This
trait is where user-code specifies what they want to happen when a work
item is executed. It also specifies what the correct pointer type is.
Finally, to make the work item struct know the offset of its
`work_struct` field, we use a trait called `HasWork<T, ID>`. If a type
implements this trait, then the type declares that, at the given offset,
there is a field of type `Work<T, ID>`. The trait is marked unsafe
because the OFFSET constant must be correct, but we provide an
`impl_has_work!` macro that can safely implement `HasWork<T>` on a type.
The macro expands to something that only compiles if the specified field
really has the type `Work<T>`. It is used like this:
```
struct MyWorkItem {
work_field: Work<MyWorkItem, 1>,
}
impl_has_work! {
impl HasWork<MyWorkItem, 1> for MyWorkItem { self.work_field }
}
```
Note that since the `Work` type is annotated with an id, you can have
several `work_struct` fields by using a different id for each one.
Co-developed-by: Gary Guo <gary@garyguo.net>
Signed-off-by: Gary Guo <gary@garyguo.net>
Signed-off-by: Alice Ryhl <aliceryhl@google.com>
Reviewed-by: Benno Lossin <benno.lossin@proton.me>
Reviewed-by: Martin Rodriguez Reboredo <yakoyoku@gmail.com>
Reviewed-by: Andreas Hindborg <a.hindborg@samsung.com>
Reviewed-by: Boqun Feng <boqun.feng@gmail.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
`rustc` outputs by default the temporary files (i.e. the ones saved
by `-Csave-temps`, such as `*.rcgu*` files) in the current working
directory when `-o` and `--out-dir` are not given (even if
`--emit=x=path` is given, i.e. it does not use those for temporaries).
Since out-of-tree modules are compiled from the `linux` tree,
`rustc` then tries to create them there, which may not be accessible.
Thus pass `--out-dir` explicitly, even if it is just for the temporary
files.
Similarly, do so for Rust host programs too.
Reported-by: Raphael Nestler <raphael.nestler@gmail.com>
Closes: https://github.com/Rust-for-Linux/linux/issues/1015
Reported-by: Andrea Righi <andrea.righi@canonical.com>
Tested-by: Raphael Nestler <raphael.nestler@gmail.com> # non-hostprogs
Tested-by: Andrea Righi <andrea.righi@canonical.com> # non-hostprogs
Fixes: 295d8398c6 ("kbuild: specify output names separately for each emission type from rustc")
Cc: stable@vger.kernel.org
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
Tested-by: Martin Rodriguez Reboredo <yakoyoku@gmail.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Pull Kbuild updates from Masahiro Yamada:
- Remove the deprecated rule to build *.dtbo from *.dts
- Refactor section mismatch detection in modpost
- Fix bogus ARM section mismatch detections
- Fix error of 'make gtags' with O= option
- Add Clang's target triple to KBUILD_CPPFLAGS to fix a build error
with the latest LLVM version
- Rebuild the built-in initrd when KBUILD_BUILD_TIMESTAMP is changed
- Ignore more compiler-generated symbols for kallsyms
- Fix 'make local*config' to handle the ${CONFIG_FOO} form in Makefiles
- Enable more kernel-doc warnings with W=2
- Refactor <linux/export.h> by generating KSYMTAB data by modpost
- Deprecate <asm/export.h> and <asm-generic/export.h>
- Remove the EXPORT_DATA_SYMBOL macro
- Move the check for static EXPORT_SYMBOL back to modpost, which makes
the build faster
- Re-implement CONFIG_TRIM_UNUSED_KSYMS with one-pass algorithm
- Warn missing MODULE_DESCRIPTION when building modules with W=1
- Make 'make clean' robust against too long argument error
- Exclude more objects from GCOV to fix CFI failures with GCOV
- Allow 'make modules_install' to install modules.builtin and
modules.builtin.modinfo even when CONFIG_MODULES is disabled
- Include modules.builtin and modules.builtin.modinfo in the
linux-image Debian package even when CONFIG_MODULES is disabled
- Revive "Entering directory" logging for the latest Make version
* tag 'kbuild-v6.5' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: (72 commits)
modpost: define more R_ARM_* for old distributions
kbuild: revive "Entering directory" for Make >= 4.4.1
kbuild: set correct abs_srctree and abs_objtree for package builds
scripts/mksysmap: Ignore prefixed KCFI symbols
kbuild: deb-pkg: remove the CONFIG_MODULES check in buildeb
kbuild: builddeb: always make modules_install, to install modules.builtin*
modpost: continue even with unknown relocation type
modpost: factor out Elf_Sym pointer calculation to section_rel()
modpost: factor out inst location calculation to section_rel()
kbuild: Disable GCOV for *.mod.o
kbuild: Fix CFI failures with GCOV
kbuild: make clean rule robust against too long argument error
script: modpost: emit a warning when the description is missing
kbuild: make modules_install copy modules.builtin(.modinfo)
linux/export.h: rename 'sec' argument to 'license'
modpost: show offset from symbol for section mismatch warnings
modpost: merge two similar section mismatch warnings
kbuild: implement CONFIG_TRIM_UNUSED_KSYMS without recursion
modpost: use null string instead of NULL pointer for default namespace
modpost: squash sym_update_namespace() into sym_add_exported()
...
When CONFIG_TRIM_UNUSED_KSYMS is enabled, Kbuild recursively traverses
the directory tree to determine which EXPORT_SYMBOL to trim. If an
EXPORT_SYMBOL turns out to be unused by anyone, Kbuild begins the
second traverse, where some source files are recompiled with their
EXPORT_SYMBOL() tuned into a no-op.
Linus stated negative opinions about this slowness in commits:
- 5cf0fd591f ("Kbuild: disable TRIM_UNUSED_KSYMS option")
- a555bdd0c5 ("Kbuild: enable TRIM_UNUSED_KSYMS again, with some guarding")
We can do this better now. The final data structures of EXPORT_SYMBOL
are generated by the modpost stage, so modpost can selectively emit
KSYMTAB entries that are really used by modules.
Commit f73edc8951 ("kbuild: unify two modpost invocations") is another
ground-work to do this in a one-pass algorithm. With the list of modules,
modpost sets sym->used if it is used by a module. modpost emits KSYMTAB
only for symbols with sym->used==true.
BTW, Nicolas explained why the trimming was implemented with recursion:
https://lore.kernel.org/all/2o2rpn97-79nq-p7s2-nq5-8p83391473r@syhkavp.arg/
Actually, we never achieved that level of optimization where the chain
reaction of trimming comes into play because:
- CONFIG_LTO_CLANG cannot remove any unused symbols
- CONFIG_LD_DEAD_CODE_DATA_ELIMINATION is enabled only for vmlinux,
but not modules
If deeper trimming is required, we need to revisit this, but I guess
that is unlikely to happen.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Commit 31cb50b559 ("kbuild: check static EXPORT_SYMBOL* by script
instead of modpost") moved the static EXPORT_SYMBOL* check from the
mostpost to a shell script because I thought it must be checked per
compilation unit to avoid false negatives.
I came up with an idea to do this in modpost, against combined ELF
files. The relocation entries in ELF will find the correct exported
symbol even if there exist symbols with the same name in different
compilation units.
Again, the same sample code.
Makefile:
obj-y += foo1.o foo2.o
foo1.c:
#include <linux/export.h>
static void foo(void) {}
EXPORT_SYMBOL(foo);
foo2.c:
void foo(void) {}
Then, modpost can catch it correctly.
MODPOST Module.symvers
ERROR: modpost: vmlinux: local symbol 'foo' was exported
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
