In order to synchronize with C or external memory, atomic operations
over raw pointers are need. Although there is already an
`Atomic::from_ptr()` to provide a `&Atomic<T>`, it's more convenient to
have helpers that directly perform atomic operations on raw pointers.
Hence a few are added, which are basically an `Atomic::from_ptr().op()`
wrapper.
Note: for naming, since `atomic_xchg()` and `atomic_cmpxchg()` have a
conflict naming to 32bit C atomic xchg/cmpxchg, hence the helpers are
just named as `xchg()` and `cmpxchg()`. For `atomic_load()` and
`atomic_store()`, their 32bit C counterparts are `atomic_read()` and
`atomic_set()`, so keep the `atomic_` prefix.
[boqun: Fix typo spotted by Alice and fix broken sentence spotted by
Gary]
Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Reviewed-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260120115207.55318-3-boqun.feng@gmail.com
Link: https://patch.msgid.link/20260303201701.12204-11-boqun@kernel.org
Atomic pointer support is an important piece of synchronization
algorithm, e.g. RCU, hence provide the support for that.
Note that instead of relying on atomic_long or the implementation of
`Atomic<usize>`, a new set of helpers (atomic_ptr_*) is introduced for
atomic pointer specifically, this is because ptr2int casting would
lose the provenance of a pointer and even though in theory there are a
few tricks the provenance can be restored, it'll still be a simpler
implementation if C could provide atomic pointers directly. The side
effects of this approach are: we don't have the arithmetic and logical
operations for pointers yet and the current implementation only works
on ARCH_SUPPORTS_ATOMIC_RMW architectures, but these are implementation
issues and can be added later.
Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Gary Guo <gary@garyguo.net>
Reviewed-by: FUJITA Tomonori <fujita.tomonori@gmail.com>
Link: https://patch.msgid.link/20260120140503.62804-3-boqun.feng@gmail.com
Link: https://patch.msgid.link/20260303201701.12204-8-boqun@kernel.org
Currently, since all the architectures that support Rust all have
CONFIG_ARCH_SUPPORTS_ATOMIC_RMW selected, the helpers of atomic
load/store on i8 and i16 relies on CONFIG_ARCH_SUPPORTS_ATOMIC_RMW=y.
It's generally fine since most of architectures support that.
The plan for CONFIG_ARCH_SUPPORTS_ATOMIC_RMW=n architectures is adding
their (probably lock-based) atomic load/store for i8 and i16 as their
atomic_{read,set}() and atomic64_{read,set}() counterpart when they
plans to support Rust.
Hence use a statis_assert!() to check this and remind the future us the
need of the helpers. This is more clear than the #[cfg] on impl blocks
of i8 and i16.
Suggested-by: Dirk Behme <dirk.behme@gmail.com>
Suggested-by: Benno Lossin <lossin@kernel.org>
Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260120140503.62804-2-boqun.feng@gmail.com
Link: https://patch.msgid.link/20260303201701.12204-7-boqun@kernel.org
Originally, `Atomic::from_ptr()` requires `T` being a `Sync` because I
thought having the ability to do `from_ptr()` meant multiplle
`&Atomic<T>`s shared by different threads, which was identical (or
similar) to multiple `&T`s shared by different threads. Hence `T` was
required to be `Sync`. However this is not true, since `&Atomic<T>` is
not the same at `&T`. Moreover, having this bound makes `Atomic::<*mut
T>::from_ptr()` impossible, which is definitely not intended. Therefore
remove the `T: Sync` bound.
[boqun: Fix title typo spotted by Alice & Gary]
Fixes: 29c32c405e ("rust: sync: atomic: Add generic atomics")
Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Reviewed-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260120115207.55318-2-boqun.feng@gmail.com
Link: https://patch.msgid.link/20260303201701.12204-2-boqun@kernel.org
Current `dma_read!`, `dma_write!` macros also use a custom
`addr_of!()`-based implementation for projecting pointers, which has
soundness issue as it relies on absence of `Deref` implementation on types.
It also has a soundness issue where it does not protect against unaligned
fields (when `#[repr(packed)]` is used) so it can generate misaligned
accesses.
This commit migrates them to use the general pointer projection
infrastructure, which handles these cases correctly.
As part of migration, the macro is updated to have an improved surface
syntax. The current macro have
dma_read!(a.b.c[d].e.f)
to mean `a.b.c` is a DMA coherent allocation and it should project into it
with `[d].e.f` and do a read, which is confusing as it makes the indexing
operator integral to the macro (so it will break if you have an array of
`CoherentAllocation`, for example).
This also is problematic as we would like to generalize
`CoherentAllocation` from just slices to arbitrary types.
Make the macro expects `dma_read!(path.to.dma, .path.inside.dma)` as the
canonical syntax. The index operator is no longer special and is just one
type of projection (in additional to field projection). Similarly, make
`dma_write!(path.to.dma, .path.inside.dma, value)` become the canonical
syntax for writing.
Another issue of the current macro is that it is always fallible. This
makes sense with existing design of `CoherentAllocation`, but once we
support fixed size arrays with `CoherentAllocation`, it is desirable to
have the ability to perform infallible indexing as well, e.g. doing a `[0]`
index of `[Foo; 2]` is okay and can be checked at build-time, so forcing
falliblity is non-ideal. To capture this, the macro is changed to use
`[idx]` as infallible projection and `[idx]?` as fallible index projection
(those syntax are part of the general projection infra). A benefit of this
is that while individual indexing operation may fail, the overall
read/write operation is not fallible.
Fixes: ad2907b4e3 ("rust: add dma coherent allocator abstraction")
Reviewed-by: Benno Lossin <lossin@kernel.org>
Signed-off-by: Gary Guo <gary@garyguo.net>
Link: https://patch.msgid.link/20260302164239.284084-4-gary@kernel.org
[ Capitalize safety comments; slightly improve wording in doc-comments.
- Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Add a generic infrastructure for performing field and index projections on
raw pointers. This will form the basis of performing I/O projections.
Pointers manipulations are intentionally using the safe wrapping variants
instead of the unsafe variants, as the latter requires pointers to be
inside an allocation which is not necessarily true for I/O pointers.
This projection macro protects against rogue `Deref` implementation, which
can causes the projected pointer to be outside the bounds of starting
pointer. This is extremely unlikely and Rust has a lint to catch this, but
is unsoundness regardless. The protection works by inducing type inference
ambiguity when `Deref` is implemented.
This projection macro also stops projecting into unaligned fields (i.e.
fields of `#[repr(packed)]` structs), as misaligned pointers require
special handling. This is implemented by attempting to create reference to
projected field inside a `if false` block. Despite being unreachable, Rust
still checks that they're not unaligned fields.
The projection macro supports both fallible and infallible index
projections. These are described in detail inside the documentation.
Signed-off-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Benno Lossin <lossin@kernel.org>
Acked-by: Miguel Ojeda <ojeda@kernel.org>
Link: https://patch.msgid.link/20260302164239.284084-3-gary@kernel.org
[ * Add intro-doc links where possible,
* Fix typos and slightly improve wording, e.g. "as documentation
describes" -> "as the documentation of [`Self::proj`] describes",
* Add an empty line between regular and safety comments, before
examples, and between logically independent comments,
* Capitalize various safety comments.
- Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Add a `KnownSize` trait which is used obtain a size from a raw pointer's
metadata. This makes it possible to obtain size information on a raw slice
pointer. This is similar to Rust `core::mem::size_of_val_raw` which is not
yet stable.
Signed-off-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Benno Lossin <lossin@kernel.org>
Acked-by: Miguel Ojeda <ojeda@kernel.org>
Link: https://patch.msgid.link/20260302164239.284084-2-gary@kernel.org
[ Fix wording in doc-comment. - Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
Pull kunit fixes from Shuah Khan:
- Fix rust warnings when CONFIG_PRINTK is disabled
- Reduce stack usage in kunit_run_tests() to fix warnings when
CONFIG_FRAME_WARN is set to a relatively low value
- Update email address for David Gow
- Copy caller args in kunit tool in run_kernel to prevent mutation
* tag 'linux_kselftest-kunit-fixes-7.0-rc3' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest:
kunit: reduce stack usage in kunit_run_tests()
kunit: tool: copy caller args in run_kernel to prevent mutation
rust: kunit: fix warning when !CONFIG_PRINTK
MAINTAINERS: Update email address for David Gow
Gary noticed [1] that the initializer macros as well as the `[Pin]Init`
traits cannot support unaligned fields, since they use operations that
require aligned pointers. This means that any code using structs with
unaligned fields in pin-init is unsound.
By default, the `init!` macro generates references to initialized fields,
which makes the compiler check that those fields are aligned. However,
we added the `#[disable_initialized_field_access]` attribute to avoid
this behavior in commit ceca298c53 ("rust: pin-init: internal: init:
add escape hatch for referencing initialized fields"). Thus remove the
`#[disable_initialized_field_access]` attribute from `init!`, which is
the only safe way to create an initializer handling unaligned fields.
If support for in-place initializing structs with unaligned fields is
required in the future, we could figure out a solution. This is tracked
in [2].
Reported-by: Gary Guo <gary@garyguo.net>
Closes: https://rust-for-linux.zulipchat.com/#narrow/channel/561532-pin-init/topic/initialized.20field.20accessor.20detection/with/576210658 [1]
Link: https://github.com/Rust-for-Linux/pin-init/issues/112 [2]
Fixes: ceca298c53 ("rust: pin-init: internal: init: add escape hatch for referencing initialized fields")
Signed-off-by: Benno Lossin <lossin@kernel.org>
Acked-by: Janne Grunau <j@jannau.net>
Reviewed-by: Gary Guo <gary@garyguo.net>
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Link: https://patch.msgid.link/20260302140424.4097655-1-lossin@kernel.org
[ Adjusted tags and reworded as discussed. - Miguel ]
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
When building with an out directory (O=), absolute paths can end up in the
file name in `#[track_caller]` or the panic message. This is not desirable
as this leaks the exact path being used to build the kernel and means that
the same location can appear in two forms (relative or absolute).
This is reported by Asahi [1] and is being workaround in [2] previously to
force everything to be absolute path. Using absolute path for everything
solves the inconsistency, however it does not address the reproducibility
issue. So, fix this by remap all absolute paths to srctree to relative path
instead.
This is previously attempted in commit dbdffaf50f ("kbuild, rust: use
-fremap-path-prefix to make paths relative") but that was reverted as
remapping debug info causes some tool (e.g. objdump) to be unable to find
sources. Therefore, use `--remap-path-scope` to only remap macros but leave
debuginfo untouched. `--remap-path-scope` is only stable in Rust 1.95, so
use `rustc-option` to detect its presence. This feature has been available
as `-Zremap-path-scope` for all versions that we support; however due to
bugs in the Rust compiler, it does not work reliably until 1.94. I opted to
not enable it for 1.94 as it's just a single version that we missed.
This change can be validated by building a kernel with O=, strip debug info
on vmlinux, and then check if the absolute path exists in `strings
vmlinux`, e.g. `strings vmlinux |grep \/home`.
Reported-by: Janne Grunau <j@jannau.net>
Reported-by: Asahi Lina <lina+kernel@asahilina.net>
Closes: https://rust-for-linux.zulipchat.com/#narrow/channel/288089-General/topic/Per-call-site.20data.20and.20lock.20class.20keys/near/572466559 [1]
Link: https://github.com/AsahiLinux/linux/commit/54ab88878869036c9d6620101bfe17a81e88c2f9 [2]
Signed-off-by: Gary Guo <gary@garyguo.net>
Acked-by: Nicolas Schier <nsc@kernel.org> # kbuild
Link: https://patch.msgid.link/20260226152112.3222886-1-gary@kernel.org
[ Reworded for few typos. - Miguel ]
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
If `CONFIG_BLOCK` is disabled, the following warnings are displayed
during build:
warning: struct `NullTerminatedFormatter` is never constructed
--> ../rust/kernel/str.rs:667:19
|
667 | pub(crate) struct NullTerminatedFormatter<'a> {
| ^^^^^^^^^^^^^^^^^^^^^^^
|
= note: `#[warn(dead_code)]` (part of `#[warn(unused)]`) on by default
warning: associated function `new` is never used
--> ../rust/kernel/str.rs:673:19
|
671 | impl<'a> NullTerminatedFormatter<'a> {
| ------------------------------------ associated function in this implementation
672 | /// Create a new [`Self`] instance.
673 | pub(crate) fn new(buffer: &'a mut [u8]) -> Option<NullTerminatedFormatter<'a>> {
Fix them by making `NullTerminatedFormatter` public, as it could be
useful for drivers anyway.
Fixes: cdde7a1951 ("rust: str: introduce `NullTerminatedFormatter`")
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Reviewed-by: Andreas Hindborg <a.hindborg@kernel.org>
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260224-nullterminatedformatter-v1-1-5bef7b9b3d4c@nvidia.com
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
If `CONFIG_PRINTK` is not set, then the following warnings are issued
during build:
warning: unused variable: `args`
--> ../rust/kernel/kunit.rs:16:12
|
16 | pub fn err(args: fmt::Arguments<'_>) {
| ^^^^ help: if this is intentional, prefix it with an underscore: `_args`
|
= note: `#[warn(unused_variables)]` (part of `#[warn(unused)]`) on by default
warning: unused variable: `args`
--> ../rust/kernel/kunit.rs:32:13
|
32 | pub fn info(args: fmt::Arguments<'_>) {
| ^^^^ help: if this is intentional, prefix it with an underscore: `_args`
Fix this by adding a no-op assignment using `args` when `CONFIG_PRINTK`
is not set.
Fixes: a66d733da8 ("rust: support running Rust documentation tests as KUnit ones")
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Reviewed-by: David Gow <david@davidgow.net>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>
