mirror of
https://github.com/linux-msm/laptops-kernel.git
synced 2026-08-13 14:19:53 -07:00
rust_binder: add Rust Binder driver
We're generally not proponents of rewrites (nasty uncomfortable things that make you late for dinner!). So why rewrite Binder? Binder has been evolving over the past 15+ years to meet the evolving needs of Android. Its responsibilities, expectations, and complexity have grown considerably during that time. While we expect Binder to continue to evolve along with Android, there are a number of factors that currently constrain our ability to develop/maintain it. Briefly those are: 1. Complexity: Binder is at the intersection of everything in Android and fulfills many responsibilities beyond IPC. It has become many things to many people, and due to its many features and their interactions with each other, its complexity is quite high. In just 6kLOC it must deliver transactions to the right threads. It must correctly parse and translate the contents of transactions, which can contain several objects of different types (e.g., pointers, fds) that can interact with each other. It controls the size of thread pools in userspace, and ensures that transactions are assigned to threads in ways that avoid deadlocks where the threadpool has run out of threads. It must track refcounts of objects that are shared by several processes by forwarding refcount changes between the processes correctly. It must handle numerous error scenarios and it combines/nests 13 different locks, 7 reference counters, and atomic variables. Finally, It must do all of this as fast and efficiently as possible. Minor performance regressions can cause a noticeably degraded user experience. 2. Things to improve: Thousand-line functions [1], error-prone error handling [2], and confusing structure can occur as a code base grows organically. After more than a decade of development, this codebase could use an overhaul. [1]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/android/binder.c?h=v6.5#n2896 [2]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/android/binder.c?h=v6.5#n3658 3. Security critical: Binder is a critical part of Android's sandboxing strategy. Even Android's most de-privileged sandboxes (e.g. the Chrome renderer, or SW Codec) have direct access to Binder. More than just about any other component, it's important that Binder provide robust security, and itself be robust against security vulnerabilities. It's #1 (high complexity) that has made continuing to evolve Binder and resolving #2 (tech debt) exceptionally difficult without causing #3 (security issues). For Binder to continue to meet Android's needs, we need better ways to manage (and reduce!) complexity without increasing the risk. The biggest change is obviously the choice of programming language. We decided to use Rust because it directly addresses a number of the challenges within Binder that we have faced during the last years. It prevents mistakes with ref counting, locking, bounds checking, and also does a lot to reduce the complexity of error handling. Additionally, we've been able to use the more expressive type system to encode the ownership semantics of the various structs and pointers, which takes the complexity of managing object lifetimes out of the hands of the programmer, reducing the risk of use-after-frees and similar problems. Rust has many different pointer types that it uses to encode ownership semantics into the type system, and this is probably one of the most important aspects of how it helps in Binder. The Binder driver has a lot of different objects that have complex ownership semantics; some pointers own a refcount, some pointers have exclusive ownership, and some pointers just reference the object and it is kept alive in some other manner. With Rust, we can use a different pointer type for each kind of pointer, which enables the compiler to enforce that the ownership semantics are implemented correctly. Another useful feature is Rust's error handling. Rust allows for more simplified error handling with features such as destructors, and you get compilation failures if errors are not properly handled. This means that even though Rust requires you to spend more lines of code than C on things such as writing down invariants that are left implicit in C, the Rust driver is still slightly smaller than C binder: Rust is 5.5kLOC and C is 5.8kLOC. (These numbers are excluding blank lines, comments, binderfs, and any debugging facilities in C that are not yet implemented in the Rust driver. The numbers include abstractions in rust/kernel/ that are unlikely to be used by other drivers than Binder.) Although this rewrite completely rethinks how the code is structured and how assumptions are enforced, we do not fundamentally change *how* the driver does the things it does. A lot of careful thought has gone into the existing design. The rewrite is aimed rather at improving code health, structure, readability, robustness, security, maintainability and extensibility. We also include more inline documentation, and improve how assumptions in the code are enforced. Furthermore, all unsafe code is annotated with a SAFETY comment that explains why it is correct. We have left the binderfs filesystem component in C. Rewriting it in Rust would be a large amount of work and requires a lot of bindings to the file system interfaces. Binderfs has not historically had the same challenges with security and complexity, so rewriting binderfs seems to have lower value than the rest of Binder. Correctness and feature parity ------------------------------ Rust binder passes all tests that validate the correctness of Binder in the Android Open Source Project. We can boot a device, and run a variety of apps and functionality without issues. We have performed this both on the Cuttlefish Android emulator device, and on a Pixel 6 Pro. As for feature parity, Rust binder currently implements all features that C binder supports, with the exception of some debugging facilities. The missing debugging facilities will be added before we submit the Rust implementation upstream. Tracepoints ----------- I did not include all of the tracepoints as I felt that the mechansim for making C access fields of Rust structs should be discussed on list separately. I also did not include the support for building Rust Binder as a module since that requires exporting a bunch of additional symbols on the C side. Original RFC Link with old benchmark numbers: https://lore.kernel.org/r/20231101-rust-binder-v1-0-08ba9197f637@google.com Co-developed-by: Wedson Almeida Filho <wedsonaf@gmail.com> Signed-off-by: Wedson Almeida Filho <wedsonaf@gmail.com> Co-developed-by: Matt Gilbride <mattgilbride@google.com> Signed-off-by: Matt Gilbride <mattgilbride@google.com> Acked-by: Carlos Llamas <cmllamas@google.com> Acked-by: Paul Moore <paul@paul-moore.com> Signed-off-by: Alice Ryhl <aliceryhl@google.com> Link: https://lore.kernel.org/r/20250919-rust-binder-v2-1-a384b09f28dd@google.com Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
This commit is contained in:
committed by
Greg Kroah-Hartman
parent
55f6ac4484
commit
eafedbc7c0
+14
-1
@@ -14,6 +14,19 @@ config ANDROID_BINDER_IPC
|
|||||||
Android process, using Binder to identify, invoke and pass arguments
|
Android process, using Binder to identify, invoke and pass arguments
|
||||||
between said processes.
|
between said processes.
|
||||||
|
|
||||||
|
config ANDROID_BINDER_IPC_RUST
|
||||||
|
bool "Rust version of Android Binder IPC Driver"
|
||||||
|
depends on RUST && MMU && !ANDROID_BINDER_IPC
|
||||||
|
help
|
||||||
|
This enables the Rust implementation of the Binder driver.
|
||||||
|
|
||||||
|
Binder is used in Android for both communication between processes,
|
||||||
|
and remote method invocation.
|
||||||
|
|
||||||
|
This means one Android process can call a method/routine in another
|
||||||
|
Android process, using Binder to identify, invoke and pass arguments
|
||||||
|
between said processes.
|
||||||
|
|
||||||
config ANDROID_BINDERFS
|
config ANDROID_BINDERFS
|
||||||
bool "Android Binderfs filesystem"
|
bool "Android Binderfs filesystem"
|
||||||
depends on ANDROID_BINDER_IPC
|
depends on ANDROID_BINDER_IPC
|
||||||
@@ -28,7 +41,7 @@ config ANDROID_BINDERFS
|
|||||||
|
|
||||||
config ANDROID_BINDER_DEVICES
|
config ANDROID_BINDER_DEVICES
|
||||||
string "Android Binder devices"
|
string "Android Binder devices"
|
||||||
depends on ANDROID_BINDER_IPC
|
depends on ANDROID_BINDER_IPC || ANDROID_BINDER_IPC_RUST
|
||||||
default "binder,hwbinder,vndbinder"
|
default "binder,hwbinder,vndbinder"
|
||||||
help
|
help
|
||||||
Default value for the binder.devices parameter.
|
Default value for the binder.devices parameter.
|
||||||
|
|||||||
@@ -4,3 +4,4 @@ ccflags-y += -I$(src) # needed for trace events
|
|||||||
obj-$(CONFIG_ANDROID_BINDERFS) += binderfs.o
|
obj-$(CONFIG_ANDROID_BINDERFS) += binderfs.o
|
||||||
obj-$(CONFIG_ANDROID_BINDER_IPC) += binder.o binder_alloc.o binder_netlink.o
|
obj-$(CONFIG_ANDROID_BINDER_IPC) += binder.o binder_alloc.o binder_netlink.o
|
||||||
obj-$(CONFIG_ANDROID_BINDER_ALLOC_KUNIT_TEST) += tests/
|
obj-$(CONFIG_ANDROID_BINDER_ALLOC_KUNIT_TEST) += tests/
|
||||||
|
obj-$(CONFIG_ANDROID_BINDER_IPC_RUST) += binder/
|
||||||
|
|||||||
@@ -0,0 +1,9 @@
|
|||||||
|
# SPDX-License-Identifier: GPL-2.0-only
|
||||||
|
ccflags-y += -I$(src) # needed for trace events
|
||||||
|
|
||||||
|
obj-$(CONFIG_ANDROID_BINDER_IPC_RUST) += rust_binder.o
|
||||||
|
rust_binder-y := \
|
||||||
|
rust_binder_main.o \
|
||||||
|
rust_binderfs.o \
|
||||||
|
rust_binder_events.o \
|
||||||
|
page_range_helper.o
|
||||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,180 @@
|
|||||||
|
// SPDX-License-Identifier: GPL-2.0
|
||||||
|
|
||||||
|
// Copyright (C) 2025 Google LLC.
|
||||||
|
|
||||||
|
use kernel::{
|
||||||
|
error::Error,
|
||||||
|
list::{List, ListArc, ListLinks},
|
||||||
|
prelude::*,
|
||||||
|
security,
|
||||||
|
str::{CStr, CString},
|
||||||
|
sync::{Arc, Mutex},
|
||||||
|
task::Kuid,
|
||||||
|
};
|
||||||
|
|
||||||
|
use crate::{error::BinderError, node::NodeRef, process::Process};
|
||||||
|
|
||||||
|
kernel::sync::global_lock! {
|
||||||
|
// SAFETY: We call `init` in the module initializer, so it's initialized before first use.
|
||||||
|
pub(crate) unsafe(uninit) static CONTEXTS: Mutex<ContextList> = ContextList {
|
||||||
|
list: List::new(),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
pub(crate) struct ContextList {
|
||||||
|
list: List<Context>,
|
||||||
|
}
|
||||||
|
|
||||||
|
pub(crate) fn get_all_contexts() -> Result<KVec<Arc<Context>>> {
|
||||||
|
let lock = CONTEXTS.lock();
|
||||||
|
|
||||||
|
let count = lock.list.iter().count();
|
||||||
|
|
||||||
|
let mut ctxs = KVec::with_capacity(count, GFP_KERNEL)?;
|
||||||
|
for ctx in &lock.list {
|
||||||
|
ctxs.push(Arc::from(ctx), GFP_KERNEL)?;
|
||||||
|
}
|
||||||
|
Ok(ctxs)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// This struct keeps track of the processes using this context, and which process is the context
|
||||||
|
/// manager.
|
||||||
|
struct Manager {
|
||||||
|
node: Option<NodeRef>,
|
||||||
|
uid: Option<Kuid>,
|
||||||
|
all_procs: List<Process>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// There is one context per binder file (/dev/binder, /dev/hwbinder, etc)
|
||||||
|
#[pin_data]
|
||||||
|
pub(crate) struct Context {
|
||||||
|
#[pin]
|
||||||
|
manager: Mutex<Manager>,
|
||||||
|
pub(crate) name: CString,
|
||||||
|
#[pin]
|
||||||
|
links: ListLinks,
|
||||||
|
}
|
||||||
|
|
||||||
|
kernel::list::impl_list_arc_safe! {
|
||||||
|
impl ListArcSafe<0> for Context { untracked; }
|
||||||
|
}
|
||||||
|
kernel::list::impl_list_item! {
|
||||||
|
impl ListItem<0> for Context {
|
||||||
|
using ListLinks { self.links };
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Context {
|
||||||
|
pub(crate) fn new(name: &CStr) -> Result<Arc<Self>> {
|
||||||
|
let name = CString::try_from(name)?;
|
||||||
|
let list_ctx = ListArc::pin_init::<Error>(
|
||||||
|
try_pin_init!(Context {
|
||||||
|
name,
|
||||||
|
links <- ListLinks::new(),
|
||||||
|
manager <- kernel::new_mutex!(Manager {
|
||||||
|
all_procs: List::new(),
|
||||||
|
node: None,
|
||||||
|
uid: None,
|
||||||
|
}, "Context::manager"),
|
||||||
|
}),
|
||||||
|
GFP_KERNEL,
|
||||||
|
)?;
|
||||||
|
|
||||||
|
let ctx = list_ctx.clone_arc();
|
||||||
|
CONTEXTS.lock().list.push_back(list_ctx);
|
||||||
|
|
||||||
|
Ok(ctx)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Called when the file for this context is unlinked.
|
||||||
|
///
|
||||||
|
/// No-op if called twice.
|
||||||
|
pub(crate) fn deregister(&self) {
|
||||||
|
// SAFETY: We never add the context to any other linked list than this one, so it is either
|
||||||
|
// in this list, or not in any list.
|
||||||
|
unsafe { CONTEXTS.lock().list.remove(self) };
|
||||||
|
}
|
||||||
|
|
||||||
|
pub(crate) fn register_process(self: &Arc<Self>, proc: ListArc<Process>) {
|
||||||
|
if !Arc::ptr_eq(self, &proc.ctx) {
|
||||||
|
pr_err!("Context::register_process called on the wrong context.");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
self.manager.lock().all_procs.push_back(proc);
|
||||||
|
}
|
||||||
|
|
||||||
|
pub(crate) fn deregister_process(self: &Arc<Self>, proc: &Process) {
|
||||||
|
if !Arc::ptr_eq(self, &proc.ctx) {
|
||||||
|
pr_err!("Context::deregister_process called on the wrong context.");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
// SAFETY: We just checked that this is the right list.
|
||||||
|
unsafe { self.manager.lock().all_procs.remove(proc) };
|
||||||
|
}
|
||||||
|
|
||||||
|
pub(crate) fn set_manager_node(&self, node_ref: NodeRef) -> Result {
|
||||||
|
let mut manager = self.manager.lock();
|
||||||
|
if manager.node.is_some() {
|
||||||
|
pr_warn!("BINDER_SET_CONTEXT_MGR already set");
|
||||||
|
return Err(EBUSY);
|
||||||
|
}
|
||||||
|
security::binder_set_context_mgr(&node_ref.node.owner.cred)?;
|
||||||
|
|
||||||
|
// If the context manager has been set before, ensure that we use the same euid.
|
||||||
|
let caller_uid = Kuid::current_euid();
|
||||||
|
if let Some(ref uid) = manager.uid {
|
||||||
|
if *uid != caller_uid {
|
||||||
|
return Err(EPERM);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
manager.node = Some(node_ref);
|
||||||
|
manager.uid = Some(caller_uid);
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
pub(crate) fn unset_manager_node(&self) {
|
||||||
|
let node_ref = self.manager.lock().node.take();
|
||||||
|
drop(node_ref);
|
||||||
|
}
|
||||||
|
|
||||||
|
pub(crate) fn get_manager_node(&self, strong: bool) -> Result<NodeRef, BinderError> {
|
||||||
|
self.manager
|
||||||
|
.lock()
|
||||||
|
.node
|
||||||
|
.as_ref()
|
||||||
|
.ok_or_else(BinderError::new_dead)?
|
||||||
|
.clone(strong)
|
||||||
|
.map_err(BinderError::from)
|
||||||
|
}
|
||||||
|
|
||||||
|
pub(crate) fn for_each_proc<F>(&self, mut func: F)
|
||||||
|
where
|
||||||
|
F: FnMut(&Process),
|
||||||
|
{
|
||||||
|
let lock = self.manager.lock();
|
||||||
|
for proc in &lock.all_procs {
|
||||||
|
func(&proc);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
pub(crate) fn get_all_procs(&self) -> Result<KVec<Arc<Process>>> {
|
||||||
|
let lock = self.manager.lock();
|
||||||
|
let count = lock.all_procs.iter().count();
|
||||||
|
|
||||||
|
let mut procs = KVec::with_capacity(count, GFP_KERNEL)?;
|
||||||
|
for proc in &lock.all_procs {
|
||||||
|
procs.push(Arc::from(proc), GFP_KERNEL)?;
|
||||||
|
}
|
||||||
|
Ok(procs)
|
||||||
|
}
|
||||||
|
|
||||||
|
pub(crate) fn get_procs_with_pid(&self, pid: i32) -> Result<KVec<Arc<Process>>> {
|
||||||
|
let orig = self.get_all_procs()?;
|
||||||
|
let mut backing = KVec::with_capacity(orig.len(), GFP_KERNEL)?;
|
||||||
|
for proc in orig.into_iter().filter(|proc| proc.task.pid() == pid) {
|
||||||
|
backing.push(proc, GFP_KERNEL)?;
|
||||||
|
}
|
||||||
|
Ok(backing)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,204 @@
|
|||||||
|
// SPDX-License-Identifier: GPL-2.0
|
||||||
|
|
||||||
|
// Copyright (C) 2025 Google LLC.
|
||||||
|
|
||||||
|
//! Logic for closing files in a deferred manner.
|
||||||
|
//!
|
||||||
|
//! This file could make sense to have in `kernel::fs`, but it was rejected for being too
|
||||||
|
//! Binder-specific.
|
||||||
|
|
||||||
|
use core::mem::MaybeUninit;
|
||||||
|
use kernel::{
|
||||||
|
alloc::{AllocError, Flags},
|
||||||
|
bindings,
|
||||||
|
prelude::*,
|
||||||
|
};
|
||||||
|
|
||||||
|
/// Helper used for closing file descriptors in a way that is safe even if the file is currently
|
||||||
|
/// held using `fdget`.
|
||||||
|
///
|
||||||
|
/// Additional motivation can be found in commit 80cd795630d6 ("binder: fix use-after-free due to
|
||||||
|
/// ksys_close() during fdget()") and in the comments on `binder_do_fd_close`.
|
||||||
|
pub(crate) struct DeferredFdCloser {
|
||||||
|
inner: KBox<DeferredFdCloserInner>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// SAFETY: This just holds an allocation with no real content, so there's no safety issue with
|
||||||
|
/// moving it across threads.
|
||||||
|
unsafe impl Send for DeferredFdCloser {}
|
||||||
|
/// SAFETY: This just holds an allocation with no real content, so there's no safety issue with
|
||||||
|
/// moving it across threads.
|
||||||
|
unsafe impl Sync for DeferredFdCloser {}
|
||||||
|
|
||||||
|
/// # Invariants
|
||||||
|
///
|
||||||
|
/// If the `file` pointer is non-null, then it points at a `struct file` and owns a refcount to
|
||||||
|
/// that file.
|
||||||
|
#[repr(C)]
|
||||||
|
struct DeferredFdCloserInner {
|
||||||
|
twork: MaybeUninit<bindings::callback_head>,
|
||||||
|
file: *mut bindings::file,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl DeferredFdCloser {
|
||||||
|
/// Create a new [`DeferredFdCloser`].
|
||||||
|
pub(crate) fn new(flags: Flags) -> Result<Self, AllocError> {
|
||||||
|
Ok(Self {
|
||||||
|
// INVARIANT: The `file` pointer is null, so the type invariant does not apply.
|
||||||
|
inner: KBox::new(
|
||||||
|
DeferredFdCloserInner {
|
||||||
|
twork: MaybeUninit::uninit(),
|
||||||
|
file: core::ptr::null_mut(),
|
||||||
|
},
|
||||||
|
flags,
|
||||||
|
)?,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Schedule a task work that closes the file descriptor when this task returns to userspace.
|
||||||
|
///
|
||||||
|
/// Fails if this is called from a context where we cannot run work when returning to
|
||||||
|
/// userspace. (E.g., from a kthread.)
|
||||||
|
pub(crate) fn close_fd(self, fd: u32) -> Result<(), DeferredFdCloseError> {
|
||||||
|
use bindings::task_work_notify_mode_TWA_RESUME as TWA_RESUME;
|
||||||
|
|
||||||
|
// In this method, we schedule the task work before closing the file. This is because
|
||||||
|
// scheduling a task work is fallible, and we need to know whether it will fail before we
|
||||||
|
// attempt to close the file.
|
||||||
|
|
||||||
|
// Task works are not available on kthreads.
|
||||||
|
let current = kernel::current!();
|
||||||
|
|
||||||
|
// Check if this is a kthread.
|
||||||
|
// SAFETY: Reading `flags` from a task is always okay.
|
||||||
|
if unsafe { ((*current.as_ptr()).flags & bindings::PF_KTHREAD) != 0 } {
|
||||||
|
return Err(DeferredFdCloseError::TaskWorkUnavailable);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Transfer ownership of the box's allocation to a raw pointer. This disables the
|
||||||
|
// destructor, so we must manually convert it back to a KBox to drop it.
|
||||||
|
//
|
||||||
|
// Until we convert it back to a `KBox`, there are no aliasing requirements on this
|
||||||
|
// pointer.
|
||||||
|
let inner = KBox::into_raw(self.inner);
|
||||||
|
|
||||||
|
// The `callback_head` field is first in the struct, so this cast correctly gives us a
|
||||||
|
// pointer to the field.
|
||||||
|
let callback_head = inner.cast::<bindings::callback_head>();
|
||||||
|
// SAFETY: This pointer offset operation does not go out-of-bounds.
|
||||||
|
let file_field = unsafe { core::ptr::addr_of_mut!((*inner).file) };
|
||||||
|
|
||||||
|
let current = current.as_ptr();
|
||||||
|
|
||||||
|
// SAFETY: This function currently has exclusive access to the `DeferredFdCloserInner`, so
|
||||||
|
// it is okay for us to perform unsynchronized writes to its `callback_head` field.
|
||||||
|
unsafe { bindings::init_task_work(callback_head, Some(Self::do_close_fd)) };
|
||||||
|
|
||||||
|
// SAFETY: This inserts the `DeferredFdCloserInner` into the task workqueue for the current
|
||||||
|
// task. If this operation is successful, then this transfers exclusive ownership of the
|
||||||
|
// `callback_head` field to the C side until it calls `do_close_fd`, and we don't touch or
|
||||||
|
// invalidate the field during that time.
|
||||||
|
//
|
||||||
|
// When the C side calls `do_close_fd`, the safety requirements of that method are
|
||||||
|
// satisfied because when a task work is executed, the callback is given ownership of the
|
||||||
|
// pointer.
|
||||||
|
//
|
||||||
|
// The file pointer is currently null. If it is changed to be non-null before `do_close_fd`
|
||||||
|
// is called, then that change happens due to the write at the end of this function, and
|
||||||
|
// that write has a safety comment that explains why the refcount can be dropped when
|
||||||
|
// `do_close_fd` runs.
|
||||||
|
let res = unsafe { bindings::task_work_add(current, callback_head, TWA_RESUME) };
|
||||||
|
|
||||||
|
if res != 0 {
|
||||||
|
// SAFETY: Scheduling the task work failed, so we still have ownership of the box, so
|
||||||
|
// we may destroy it.
|
||||||
|
unsafe { drop(KBox::from_raw(inner)) };
|
||||||
|
|
||||||
|
return Err(DeferredFdCloseError::TaskWorkUnavailable);
|
||||||
|
}
|
||||||
|
|
||||||
|
// This removes the fd from the fd table in `current`. The file is not fully closed until
|
||||||
|
// `filp_close` is called. We are given ownership of one refcount to the file.
|
||||||
|
//
|
||||||
|
// SAFETY: This is safe no matter what `fd` is. If the `fd` is valid (that is, if the
|
||||||
|
// pointer is non-null), then we call `filp_close` on the returned pointer as required by
|
||||||
|
// `file_close_fd`.
|
||||||
|
let file = unsafe { bindings::file_close_fd(fd) };
|
||||||
|
if file.is_null() {
|
||||||
|
// We don't clean up the task work since that might be expensive if the task work queue
|
||||||
|
// is long. Just let it execute and let it clean up for itself.
|
||||||
|
return Err(DeferredFdCloseError::BadFd);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Acquire a second refcount to the file.
|
||||||
|
//
|
||||||
|
// SAFETY: The `file` pointer points at a file with a non-zero refcount.
|
||||||
|
unsafe { bindings::get_file(file) };
|
||||||
|
|
||||||
|
// This method closes the fd, consuming one of our two refcounts. There could be active
|
||||||
|
// light refcounts created from that fd, so we must ensure that the file has a positive
|
||||||
|
// refcount for the duration of those active light refcounts. We do that by holding on to
|
||||||
|
// the second refcount until the current task returns to userspace.
|
||||||
|
//
|
||||||
|
// SAFETY: The `file` pointer is valid. Passing `current->files` as the file table to close
|
||||||
|
// it in is correct, since we just got the `fd` from `file_close_fd` which also uses
|
||||||
|
// `current->files`.
|
||||||
|
//
|
||||||
|
// Note: fl_owner_t is currently a void pointer.
|
||||||
|
unsafe { bindings::filp_close(file, (*current).files as bindings::fl_owner_t) };
|
||||||
|
|
||||||
|
// We update the file pointer that the task work is supposed to fput. This transfers
|
||||||
|
// ownership of our last refcount.
|
||||||
|
//
|
||||||
|
// INVARIANT: This changes the `file` field of a `DeferredFdCloserInner` from null to
|
||||||
|
// non-null. This doesn't break the type invariant for `DeferredFdCloserInner` because we
|
||||||
|
// still own a refcount to the file, so we can pass ownership of that refcount to the
|
||||||
|
// `DeferredFdCloserInner`.
|
||||||
|
//
|
||||||
|
// When `do_close_fd` runs, it must be safe for it to `fput` the refcount. However, this is
|
||||||
|
// the case because all light refcounts that are associated with the fd we closed
|
||||||
|
// previously must be dropped when `do_close_fd`, since light refcounts must be dropped
|
||||||
|
// before returning to userspace.
|
||||||
|
//
|
||||||
|
// SAFETY: Task works are executed on the current thread right before we return to
|
||||||
|
// userspace, so this write is guaranteed to happen before `do_close_fd` is called, which
|
||||||
|
// means that a race is not possible here.
|
||||||
|
unsafe { *file_field = file };
|
||||||
|
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// # Safety
|
||||||
|
///
|
||||||
|
/// The provided pointer must point at the `twork` field of a `DeferredFdCloserInner` stored in
|
||||||
|
/// a `KBox`, and the caller must pass exclusive ownership of that `KBox`. Furthermore, if the
|
||||||
|
/// file pointer is non-null, then it must be okay to release the refcount by calling `fput`.
|
||||||
|
unsafe extern "C" fn do_close_fd(inner: *mut bindings::callback_head) {
|
||||||
|
// SAFETY: The caller just passed us ownership of this box.
|
||||||
|
let inner = unsafe { KBox::from_raw(inner.cast::<DeferredFdCloserInner>()) };
|
||||||
|
if !inner.file.is_null() {
|
||||||
|
// SAFETY: By the type invariants, we own a refcount to this file, and the caller
|
||||||
|
// guarantees that dropping the refcount now is okay.
|
||||||
|
unsafe { bindings::fput(inner.file) };
|
||||||
|
}
|
||||||
|
// The allocation is freed when `inner` goes out of scope.
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Represents a failure to close an fd in a deferred manner.
|
||||||
|
#[derive(Copy, Clone, Debug, Eq, PartialEq)]
|
||||||
|
pub(crate) enum DeferredFdCloseError {
|
||||||
|
/// Closing the fd failed because we were unable to schedule a task work.
|
||||||
|
TaskWorkUnavailable,
|
||||||
|
/// Closing the fd failed because the fd does not exist.
|
||||||
|
BadFd,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl From<DeferredFdCloseError> for Error {
|
||||||
|
fn from(err: DeferredFdCloseError) -> Error {
|
||||||
|
match err {
|
||||||
|
DeferredFdCloseError::TaskWorkUnavailable => ESRCH,
|
||||||
|
DeferredFdCloseError::BadFd => EBADF,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,182 @@
|
|||||||
|
// SPDX-License-Identifier: GPL-2.0
|
||||||
|
|
||||||
|
// Copyright (C) 2025 Google LLC.
|
||||||
|
|
||||||
|
use core::mem::MaybeUninit;
|
||||||
|
use core::ops::{Deref, DerefMut};
|
||||||
|
use kernel::{
|
||||||
|
transmute::{AsBytes, FromBytes},
|
||||||
|
uapi::{self, *},
|
||||||
|
};
|
||||||
|
|
||||||
|
macro_rules! pub_no_prefix {
|
||||||
|
($prefix:ident, $($newname:ident),+ $(,)?) => {
|
||||||
|
$(pub(crate) const $newname: u32 = kernel::macros::concat_idents!($prefix, $newname);)+
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
pub_no_prefix!(
|
||||||
|
binder_driver_return_protocol_,
|
||||||
|
BR_TRANSACTION,
|
||||||
|
BR_TRANSACTION_SEC_CTX,
|
||||||
|
BR_REPLY,
|
||||||
|
BR_DEAD_REPLY,
|
||||||
|
BR_FAILED_REPLY,
|
||||||
|
BR_FROZEN_REPLY,
|
||||||
|
BR_NOOP,
|
||||||
|
BR_SPAWN_LOOPER,
|
||||||
|
BR_TRANSACTION_COMPLETE,
|
||||||
|
BR_TRANSACTION_PENDING_FROZEN,
|
||||||
|
BR_ONEWAY_SPAM_SUSPECT,
|
||||||
|
BR_OK,
|
||||||
|
BR_ERROR,
|
||||||
|
BR_INCREFS,
|
||||||
|
BR_ACQUIRE,
|
||||||
|
BR_RELEASE,
|
||||||
|
BR_DECREFS,
|
||||||
|
BR_DEAD_BINDER,
|
||||||
|
BR_CLEAR_DEATH_NOTIFICATION_DONE,
|
||||||
|
BR_FROZEN_BINDER,
|
||||||
|
BR_CLEAR_FREEZE_NOTIFICATION_DONE,
|
||||||
|
);
|
||||||
|
|
||||||
|
pub_no_prefix!(
|
||||||
|
binder_driver_command_protocol_,
|
||||||
|
BC_TRANSACTION,
|
||||||
|
BC_TRANSACTION_SG,
|
||||||
|
BC_REPLY,
|
||||||
|
BC_REPLY_SG,
|
||||||
|
BC_FREE_BUFFER,
|
||||||
|
BC_ENTER_LOOPER,
|
||||||
|
BC_EXIT_LOOPER,
|
||||||
|
BC_REGISTER_LOOPER,
|
||||||
|
BC_INCREFS,
|
||||||
|
BC_ACQUIRE,
|
||||||
|
BC_RELEASE,
|
||||||
|
BC_DECREFS,
|
||||||
|
BC_INCREFS_DONE,
|
||||||
|
BC_ACQUIRE_DONE,
|
||||||
|
BC_REQUEST_DEATH_NOTIFICATION,
|
||||||
|
BC_CLEAR_DEATH_NOTIFICATION,
|
||||||
|
BC_DEAD_BINDER_DONE,
|
||||||
|
BC_REQUEST_FREEZE_NOTIFICATION,
|
||||||
|
BC_CLEAR_FREEZE_NOTIFICATION,
|
||||||
|
BC_FREEZE_NOTIFICATION_DONE,
|
||||||
|
);
|
||||||
|
|
||||||
|
pub_no_prefix!(
|
||||||
|
flat_binder_object_flags_,
|
||||||
|
FLAT_BINDER_FLAG_ACCEPTS_FDS,
|
||||||
|
FLAT_BINDER_FLAG_TXN_SECURITY_CTX
|
||||||
|
);
|
||||||
|
|
||||||
|
pub_no_prefix!(
|
||||||
|
transaction_flags_,
|
||||||
|
TF_ONE_WAY,
|
||||||
|
TF_ACCEPT_FDS,
|
||||||
|
TF_CLEAR_BUF,
|
||||||
|
TF_UPDATE_TXN
|
||||||
|
);
|
||||||
|
|
||||||
|
pub(crate) use uapi::{
|
||||||
|
BINDER_TYPE_BINDER, BINDER_TYPE_FD, BINDER_TYPE_FDA, BINDER_TYPE_HANDLE, BINDER_TYPE_PTR,
|
||||||
|
BINDER_TYPE_WEAK_BINDER, BINDER_TYPE_WEAK_HANDLE,
|
||||||
|
};
|
||||||
|
|
||||||
|
macro_rules! decl_wrapper {
|
||||||
|
($newname:ident, $wrapped:ty) => {
|
||||||
|
// Define a wrapper around the C type. Use `MaybeUninit` to enforce that the value of
|
||||||
|
// padding bytes must be preserved.
|
||||||
|
#[derive(Copy, Clone)]
|
||||||
|
#[repr(transparent)]
|
||||||
|
pub(crate) struct $newname(MaybeUninit<$wrapped>);
|
||||||
|
|
||||||
|
// SAFETY: This macro is only used with types where this is ok.
|
||||||
|
unsafe impl FromBytes for $newname {}
|
||||||
|
// SAFETY: This macro is only used with types where this is ok.
|
||||||
|
unsafe impl AsBytes for $newname {}
|
||||||
|
|
||||||
|
impl Deref for $newname {
|
||||||
|
type Target = $wrapped;
|
||||||
|
fn deref(&self) -> &Self::Target {
|
||||||
|
// SAFETY: We use `MaybeUninit` only to preserve padding. The value must still
|
||||||
|
// always be valid.
|
||||||
|
unsafe { self.0.assume_init_ref() }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl DerefMut for $newname {
|
||||||
|
fn deref_mut(&mut self) -> &mut Self::Target {
|
||||||
|
// SAFETY: We use `MaybeUninit` only to preserve padding. The value must still
|
||||||
|
// always be valid.
|
||||||
|
unsafe { self.0.assume_init_mut() }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Default for $newname {
|
||||||
|
fn default() -> Self {
|
||||||
|
// Create a new value of this type where all bytes (including padding) are zeroed.
|
||||||
|
Self(MaybeUninit::zeroed())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
decl_wrapper!(BinderNodeDebugInfo, uapi::binder_node_debug_info);
|
||||||
|
decl_wrapper!(BinderNodeInfoForRef, uapi::binder_node_info_for_ref);
|
||||||
|
decl_wrapper!(FlatBinderObject, uapi::flat_binder_object);
|
||||||
|
decl_wrapper!(BinderFdObject, uapi::binder_fd_object);
|
||||||
|
decl_wrapper!(BinderFdArrayObject, uapi::binder_fd_array_object);
|
||||||
|
decl_wrapper!(BinderObjectHeader, uapi::binder_object_header);
|
||||||
|
decl_wrapper!(BinderBufferObject, uapi::binder_buffer_object);
|
||||||
|
decl_wrapper!(BinderTransactionData, uapi::binder_transaction_data);
|
||||||
|
decl_wrapper!(
|
||||||
|
BinderTransactionDataSecctx,
|
||||||
|
uapi::binder_transaction_data_secctx
|
||||||
|
);
|
||||||
|
decl_wrapper!(BinderTransactionDataSg, uapi::binder_transaction_data_sg);
|
||||||
|
decl_wrapper!(BinderWriteRead, uapi::binder_write_read);
|
||||||
|
decl_wrapper!(BinderVersion, uapi::binder_version);
|
||||||
|
decl_wrapper!(BinderFrozenStatusInfo, uapi::binder_frozen_status_info);
|
||||||
|
decl_wrapper!(BinderFreezeInfo, uapi::binder_freeze_info);
|
||||||
|
decl_wrapper!(BinderFrozenStateInfo, uapi::binder_frozen_state_info);
|
||||||
|
decl_wrapper!(BinderHandleCookie, uapi::binder_handle_cookie);
|
||||||
|
decl_wrapper!(ExtendedError, uapi::binder_extended_error);
|
||||||
|
|
||||||
|
impl BinderVersion {
|
||||||
|
pub(crate) fn current() -> Self {
|
||||||
|
Self(MaybeUninit::new(uapi::binder_version {
|
||||||
|
protocol_version: BINDER_CURRENT_PROTOCOL_VERSION as _,
|
||||||
|
}))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl BinderTransactionData {
|
||||||
|
pub(crate) fn with_buffers_size(self, buffers_size: u64) -> BinderTransactionDataSg {
|
||||||
|
BinderTransactionDataSg(MaybeUninit::new(uapi::binder_transaction_data_sg {
|
||||||
|
transaction_data: *self,
|
||||||
|
buffers_size,
|
||||||
|
}))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl BinderTransactionDataSecctx {
|
||||||
|
/// View the inner data as wrapped in `BinderTransactionData`.
|
||||||
|
pub(crate) fn tr_data(&mut self) -> &mut BinderTransactionData {
|
||||||
|
// SAFETY: Transparent wrapper is safe to transmute.
|
||||||
|
unsafe {
|
||||||
|
&mut *(&mut self.transaction_data as *mut uapi::binder_transaction_data
|
||||||
|
as *mut BinderTransactionData)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl ExtendedError {
|
||||||
|
pub(crate) fn new(id: u32, command: u32, param: i32) -> Self {
|
||||||
|
Self(MaybeUninit::new(uapi::binder_extended_error {
|
||||||
|
id,
|
||||||
|
command,
|
||||||
|
param,
|
||||||
|
}))
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,99 @@
|
|||||||
|
// SPDX-License-Identifier: GPL-2.0
|
||||||
|
|
||||||
|
// Copyright (C) 2025 Google LLC.
|
||||||
|
|
||||||
|
use kernel::prelude::*;
|
||||||
|
|
||||||
|
use crate::defs::*;
|
||||||
|
|
||||||
|
pub(crate) type BinderResult<T = ()> = core::result::Result<T, BinderError>;
|
||||||
|
|
||||||
|
/// An error that will be returned to userspace via the `BINDER_WRITE_READ` ioctl rather than via
|
||||||
|
/// errno.
|
||||||
|
pub(crate) struct BinderError {
|
||||||
|
pub(crate) reply: u32,
|
||||||
|
source: Option<Error>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl BinderError {
|
||||||
|
pub(crate) fn new_dead() -> Self {
|
||||||
|
Self {
|
||||||
|
reply: BR_DEAD_REPLY,
|
||||||
|
source: None,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
pub(crate) fn new_frozen() -> Self {
|
||||||
|
Self {
|
||||||
|
reply: BR_FROZEN_REPLY,
|
||||||
|
source: None,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
pub(crate) fn new_frozen_oneway() -> Self {
|
||||||
|
Self {
|
||||||
|
reply: BR_TRANSACTION_PENDING_FROZEN,
|
||||||
|
source: None,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
pub(crate) fn is_dead(&self) -> bool {
|
||||||
|
self.reply == BR_DEAD_REPLY
|
||||||
|
}
|
||||||
|
|
||||||
|
pub(crate) fn as_errno(&self) -> kernel::ffi::c_int {
|
||||||
|
self.source.unwrap_or(EINVAL).to_errno()
|
||||||
|
}
|
||||||
|
|
||||||
|
pub(crate) fn should_pr_warn(&self) -> bool {
|
||||||
|
self.source.is_some()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Convert an errno into a `BinderError` and store the errno used to construct it. The errno
|
||||||
|
/// should be stored as the thread's extended error when given to userspace.
|
||||||
|
impl From<Error> for BinderError {
|
||||||
|
fn from(source: Error) -> Self {
|
||||||
|
Self {
|
||||||
|
reply: BR_FAILED_REPLY,
|
||||||
|
source: Some(source),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl From<kernel::fs::file::BadFdError> for BinderError {
|
||||||
|
fn from(source: kernel::fs::file::BadFdError) -> Self {
|
||||||
|
BinderError::from(Error::from(source))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl From<kernel::alloc::AllocError> for BinderError {
|
||||||
|
fn from(_: kernel::alloc::AllocError) -> Self {
|
||||||
|
Self {
|
||||||
|
reply: BR_FAILED_REPLY,
|
||||||
|
source: Some(ENOMEM),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl core::fmt::Debug for BinderError {
|
||||||
|
fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
|
||||||
|
match self.reply {
|
||||||
|
BR_FAILED_REPLY => match self.source.as_ref() {
|
||||||
|
Some(source) => f
|
||||||
|
.debug_struct("BR_FAILED_REPLY")
|
||||||
|
.field("source", source)
|
||||||
|
.finish(),
|
||||||
|
None => f.pad("BR_FAILED_REPLY"),
|
||||||
|
},
|
||||||
|
BR_DEAD_REPLY => f.pad("BR_DEAD_REPLY"),
|
||||||
|
BR_FROZEN_REPLY => f.pad("BR_FROZEN_REPLY"),
|
||||||
|
BR_TRANSACTION_PENDING_FROZEN => f.pad("BR_TRANSACTION_PENDING_FROZEN"),
|
||||||
|
BR_TRANSACTION_COMPLETE => f.pad("BR_TRANSACTION_COMPLETE"),
|
||||||
|
_ => f
|
||||||
|
.debug_struct("BinderError")
|
||||||
|
.field("reply", &self.reply)
|
||||||
|
.finish(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,388 @@
|
|||||||
|
// SPDX-License-Identifier: GPL-2.0
|
||||||
|
|
||||||
|
// Copyright (C) 2025 Google LLC.
|
||||||
|
|
||||||
|
use kernel::{
|
||||||
|
alloc::AllocError,
|
||||||
|
list::ListArc,
|
||||||
|
prelude::*,
|
||||||
|
rbtree::{self, RBTreeNodeReservation},
|
||||||
|
seq_file::SeqFile,
|
||||||
|
seq_print,
|
||||||
|
sync::{Arc, UniqueArc},
|
||||||
|
uaccess::UserSliceReader,
|
||||||
|
};
|
||||||
|
|
||||||
|
use crate::{
|
||||||
|
defs::*, node::Node, process::Process, thread::Thread, BinderReturnWriter, DArc, DLArc,
|
||||||
|
DTRWrap, DeliverToRead,
|
||||||
|
};
|
||||||
|
|
||||||
|
#[derive(Clone, Copy, Eq, PartialEq, Ord, PartialOrd)]
|
||||||
|
pub(crate) struct FreezeCookie(u64);
|
||||||
|
|
||||||
|
/// Represents a listener for changes to the frozen state of a process.
|
||||||
|
pub(crate) struct FreezeListener {
|
||||||
|
/// The node we are listening for.
|
||||||
|
pub(crate) node: DArc<Node>,
|
||||||
|
/// The cookie of this freeze listener.
|
||||||
|
cookie: FreezeCookie,
|
||||||
|
/// What value of `is_frozen` did we most recently tell userspace about?
|
||||||
|
last_is_frozen: Option<bool>,
|
||||||
|
/// We sent a `BR_FROZEN_BINDER` and we are waiting for `BC_FREEZE_NOTIFICATION_DONE` before
|
||||||
|
/// sending any other commands.
|
||||||
|
is_pending: bool,
|
||||||
|
/// Userspace sent `BC_CLEAR_FREEZE_NOTIFICATION` and we need to reply with
|
||||||
|
/// `BR_CLEAR_FREEZE_NOTIFICATION_DONE` as soon as possible. If `is_pending` is set, then we
|
||||||
|
/// must wait for it to be unset before we can reply.
|
||||||
|
is_clearing: bool,
|
||||||
|
/// Number of cleared duplicates that can't be deleted until userspace sends
|
||||||
|
/// `BC_FREEZE_NOTIFICATION_DONE`.
|
||||||
|
num_pending_duplicates: u64,
|
||||||
|
/// Number of cleared duplicates that can be deleted.
|
||||||
|
num_cleared_duplicates: u64,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl FreezeListener {
|
||||||
|
/// Is it okay to create a new listener with the same cookie as this one for the provided node?
|
||||||
|
///
|
||||||
|
/// Under some scenarios, userspace may delete a freeze listener and immediately recreate it
|
||||||
|
/// with the same cookie. This results in duplicate listeners. To avoid issues with ambiguity,
|
||||||
|
/// we allow this only if the new listener is for the same node, and we also require that the
|
||||||
|
/// old listener has already been cleared.
|
||||||
|
fn allow_duplicate(&self, node: &DArc<Node>) -> bool {
|
||||||
|
Arc::ptr_eq(&self.node, node) && self.is_clearing
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
type UninitFM = UniqueArc<core::mem::MaybeUninit<DTRWrap<FreezeMessage>>>;
|
||||||
|
|
||||||
|
/// Represents a notification that the freeze state has changed.
|
||||||
|
pub(crate) struct FreezeMessage {
|
||||||
|
cookie: FreezeCookie,
|
||||||
|
}
|
||||||
|
|
||||||
|
kernel::list::impl_list_arc_safe! {
|
||||||
|
impl ListArcSafe<0> for FreezeMessage {
|
||||||
|
untracked;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl FreezeMessage {
|
||||||
|
fn new(flags: kernel::alloc::Flags) -> Result<UninitFM, AllocError> {
|
||||||
|
UniqueArc::new_uninit(flags)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn init(ua: UninitFM, cookie: FreezeCookie) -> DLArc<FreezeMessage> {
|
||||||
|
match ua.pin_init_with(DTRWrap::new(FreezeMessage { cookie })) {
|
||||||
|
Ok(msg) => ListArc::from(msg),
|
||||||
|
Err(err) => match err {},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl DeliverToRead for FreezeMessage {
|
||||||
|
fn do_work(
|
||||||
|
self: DArc<Self>,
|
||||||
|
thread: &Thread,
|
||||||
|
writer: &mut BinderReturnWriter<'_>,
|
||||||
|
) -> Result<bool> {
|
||||||
|
let _removed_listener;
|
||||||
|
let mut node_refs = thread.process.node_refs.lock();
|
||||||
|
let Some(mut freeze_entry) = node_refs.freeze_listeners.find_mut(&self.cookie) else {
|
||||||
|
return Ok(true);
|
||||||
|
};
|
||||||
|
let freeze = freeze_entry.get_mut();
|
||||||
|
|
||||||
|
if freeze.num_cleared_duplicates > 0 {
|
||||||
|
freeze.num_cleared_duplicates -= 1;
|
||||||
|
drop(node_refs);
|
||||||
|
writer.write_code(BR_CLEAR_FREEZE_NOTIFICATION_DONE)?;
|
||||||
|
writer.write_payload(&self.cookie.0)?;
|
||||||
|
return Ok(true);
|
||||||
|
}
|
||||||
|
|
||||||
|
if freeze.is_pending {
|
||||||
|
return Ok(true);
|
||||||
|
}
|
||||||
|
if freeze.is_clearing {
|
||||||
|
_removed_listener = freeze_entry.remove_node();
|
||||||
|
drop(node_refs);
|
||||||
|
writer.write_code(BR_CLEAR_FREEZE_NOTIFICATION_DONE)?;
|
||||||
|
writer.write_payload(&self.cookie.0)?;
|
||||||
|
Ok(true)
|
||||||
|
} else {
|
||||||
|
let is_frozen = freeze.node.owner.inner.lock().is_frozen;
|
||||||
|
if freeze.last_is_frozen == Some(is_frozen) {
|
||||||
|
return Ok(true);
|
||||||
|
}
|
||||||
|
|
||||||
|
let mut state_info = BinderFrozenStateInfo::default();
|
||||||
|
state_info.is_frozen = is_frozen as u32;
|
||||||
|
state_info.cookie = freeze.cookie.0;
|
||||||
|
freeze.is_pending = true;
|
||||||
|
freeze.last_is_frozen = Some(is_frozen);
|
||||||
|
drop(node_refs);
|
||||||
|
|
||||||
|
writer.write_code(BR_FROZEN_BINDER)?;
|
||||||
|
writer.write_payload(&state_info)?;
|
||||||
|
// BR_FROZEN_BINDER notifications can cause transactions
|
||||||
|
Ok(false)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn cancel(self: DArc<Self>) {}
|
||||||
|
|
||||||
|
fn should_sync_wakeup(&self) -> bool {
|
||||||
|
false
|
||||||
|
}
|
||||||
|
|
||||||
|
#[inline(never)]
|
||||||
|
fn debug_print(&self, m: &SeqFile, prefix: &str, _tprefix: &str) -> Result<()> {
|
||||||
|
seq_print!(m, "{}has frozen binder\n", prefix);
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl FreezeListener {
|
||||||
|
pub(crate) fn on_process_exit(&self, proc: &Arc<Process>) {
|
||||||
|
if !self.is_clearing {
|
||||||
|
self.node.remove_freeze_listener(proc);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Process {
|
||||||
|
pub(crate) fn request_freeze_notif(
|
||||||
|
self: &Arc<Self>,
|
||||||
|
reader: &mut UserSliceReader,
|
||||||
|
) -> Result<()> {
|
||||||
|
let hc = reader.read::<BinderHandleCookie>()?;
|
||||||
|
let handle = hc.handle;
|
||||||
|
let cookie = FreezeCookie(hc.cookie);
|
||||||
|
|
||||||
|
let msg = FreezeMessage::new(GFP_KERNEL)?;
|
||||||
|
let alloc = RBTreeNodeReservation::new(GFP_KERNEL)?;
|
||||||
|
|
||||||
|
let mut node_refs_guard = self.node_refs.lock();
|
||||||
|
let node_refs = &mut *node_refs_guard;
|
||||||
|
let Some(info) = node_refs.by_handle.get_mut(&handle) else {
|
||||||
|
pr_warn!("BC_REQUEST_FREEZE_NOTIFICATION invalid ref {}\n", handle);
|
||||||
|
return Err(EINVAL);
|
||||||
|
};
|
||||||
|
if info.freeze().is_some() {
|
||||||
|
pr_warn!("BC_REQUEST_FREEZE_NOTIFICATION already set\n");
|
||||||
|
return Err(EINVAL);
|
||||||
|
}
|
||||||
|
let node_ref = info.node_ref();
|
||||||
|
let freeze_entry = node_refs.freeze_listeners.entry(cookie);
|
||||||
|
|
||||||
|
if let rbtree::Entry::Occupied(ref dupe) = freeze_entry {
|
||||||
|
if !dupe.get().allow_duplicate(&node_ref.node) {
|
||||||
|
pr_warn!("BC_REQUEST_FREEZE_NOTIFICATION duplicate cookie\n");
|
||||||
|
return Err(EINVAL);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// All failure paths must come before this call, and all modifications must come after this
|
||||||
|
// call.
|
||||||
|
node_ref.node.add_freeze_listener(self, GFP_KERNEL)?;
|
||||||
|
|
||||||
|
match freeze_entry {
|
||||||
|
rbtree::Entry::Vacant(entry) => {
|
||||||
|
entry.insert(
|
||||||
|
FreezeListener {
|
||||||
|
cookie,
|
||||||
|
node: node_ref.node.clone(),
|
||||||
|
last_is_frozen: None,
|
||||||
|
is_pending: false,
|
||||||
|
is_clearing: false,
|
||||||
|
num_pending_duplicates: 0,
|
||||||
|
num_cleared_duplicates: 0,
|
||||||
|
},
|
||||||
|
alloc,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
rbtree::Entry::Occupied(mut dupe) => {
|
||||||
|
let dupe = dupe.get_mut();
|
||||||
|
if dupe.is_pending {
|
||||||
|
dupe.num_pending_duplicates += 1;
|
||||||
|
} else {
|
||||||
|
dupe.num_cleared_duplicates += 1;
|
||||||
|
}
|
||||||
|
dupe.last_is_frozen = None;
|
||||||
|
dupe.is_pending = false;
|
||||||
|
dupe.is_clearing = false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
*info.freeze() = Some(cookie);
|
||||||
|
let msg = FreezeMessage::init(msg, cookie);
|
||||||
|
drop(node_refs_guard);
|
||||||
|
let _ = self.push_work(msg);
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
pub(crate) fn freeze_notif_done(self: &Arc<Self>, reader: &mut UserSliceReader) -> Result<()> {
|
||||||
|
let cookie = FreezeCookie(reader.read()?);
|
||||||
|
let alloc = FreezeMessage::new(GFP_KERNEL)?;
|
||||||
|
let mut node_refs_guard = self.node_refs.lock();
|
||||||
|
let node_refs = &mut *node_refs_guard;
|
||||||
|
let Some(freeze) = node_refs.freeze_listeners.get_mut(&cookie) else {
|
||||||
|
pr_warn!("BC_FREEZE_NOTIFICATION_DONE {:016x} not found\n", cookie.0);
|
||||||
|
return Err(EINVAL);
|
||||||
|
};
|
||||||
|
let mut clear_msg = None;
|
||||||
|
if freeze.num_pending_duplicates > 0 {
|
||||||
|
clear_msg = Some(FreezeMessage::init(alloc, cookie));
|
||||||
|
freeze.num_pending_duplicates -= 1;
|
||||||
|
freeze.num_cleared_duplicates += 1;
|
||||||
|
} else {
|
||||||
|
if !freeze.is_pending {
|
||||||
|
pr_warn!(
|
||||||
|
"BC_FREEZE_NOTIFICATION_DONE {:016x} not pending\n",
|
||||||
|
cookie.0
|
||||||
|
);
|
||||||
|
return Err(EINVAL);
|
||||||
|
}
|
||||||
|
if freeze.is_clearing {
|
||||||
|
// Immediately send another FreezeMessage for BR_CLEAR_FREEZE_NOTIFICATION_DONE.
|
||||||
|
clear_msg = Some(FreezeMessage::init(alloc, cookie));
|
||||||
|
}
|
||||||
|
freeze.is_pending = false;
|
||||||
|
}
|
||||||
|
drop(node_refs_guard);
|
||||||
|
if let Some(clear_msg) = clear_msg {
|
||||||
|
let _ = self.push_work(clear_msg);
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
pub(crate) fn clear_freeze_notif(self: &Arc<Self>, reader: &mut UserSliceReader) -> Result<()> {
|
||||||
|
let hc = reader.read::<BinderHandleCookie>()?;
|
||||||
|
let handle = hc.handle;
|
||||||
|
let cookie = FreezeCookie(hc.cookie);
|
||||||
|
|
||||||
|
let alloc = FreezeMessage::new(GFP_KERNEL)?;
|
||||||
|
let mut node_refs_guard = self.node_refs.lock();
|
||||||
|
let node_refs = &mut *node_refs_guard;
|
||||||
|
let Some(info) = node_refs.by_handle.get_mut(&handle) else {
|
||||||
|
pr_warn!("BC_CLEAR_FREEZE_NOTIFICATION invalid ref {}\n", handle);
|
||||||
|
return Err(EINVAL);
|
||||||
|
};
|
||||||
|
let Some(info_cookie) = info.freeze() else {
|
||||||
|
pr_warn!("BC_CLEAR_FREEZE_NOTIFICATION freeze notification not active\n");
|
||||||
|
return Err(EINVAL);
|
||||||
|
};
|
||||||
|
if *info_cookie != cookie {
|
||||||
|
pr_warn!("BC_CLEAR_FREEZE_NOTIFICATION freeze notification cookie mismatch\n");
|
||||||
|
return Err(EINVAL);
|
||||||
|
}
|
||||||
|
let Some(listener) = node_refs.freeze_listeners.get_mut(&cookie) else {
|
||||||
|
pr_warn!("BC_CLEAR_FREEZE_NOTIFICATION invalid cookie {}\n", handle);
|
||||||
|
return Err(EINVAL);
|
||||||
|
};
|
||||||
|
listener.is_clearing = true;
|
||||||
|
listener.node.remove_freeze_listener(self);
|
||||||
|
*info.freeze() = None;
|
||||||
|
let mut msg = None;
|
||||||
|
if !listener.is_pending {
|
||||||
|
msg = Some(FreezeMessage::init(alloc, cookie));
|
||||||
|
}
|
||||||
|
drop(node_refs_guard);
|
||||||
|
|
||||||
|
if let Some(msg) = msg {
|
||||||
|
let _ = self.push_work(msg);
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
fn get_freeze_cookie(&self, node: &DArc<Node>) -> Option<FreezeCookie> {
|
||||||
|
let node_refs = &mut *self.node_refs.lock();
|
||||||
|
let handle = node_refs.by_node.get(&node.global_id())?;
|
||||||
|
let node_ref = node_refs.by_handle.get_mut(handle)?;
|
||||||
|
*node_ref.freeze()
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Creates a vector of every freeze listener on this process.
|
||||||
|
///
|
||||||
|
/// Returns pairs of the remote process listening for notifications and the local node it is
|
||||||
|
/// listening on.
|
||||||
|
#[expect(clippy::type_complexity)]
|
||||||
|
fn find_freeze_recipients(&self) -> Result<KVVec<(DArc<Node>, Arc<Process>)>, AllocError> {
|
||||||
|
// Defined before `inner` to drop after releasing spinlock if `push_within_capacity` fails.
|
||||||
|
let mut node_proc_pair;
|
||||||
|
|
||||||
|
// We pre-allocate space for up to 8 recipients before we take the spinlock. However, if
|
||||||
|
// the allocation fails, use a vector with a capacity of zero instead of failing. After
|
||||||
|
// all, there might not be any freeze listeners, in which case this operation could still
|
||||||
|
// succeed.
|
||||||
|
let mut recipients =
|
||||||
|
KVVec::with_capacity(8, GFP_KERNEL).unwrap_or_else(|_err| KVVec::new());
|
||||||
|
|
||||||
|
let mut inner = self.lock_with_nodes();
|
||||||
|
let mut curr = inner.nodes.cursor_front();
|
||||||
|
while let Some(cursor) = curr {
|
||||||
|
let (key, node) = cursor.current();
|
||||||
|
let key = *key;
|
||||||
|
let list = node.freeze_list(&inner.inner);
|
||||||
|
let len = list.len();
|
||||||
|
|
||||||
|
if recipients.spare_capacity_mut().len() < len {
|
||||||
|
drop(inner);
|
||||||
|
recipients.reserve(len, GFP_KERNEL)?;
|
||||||
|
inner = self.lock_with_nodes();
|
||||||
|
// Find the node we were looking at and try again. If the set of nodes was changed,
|
||||||
|
// then just proceed to the next node. This is ok because we don't guarantee the
|
||||||
|
// inclusion of nodes that are added or removed in parallel with this operation.
|
||||||
|
curr = inner.nodes.cursor_lower_bound(&key);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
for proc in list {
|
||||||
|
node_proc_pair = (node.clone(), proc.clone());
|
||||||
|
recipients
|
||||||
|
.push_within_capacity(node_proc_pair)
|
||||||
|
.map_err(|_| {
|
||||||
|
pr_err!(
|
||||||
|
"push_within_capacity failed even though we checked the capacity\n"
|
||||||
|
);
|
||||||
|
AllocError
|
||||||
|
})?;
|
||||||
|
}
|
||||||
|
|
||||||
|
curr = cursor.move_next();
|
||||||
|
}
|
||||||
|
Ok(recipients)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Prepare allocations for sending freeze messages.
|
||||||
|
pub(crate) fn prepare_freeze_messages(&self) -> Result<FreezeMessages, AllocError> {
|
||||||
|
let recipients = self.find_freeze_recipients()?;
|
||||||
|
let mut batch = KVVec::with_capacity(recipients.len(), GFP_KERNEL)?;
|
||||||
|
for (node, proc) in recipients {
|
||||||
|
let Some(cookie) = proc.get_freeze_cookie(&node) else {
|
||||||
|
// If the freeze listener was removed in the meantime, just discard the
|
||||||
|
// notification.
|
||||||
|
continue;
|
||||||
|
};
|
||||||
|
let msg_alloc = FreezeMessage::new(GFP_KERNEL)?;
|
||||||
|
let msg = FreezeMessage::init(msg_alloc, cookie);
|
||||||
|
batch.push((proc, msg), GFP_KERNEL)?;
|
||||||
|
}
|
||||||
|
|
||||||
|
Ok(FreezeMessages { batch })
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
pub(crate) struct FreezeMessages {
|
||||||
|
batch: KVVec<(Arc<Process>, DLArc<FreezeMessage>)>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl FreezeMessages {
|
||||||
|
pub(crate) fn send_messages(self) {
|
||||||
|
for (proc, msg) in self.batch {
|
||||||
|
let _ = proc.push_work(msg);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,78 @@
|
|||||||
|
// SPDX-License-Identifier: GPL-2.0
|
||||||
|
|
||||||
|
// Copyright (C) 2025 Google LLC.
|
||||||
|
|
||||||
|
use kernel::{list::ListArc, prelude::*, seq_file::SeqFile, seq_print, sync::UniqueArc};
|
||||||
|
|
||||||
|
use crate::{node::Node, thread::Thread, BinderReturnWriter, DArc, DLArc, DTRWrap, DeliverToRead};
|
||||||
|
|
||||||
|
use core::mem::MaybeUninit;
|
||||||
|
|
||||||
|
pub(crate) struct CritIncrWrapper {
|
||||||
|
inner: UniqueArc<MaybeUninit<DTRWrap<NodeWrapper>>>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl CritIncrWrapper {
|
||||||
|
pub(crate) fn new() -> Result<Self> {
|
||||||
|
Ok(CritIncrWrapper {
|
||||||
|
inner: UniqueArc::new_uninit(GFP_KERNEL)?,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
pub(super) fn init(self, node: DArc<Node>) -> DLArc<dyn DeliverToRead> {
|
||||||
|
match self.inner.pin_init_with(DTRWrap::new(NodeWrapper { node })) {
|
||||||
|
Ok(initialized) => ListArc::from(initialized) as _,
|
||||||
|
Err(err) => match err {},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
struct NodeWrapper {
|
||||||
|
node: DArc<Node>,
|
||||||
|
}
|
||||||
|
|
||||||
|
kernel::list::impl_list_arc_safe! {
|
||||||
|
impl ListArcSafe<0> for NodeWrapper {
|
||||||
|
untracked;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl DeliverToRead for NodeWrapper {
|
||||||
|
fn do_work(
|
||||||
|
self: DArc<Self>,
|
||||||
|
_thread: &Thread,
|
||||||
|
writer: &mut BinderReturnWriter<'_>,
|
||||||
|
) -> Result<bool> {
|
||||||
|
let node = &self.node;
|
||||||
|
let mut owner_inner = node.owner.inner.lock();
|
||||||
|
let inner = node.inner.access_mut(&mut owner_inner);
|
||||||
|
|
||||||
|
let ds = &mut inner.delivery_state;
|
||||||
|
|
||||||
|
assert!(ds.has_pushed_wrapper);
|
||||||
|
assert!(ds.has_strong_zero2one);
|
||||||
|
ds.has_pushed_wrapper = false;
|
||||||
|
ds.has_strong_zero2one = false;
|
||||||
|
|
||||||
|
node.do_work_locked(writer, owner_inner)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn cancel(self: DArc<Self>) {}
|
||||||
|
|
||||||
|
fn should_sync_wakeup(&self) -> bool {
|
||||||
|
false
|
||||||
|
}
|
||||||
|
|
||||||
|
#[inline(never)]
|
||||||
|
fn debug_print(&self, m: &SeqFile, prefix: &str, _tprefix: &str) -> Result<()> {
|
||||||
|
seq_print!(
|
||||||
|
m,
|
||||||
|
"{}node work {}: u{:016x} c{:016x}\n",
|
||||||
|
prefix,
|
||||||
|
self.node.debug_id,
|
||||||
|
self.node.ptr,
|
||||||
|
self.node.cookie,
|
||||||
|
);
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
}
|
||||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,24 @@
|
|||||||
|
// SPDX-License-Identifier: GPL-2.0
|
||||||
|
|
||||||
|
/* C helper for page_range.rs to work around a CFI violation.
|
||||||
|
*
|
||||||
|
* Bindgen currently pretends that `enum lru_status` is the same as an integer.
|
||||||
|
* This assumption is fine ABI-wise, but once you add CFI to the mix, it
|
||||||
|
* triggers a CFI violation because `enum lru_status` gets a different CFI tag.
|
||||||
|
*
|
||||||
|
* This file contains a workaround until bindgen can be fixed.
|
||||||
|
*
|
||||||
|
* Copyright (C) 2025 Google LLC.
|
||||||
|
*/
|
||||||
|
#include "page_range_helper.h"
|
||||||
|
|
||||||
|
unsigned int rust_shrink_free_page(struct list_head *item,
|
||||||
|
struct list_lru_one *list,
|
||||||
|
void *cb_arg);
|
||||||
|
|
||||||
|
enum lru_status
|
||||||
|
rust_shrink_free_page_wrap(struct list_head *item, struct list_lru_one *list,
|
||||||
|
void *cb_arg)
|
||||||
|
{
|
||||||
|
return rust_shrink_free_page(item, list, cb_arg);
|
||||||
|
}
|
||||||
@@ -0,0 +1,15 @@
|
|||||||
|
/* SPDX-License-Identifier: GPL-2.0 */
|
||||||
|
/*
|
||||||
|
* Copyright (C) 2025 Google, Inc.
|
||||||
|
*/
|
||||||
|
|
||||||
|
#ifndef _LINUX_PAGE_RANGE_HELPER_H
|
||||||
|
#define _LINUX_PAGE_RANGE_HELPER_H
|
||||||
|
|
||||||
|
#include <linux/list_lru.h>
|
||||||
|
|
||||||
|
enum lru_status
|
||||||
|
rust_shrink_free_page_wrap(struct list_head *item, struct list_lru_one *list,
|
||||||
|
void *cb_arg);
|
||||||
|
|
||||||
|
#endif /* _LINUX_PAGE_RANGE_HELPER_H */
|
||||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,251 @@
|
|||||||
|
// SPDX-License-Identifier: GPL-2.0
|
||||||
|
|
||||||
|
// Copyright (C) 2025 Google LLC.
|
||||||
|
|
||||||
|
use kernel::{
|
||||||
|
page::{PAGE_MASK, PAGE_SIZE},
|
||||||
|
prelude::*,
|
||||||
|
seq_file::SeqFile,
|
||||||
|
seq_print,
|
||||||
|
task::Pid,
|
||||||
|
};
|
||||||
|
|
||||||
|
use crate::range_alloc::{DescriptorState, FreedRange, Range};
|
||||||
|
|
||||||
|
/// Keeps track of allocations in a process' mmap.
|
||||||
|
///
|
||||||
|
/// Each process has an mmap where the data for incoming transactions will be placed. This struct
|
||||||
|
/// keeps track of allocations made in the mmap. For each allocation, we store a descriptor that
|
||||||
|
/// has metadata related to the allocation. We also keep track of available free space.
|
||||||
|
pub(super) struct ArrayRangeAllocator<T> {
|
||||||
|
/// This stores all ranges that are allocated. Unlike the tree based allocator, we do *not*
|
||||||
|
/// store the free ranges.
|
||||||
|
///
|
||||||
|
/// Sorted by offset.
|
||||||
|
pub(super) ranges: KVec<Range<T>>,
|
||||||
|
size: usize,
|
||||||
|
free_oneway_space: usize,
|
||||||
|
}
|
||||||
|
|
||||||
|
struct FindEmptyRes {
|
||||||
|
/// Which index in `ranges` should we insert the new range at?
|
||||||
|
///
|
||||||
|
/// Inserting the new range at this index keeps `ranges` sorted.
|
||||||
|
insert_at_idx: usize,
|
||||||
|
/// Which offset should we insert the new range at?
|
||||||
|
insert_at_offset: usize,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl<T> ArrayRangeAllocator<T> {
|
||||||
|
pub(crate) fn new(size: usize, alloc: EmptyArrayAlloc<T>) -> Self {
|
||||||
|
Self {
|
||||||
|
ranges: alloc.ranges,
|
||||||
|
size,
|
||||||
|
free_oneway_space: size / 2,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
pub(crate) fn free_oneway_space(&self) -> usize {
|
||||||
|
self.free_oneway_space
|
||||||
|
}
|
||||||
|
|
||||||
|
pub(crate) fn count_buffers(&self) -> usize {
|
||||||
|
self.ranges.len()
|
||||||
|
}
|
||||||
|
|
||||||
|
pub(crate) fn total_size(&self) -> usize {
|
||||||
|
self.size
|
||||||
|
}
|
||||||
|
|
||||||
|
pub(crate) fn is_full(&self) -> bool {
|
||||||
|
self.ranges.len() == self.ranges.capacity()
|
||||||
|
}
|
||||||
|
|
||||||
|
pub(crate) fn debug_print(&self, m: &SeqFile) -> Result<()> {
|
||||||
|
for range in &self.ranges {
|
||||||
|
seq_print!(
|
||||||
|
m,
|
||||||
|
" buffer {}: {} size {} pid {} oneway {}",
|
||||||
|
0,
|
||||||
|
range.offset,
|
||||||
|
range.size,
|
||||||
|
range.state.pid(),
|
||||||
|
range.state.is_oneway(),
|
||||||
|
);
|
||||||
|
if let DescriptorState::Reserved(_) = range.state {
|
||||||
|
seq_print!(m, " reserved\n");
|
||||||
|
} else {
|
||||||
|
seq_print!(m, " allocated\n");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Find somewhere to put a new range.
|
||||||
|
///
|
||||||
|
/// Unlike the tree implementation, we do not bother to find the smallest gap. The idea is that
|
||||||
|
/// fragmentation isn't a big issue when we don't have many ranges.
|
||||||
|
///
|
||||||
|
/// Returns the index that the new range should have in `self.ranges` after insertion.
|
||||||
|
fn find_empty_range(&self, size: usize) -> Option<FindEmptyRes> {
|
||||||
|
let after_last_range = self.ranges.last().map(Range::endpoint).unwrap_or(0);
|
||||||
|
|
||||||
|
if size <= self.total_size() - after_last_range {
|
||||||
|
// We can put the range at the end, so just do that.
|
||||||
|
Some(FindEmptyRes {
|
||||||
|
insert_at_idx: self.ranges.len(),
|
||||||
|
insert_at_offset: after_last_range,
|
||||||
|
})
|
||||||
|
} else {
|
||||||
|
let mut end_of_prev = 0;
|
||||||
|
for (i, range) in self.ranges.iter().enumerate() {
|
||||||
|
// Does it fit before the i'th range?
|
||||||
|
if size <= range.offset - end_of_prev {
|
||||||
|
return Some(FindEmptyRes {
|
||||||
|
insert_at_idx: i,
|
||||||
|
insert_at_offset: end_of_prev,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
end_of_prev = range.endpoint();
|
||||||
|
}
|
||||||
|
None
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
pub(crate) fn reserve_new(
|
||||||
|
&mut self,
|
||||||
|
debug_id: usize,
|
||||||
|
size: usize,
|
||||||
|
is_oneway: bool,
|
||||||
|
pid: Pid,
|
||||||
|
) -> Result<usize> {
|
||||||
|
// Compute new value of free_oneway_space, which is set only on success.
|
||||||
|
let new_oneway_space = if is_oneway {
|
||||||
|
match self.free_oneway_space.checked_sub(size) {
|
||||||
|
Some(new_oneway_space) => new_oneway_space,
|
||||||
|
None => return Err(ENOSPC),
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
self.free_oneway_space
|
||||||
|
};
|
||||||
|
|
||||||
|
let FindEmptyRes {
|
||||||
|
insert_at_idx,
|
||||||
|
insert_at_offset,
|
||||||
|
} = self.find_empty_range(size).ok_or(ENOSPC)?;
|
||||||
|
self.free_oneway_space = new_oneway_space;
|
||||||
|
|
||||||
|
let new_range = Range {
|
||||||
|
offset: insert_at_offset,
|
||||||
|
size,
|
||||||
|
state: DescriptorState::new(is_oneway, debug_id, pid),
|
||||||
|
};
|
||||||
|
// Insert the value at the given index to keep the array sorted.
|
||||||
|
self.ranges
|
||||||
|
.insert_within_capacity(insert_at_idx, new_range)
|
||||||
|
.ok()
|
||||||
|
.unwrap();
|
||||||
|
|
||||||
|
Ok(insert_at_offset)
|
||||||
|
}
|
||||||
|
|
||||||
|
pub(crate) fn reservation_abort(&mut self, offset: usize) -> Result<FreedRange> {
|
||||||
|
// This could use a binary search, but linear scans are usually faster for small arrays.
|
||||||
|
let i = self
|
||||||
|
.ranges
|
||||||
|
.iter()
|
||||||
|
.position(|range| range.offset == offset)
|
||||||
|
.ok_or(EINVAL)?;
|
||||||
|
let range = &self.ranges[i];
|
||||||
|
|
||||||
|
if let DescriptorState::Allocated(_) = range.state {
|
||||||
|
return Err(EPERM);
|
||||||
|
}
|
||||||
|
|
||||||
|
let size = range.size;
|
||||||
|
let offset = range.offset;
|
||||||
|
|
||||||
|
if range.state.is_oneway() {
|
||||||
|
self.free_oneway_space += size;
|
||||||
|
}
|
||||||
|
|
||||||
|
// This computes the range of pages that are no longer used by *any* allocated range. The
|
||||||
|
// caller will mark them as unused, which means that they can be freed if the system comes
|
||||||
|
// under memory pressure.
|
||||||
|
let mut freed_range = FreedRange::interior_pages(offset, size);
|
||||||
|
#[expect(clippy::collapsible_if)] // reads better like this
|
||||||
|
if offset % PAGE_SIZE != 0 {
|
||||||
|
if i == 0 || self.ranges[i - 1].endpoint() <= (offset & PAGE_MASK) {
|
||||||
|
freed_range.start_page_idx -= 1;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if range.endpoint() % PAGE_SIZE != 0 {
|
||||||
|
let page_after = (range.endpoint() & PAGE_MASK) + PAGE_SIZE;
|
||||||
|
if i + 1 == self.ranges.len() || page_after <= self.ranges[i + 1].offset {
|
||||||
|
freed_range.end_page_idx += 1;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
self.ranges.remove(i)?;
|
||||||
|
Ok(freed_range)
|
||||||
|
}
|
||||||
|
|
||||||
|
pub(crate) fn reservation_commit(&mut self, offset: usize, data: &mut Option<T>) -> Result {
|
||||||
|
// This could use a binary search, but linear scans are usually faster for small arrays.
|
||||||
|
let range = self
|
||||||
|
.ranges
|
||||||
|
.iter_mut()
|
||||||
|
.find(|range| range.offset == offset)
|
||||||
|
.ok_or(ENOENT)?;
|
||||||
|
|
||||||
|
let DescriptorState::Reserved(reservation) = &range.state else {
|
||||||
|
return Err(ENOENT);
|
||||||
|
};
|
||||||
|
|
||||||
|
range.state = DescriptorState::Allocated(reservation.clone().allocate(data.take()));
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
pub(crate) fn reserve_existing(&mut self, offset: usize) -> Result<(usize, usize, Option<T>)> {
|
||||||
|
// This could use a binary search, but linear scans are usually faster for small arrays.
|
||||||
|
let range = self
|
||||||
|
.ranges
|
||||||
|
.iter_mut()
|
||||||
|
.find(|range| range.offset == offset)
|
||||||
|
.ok_or(ENOENT)?;
|
||||||
|
|
||||||
|
let DescriptorState::Allocated(allocation) = &mut range.state else {
|
||||||
|
return Err(ENOENT);
|
||||||
|
};
|
||||||
|
|
||||||
|
let data = allocation.take();
|
||||||
|
let debug_id = allocation.reservation.debug_id;
|
||||||
|
range.state = DescriptorState::Reserved(allocation.reservation.clone());
|
||||||
|
Ok((range.size, debug_id, data))
|
||||||
|
}
|
||||||
|
|
||||||
|
pub(crate) fn take_for_each<F: Fn(usize, usize, usize, Option<T>)>(&mut self, callback: F) {
|
||||||
|
for range in self.ranges.iter_mut() {
|
||||||
|
if let DescriptorState::Allocated(allocation) = &mut range.state {
|
||||||
|
callback(
|
||||||
|
range.offset,
|
||||||
|
range.size,
|
||||||
|
allocation.reservation.debug_id,
|
||||||
|
allocation.data.take(),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
pub(crate) struct EmptyArrayAlloc<T> {
|
||||||
|
ranges: KVec<Range<T>>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl<T> EmptyArrayAlloc<T> {
|
||||||
|
pub(crate) fn try_new(capacity: usize) -> Result<Self> {
|
||||||
|
Ok(Self {
|
||||||
|
ranges: KVec::with_capacity(capacity, GFP_KERNEL)?,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,329 @@
|
|||||||
|
// SPDX-License-Identifier: GPL-2.0
|
||||||
|
|
||||||
|
// Copyright (C) 2025 Google LLC.
|
||||||
|
|
||||||
|
use kernel::{page::PAGE_SIZE, prelude::*, seq_file::SeqFile, task::Pid};
|
||||||
|
|
||||||
|
mod tree;
|
||||||
|
use self::tree::{FromArrayAllocs, ReserveNewTreeAlloc, TreeRangeAllocator};
|
||||||
|
|
||||||
|
mod array;
|
||||||
|
use self::array::{ArrayRangeAllocator, EmptyArrayAlloc};
|
||||||
|
|
||||||
|
enum DescriptorState<T> {
|
||||||
|
Reserved(Reservation),
|
||||||
|
Allocated(Allocation<T>),
|
||||||
|
}
|
||||||
|
|
||||||
|
impl<T> DescriptorState<T> {
|
||||||
|
fn new(is_oneway: bool, debug_id: usize, pid: Pid) -> Self {
|
||||||
|
DescriptorState::Reserved(Reservation {
|
||||||
|
debug_id,
|
||||||
|
is_oneway,
|
||||||
|
pid,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
fn pid(&self) -> Pid {
|
||||||
|
match self {
|
||||||
|
DescriptorState::Reserved(inner) => inner.pid,
|
||||||
|
DescriptorState::Allocated(inner) => inner.reservation.pid,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn is_oneway(&self) -> bool {
|
||||||
|
match self {
|
||||||
|
DescriptorState::Reserved(inner) => inner.is_oneway,
|
||||||
|
DescriptorState::Allocated(inner) => inner.reservation.is_oneway,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Clone)]
|
||||||
|
struct Reservation {
|
||||||
|
debug_id: usize,
|
||||||
|
is_oneway: bool,
|
||||||
|
pid: Pid,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Reservation {
|
||||||
|
fn allocate<T>(self, data: Option<T>) -> Allocation<T> {
|
||||||
|
Allocation {
|
||||||
|
data,
|
||||||
|
reservation: self,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
struct Allocation<T> {
|
||||||
|
reservation: Reservation,
|
||||||
|
data: Option<T>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl<T> Allocation<T> {
|
||||||
|
fn deallocate(self) -> (Reservation, Option<T>) {
|
||||||
|
(self.reservation, self.data)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn debug_id(&self) -> usize {
|
||||||
|
self.reservation.debug_id
|
||||||
|
}
|
||||||
|
|
||||||
|
fn take(&mut self) -> Option<T> {
|
||||||
|
self.data.take()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The array implementation must switch to the tree if it wants to go beyond this number of
|
||||||
|
/// ranges.
|
||||||
|
const TREE_THRESHOLD: usize = 8;
|
||||||
|
|
||||||
|
/// Represents a range of pages that have just become completely free.
|
||||||
|
#[derive(Copy, Clone)]
|
||||||
|
pub(crate) struct FreedRange {
|
||||||
|
pub(crate) start_page_idx: usize,
|
||||||
|
pub(crate) end_page_idx: usize,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl FreedRange {
|
||||||
|
fn interior_pages(offset: usize, size: usize) -> FreedRange {
|
||||||
|
FreedRange {
|
||||||
|
// Divide round up
|
||||||
|
start_page_idx: offset.div_ceil(PAGE_SIZE),
|
||||||
|
// Divide round down
|
||||||
|
end_page_idx: (offset + size) / PAGE_SIZE,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
struct Range<T> {
|
||||||
|
offset: usize,
|
||||||
|
size: usize,
|
||||||
|
state: DescriptorState<T>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl<T> Range<T> {
|
||||||
|
fn endpoint(&self) -> usize {
|
||||||
|
self.offset + self.size
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
pub(crate) struct RangeAllocator<T> {
|
||||||
|
inner: Impl<T>,
|
||||||
|
}
|
||||||
|
|
||||||
|
enum Impl<T> {
|
||||||
|
Empty(usize),
|
||||||
|
Array(ArrayRangeAllocator<T>),
|
||||||
|
Tree(TreeRangeAllocator<T>),
|
||||||
|
}
|
||||||
|
|
||||||
|
impl<T> RangeAllocator<T> {
|
||||||
|
pub(crate) fn new(size: usize) -> Self {
|
||||||
|
Self {
|
||||||
|
inner: Impl::Empty(size),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
pub(crate) fn free_oneway_space(&self) -> usize {
|
||||||
|
match &self.inner {
|
||||||
|
Impl::Empty(size) => size / 2,
|
||||||
|
Impl::Array(array) => array.free_oneway_space(),
|
||||||
|
Impl::Tree(tree) => tree.free_oneway_space(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
pub(crate) fn count_buffers(&self) -> usize {
|
||||||
|
match &self.inner {
|
||||||
|
Impl::Empty(_size) => 0,
|
||||||
|
Impl::Array(array) => array.count_buffers(),
|
||||||
|
Impl::Tree(tree) => tree.count_buffers(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
pub(crate) fn debug_print(&self, m: &SeqFile) -> Result<()> {
|
||||||
|
match &self.inner {
|
||||||
|
Impl::Empty(_size) => Ok(()),
|
||||||
|
Impl::Array(array) => array.debug_print(m),
|
||||||
|
Impl::Tree(tree) => tree.debug_print(m),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Try to reserve a new buffer, using the provided allocation if necessary.
|
||||||
|
pub(crate) fn reserve_new(&mut self, mut args: ReserveNewArgs<T>) -> Result<ReserveNew<T>> {
|
||||||
|
match &mut self.inner {
|
||||||
|
Impl::Empty(size) => {
|
||||||
|
let empty_array = match args.empty_array_alloc.take() {
|
||||||
|
Some(empty_array) => ArrayRangeAllocator::new(*size, empty_array),
|
||||||
|
None => {
|
||||||
|
return Ok(ReserveNew::NeedAlloc(ReserveNewNeedAlloc {
|
||||||
|
args,
|
||||||
|
need_empty_array_alloc: true,
|
||||||
|
need_new_tree_alloc: false,
|
||||||
|
need_tree_alloc: false,
|
||||||
|
}))
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
self.inner = Impl::Array(empty_array);
|
||||||
|
self.reserve_new(args)
|
||||||
|
}
|
||||||
|
Impl::Array(array) if array.is_full() => {
|
||||||
|
let allocs = match args.new_tree_alloc {
|
||||||
|
Some(ref mut allocs) => allocs,
|
||||||
|
None => {
|
||||||
|
return Ok(ReserveNew::NeedAlloc(ReserveNewNeedAlloc {
|
||||||
|
args,
|
||||||
|
need_empty_array_alloc: false,
|
||||||
|
need_new_tree_alloc: true,
|
||||||
|
need_tree_alloc: true,
|
||||||
|
}))
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
let new_tree =
|
||||||
|
TreeRangeAllocator::from_array(array.total_size(), &mut array.ranges, allocs);
|
||||||
|
|
||||||
|
self.inner = Impl::Tree(new_tree);
|
||||||
|
self.reserve_new(args)
|
||||||
|
}
|
||||||
|
Impl::Array(array) => {
|
||||||
|
let offset =
|
||||||
|
array.reserve_new(args.debug_id, args.size, args.is_oneway, args.pid)?;
|
||||||
|
Ok(ReserveNew::Success(ReserveNewSuccess {
|
||||||
|
offset,
|
||||||
|
oneway_spam_detected: false,
|
||||||
|
_empty_array_alloc: args.empty_array_alloc,
|
||||||
|
_new_tree_alloc: args.new_tree_alloc,
|
||||||
|
_tree_alloc: args.tree_alloc,
|
||||||
|
}))
|
||||||
|
}
|
||||||
|
Impl::Tree(tree) => {
|
||||||
|
let alloc = match args.tree_alloc {
|
||||||
|
Some(alloc) => alloc,
|
||||||
|
None => {
|
||||||
|
return Ok(ReserveNew::NeedAlloc(ReserveNewNeedAlloc {
|
||||||
|
args,
|
||||||
|
need_empty_array_alloc: false,
|
||||||
|
need_new_tree_alloc: false,
|
||||||
|
need_tree_alloc: true,
|
||||||
|
}));
|
||||||
|
}
|
||||||
|
};
|
||||||
|
let (offset, oneway_spam_detected) =
|
||||||
|
tree.reserve_new(args.debug_id, args.size, args.is_oneway, args.pid, alloc)?;
|
||||||
|
Ok(ReserveNew::Success(ReserveNewSuccess {
|
||||||
|
offset,
|
||||||
|
oneway_spam_detected,
|
||||||
|
_empty_array_alloc: args.empty_array_alloc,
|
||||||
|
_new_tree_alloc: args.new_tree_alloc,
|
||||||
|
_tree_alloc: None,
|
||||||
|
}))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Deletes the allocations at `offset`.
|
||||||
|
pub(crate) fn reservation_abort(&mut self, offset: usize) -> Result<FreedRange> {
|
||||||
|
match &mut self.inner {
|
||||||
|
Impl::Empty(_size) => Err(EINVAL),
|
||||||
|
Impl::Array(array) => array.reservation_abort(offset),
|
||||||
|
Impl::Tree(tree) => {
|
||||||
|
let freed_range = tree.reservation_abort(offset)?;
|
||||||
|
if tree.is_empty() {
|
||||||
|
self.inner = Impl::Empty(tree.total_size());
|
||||||
|
}
|
||||||
|
Ok(freed_range)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Called when an allocation is no longer in use by the kernel.
|
||||||
|
///
|
||||||
|
/// The value in `data` will be stored, if any. A mutable reference is used to avoid dropping
|
||||||
|
/// the `T` when an error is returned.
|
||||||
|
pub(crate) fn reservation_commit(&mut self, offset: usize, data: &mut Option<T>) -> Result {
|
||||||
|
match &mut self.inner {
|
||||||
|
Impl::Empty(_size) => Err(EINVAL),
|
||||||
|
Impl::Array(array) => array.reservation_commit(offset, data),
|
||||||
|
Impl::Tree(tree) => tree.reservation_commit(offset, data),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Called when the kernel starts using an allocation.
|
||||||
|
///
|
||||||
|
/// Returns the size of the existing entry and the data associated with it.
|
||||||
|
pub(crate) fn reserve_existing(&mut self, offset: usize) -> Result<(usize, usize, Option<T>)> {
|
||||||
|
match &mut self.inner {
|
||||||
|
Impl::Empty(_size) => Err(EINVAL),
|
||||||
|
Impl::Array(array) => array.reserve_existing(offset),
|
||||||
|
Impl::Tree(tree) => tree.reserve_existing(offset),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Call the provided callback at every allocated region.
|
||||||
|
///
|
||||||
|
/// This destroys the range allocator. Used only during shutdown.
|
||||||
|
pub(crate) fn take_for_each<F: Fn(usize, usize, usize, Option<T>)>(&mut self, callback: F) {
|
||||||
|
match &mut self.inner {
|
||||||
|
Impl::Empty(_size) => {}
|
||||||
|
Impl::Array(array) => array.take_for_each(callback),
|
||||||
|
Impl::Tree(tree) => tree.take_for_each(callback),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The arguments for `reserve_new`.
|
||||||
|
#[derive(Default)]
|
||||||
|
pub(crate) struct ReserveNewArgs<T> {
|
||||||
|
pub(crate) size: usize,
|
||||||
|
pub(crate) is_oneway: bool,
|
||||||
|
pub(crate) debug_id: usize,
|
||||||
|
pub(crate) pid: Pid,
|
||||||
|
pub(crate) empty_array_alloc: Option<EmptyArrayAlloc<T>>,
|
||||||
|
pub(crate) new_tree_alloc: Option<FromArrayAllocs<T>>,
|
||||||
|
pub(crate) tree_alloc: Option<ReserveNewTreeAlloc<T>>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The return type of `ReserveNew`.
|
||||||
|
pub(crate) enum ReserveNew<T> {
|
||||||
|
Success(ReserveNewSuccess<T>),
|
||||||
|
NeedAlloc(ReserveNewNeedAlloc<T>),
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Returned by `reserve_new` when the reservation was successul.
|
||||||
|
pub(crate) struct ReserveNewSuccess<T> {
|
||||||
|
pub(crate) offset: usize,
|
||||||
|
pub(crate) oneway_spam_detected: bool,
|
||||||
|
|
||||||
|
// If the user supplied an allocation that we did not end up using, then we return it here.
|
||||||
|
// The caller will kfree it outside of the lock.
|
||||||
|
_empty_array_alloc: Option<EmptyArrayAlloc<T>>,
|
||||||
|
_new_tree_alloc: Option<FromArrayAllocs<T>>,
|
||||||
|
_tree_alloc: Option<ReserveNewTreeAlloc<T>>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Returned by `reserve_new` to request the caller to make an allocation before calling the method
|
||||||
|
/// again.
|
||||||
|
pub(crate) struct ReserveNewNeedAlloc<T> {
|
||||||
|
args: ReserveNewArgs<T>,
|
||||||
|
need_empty_array_alloc: bool,
|
||||||
|
need_new_tree_alloc: bool,
|
||||||
|
need_tree_alloc: bool,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl<T> ReserveNewNeedAlloc<T> {
|
||||||
|
/// Make the necessary allocations for another call to `reserve_new`.
|
||||||
|
pub(crate) fn make_alloc(mut self) -> Result<ReserveNewArgs<T>> {
|
||||||
|
if self.need_empty_array_alloc && self.args.empty_array_alloc.is_none() {
|
||||||
|
self.args.empty_array_alloc = Some(EmptyArrayAlloc::try_new(TREE_THRESHOLD)?);
|
||||||
|
}
|
||||||
|
if self.need_new_tree_alloc && self.args.new_tree_alloc.is_none() {
|
||||||
|
self.args.new_tree_alloc = Some(FromArrayAllocs::try_new(TREE_THRESHOLD)?);
|
||||||
|
}
|
||||||
|
if self.need_tree_alloc && self.args.tree_alloc.is_none() {
|
||||||
|
self.args.tree_alloc = Some(ReserveNewTreeAlloc::try_new()?);
|
||||||
|
}
|
||||||
|
Ok(self.args)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,488 @@
|
|||||||
|
// SPDX-License-Identifier: GPL-2.0
|
||||||
|
|
||||||
|
// Copyright (C) 2025 Google LLC.
|
||||||
|
|
||||||
|
use kernel::{
|
||||||
|
page::PAGE_SIZE,
|
||||||
|
prelude::*,
|
||||||
|
rbtree::{RBTree, RBTreeNode, RBTreeNodeReservation},
|
||||||
|
seq_file::SeqFile,
|
||||||
|
seq_print,
|
||||||
|
task::Pid,
|
||||||
|
};
|
||||||
|
|
||||||
|
use crate::range_alloc::{DescriptorState, FreedRange, Range};
|
||||||
|
|
||||||
|
/// Keeps track of allocations in a process' mmap.
|
||||||
|
///
|
||||||
|
/// Each process has an mmap where the data for incoming transactions will be placed. This struct
|
||||||
|
/// keeps track of allocations made in the mmap. For each allocation, we store a descriptor that
|
||||||
|
/// has metadata related to the allocation. We also keep track of available free space.
|
||||||
|
pub(super) struct TreeRangeAllocator<T> {
|
||||||
|
/// This collection contains descriptors for *both* ranges containing an allocation, *and* free
|
||||||
|
/// ranges between allocations. The free ranges get merged, so there are never two free ranges
|
||||||
|
/// next to each other.
|
||||||
|
tree: RBTree<usize, Descriptor<T>>,
|
||||||
|
/// Contains an entry for every free range in `self.tree`. This tree sorts the ranges by size,
|
||||||
|
/// letting us look up the smallest range whose size is at least some lower bound.
|
||||||
|
free_tree: RBTree<FreeKey, ()>,
|
||||||
|
size: usize,
|
||||||
|
free_oneway_space: usize,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl<T> TreeRangeAllocator<T> {
|
||||||
|
pub(crate) fn from_array(
|
||||||
|
size: usize,
|
||||||
|
ranges: &mut KVec<Range<T>>,
|
||||||
|
alloc: &mut FromArrayAllocs<T>,
|
||||||
|
) -> Self {
|
||||||
|
let mut tree = TreeRangeAllocator {
|
||||||
|
tree: RBTree::new(),
|
||||||
|
free_tree: RBTree::new(),
|
||||||
|
size,
|
||||||
|
free_oneway_space: size / 2,
|
||||||
|
};
|
||||||
|
|
||||||
|
let mut free_offset = 0;
|
||||||
|
for range in ranges.drain_all() {
|
||||||
|
let free_size = range.offset - free_offset;
|
||||||
|
if free_size > 0 {
|
||||||
|
let free_node = alloc.free_tree.pop().unwrap();
|
||||||
|
tree.free_tree
|
||||||
|
.insert(free_node.into_node((free_size, free_offset), ()));
|
||||||
|
let tree_node = alloc.tree.pop().unwrap();
|
||||||
|
tree.tree.insert(
|
||||||
|
tree_node.into_node(free_offset, Descriptor::new(free_offset, free_size)),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
free_offset = range.endpoint();
|
||||||
|
|
||||||
|
if range.state.is_oneway() {
|
||||||
|
tree.free_oneway_space = tree.free_oneway_space.saturating_sub(range.size);
|
||||||
|
}
|
||||||
|
|
||||||
|
let free_res = alloc.free_tree.pop().unwrap();
|
||||||
|
let tree_node = alloc.tree.pop().unwrap();
|
||||||
|
let mut desc = Descriptor::new(range.offset, range.size);
|
||||||
|
desc.state = Some((range.state, free_res));
|
||||||
|
tree.tree.insert(tree_node.into_node(range.offset, desc));
|
||||||
|
}
|
||||||
|
|
||||||
|
// After the last range, we may need a free range.
|
||||||
|
if free_offset < size {
|
||||||
|
let free_size = size - free_offset;
|
||||||
|
let free_node = alloc.free_tree.pop().unwrap();
|
||||||
|
tree.free_tree
|
||||||
|
.insert(free_node.into_node((free_size, free_offset), ()));
|
||||||
|
let tree_node = alloc.tree.pop().unwrap();
|
||||||
|
tree.tree
|
||||||
|
.insert(tree_node.into_node(free_offset, Descriptor::new(free_offset, free_size)));
|
||||||
|
}
|
||||||
|
|
||||||
|
tree
|
||||||
|
}
|
||||||
|
|
||||||
|
pub(crate) fn is_empty(&self) -> bool {
|
||||||
|
let mut tree_iter = self.tree.values();
|
||||||
|
// There's always at least one range, because index zero is either the start of a free or
|
||||||
|
// allocated range.
|
||||||
|
let first_value = tree_iter.next().unwrap();
|
||||||
|
if tree_iter.next().is_some() {
|
||||||
|
// There are never two free ranges next to each other, so if there is more than one
|
||||||
|
// descriptor, then at least one of them must hold an allocated range.
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
// There is only one descriptor. Return true if it is for a free range.
|
||||||
|
first_value.state.is_none()
|
||||||
|
}
|
||||||
|
|
||||||
|
pub(crate) fn total_size(&self) -> usize {
|
||||||
|
self.size
|
||||||
|
}
|
||||||
|
|
||||||
|
pub(crate) fn free_oneway_space(&self) -> usize {
|
||||||
|
self.free_oneway_space
|
||||||
|
}
|
||||||
|
|
||||||
|
pub(crate) fn count_buffers(&self) -> usize {
|
||||||
|
self.tree
|
||||||
|
.values()
|
||||||
|
.filter(|desc| desc.state.is_some())
|
||||||
|
.count()
|
||||||
|
}
|
||||||
|
|
||||||
|
pub(crate) fn debug_print(&self, m: &SeqFile) -> Result<()> {
|
||||||
|
for desc in self.tree.values() {
|
||||||
|
let state = match &desc.state {
|
||||||
|
Some(state) => &state.0,
|
||||||
|
None => continue,
|
||||||
|
};
|
||||||
|
seq_print!(
|
||||||
|
m,
|
||||||
|
" buffer: {} size {} pid {}",
|
||||||
|
desc.offset,
|
||||||
|
desc.size,
|
||||||
|
state.pid(),
|
||||||
|
);
|
||||||
|
if state.is_oneway() {
|
||||||
|
seq_print!(m, " oneway");
|
||||||
|
}
|
||||||
|
match state {
|
||||||
|
DescriptorState::Reserved(_res) => {
|
||||||
|
seq_print!(m, " reserved\n");
|
||||||
|
}
|
||||||
|
DescriptorState::Allocated(_alloc) => {
|
||||||
|
seq_print!(m, " allocated\n");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
fn find_best_match(&mut self, size: usize) -> Option<&mut Descriptor<T>> {
|
||||||
|
let free_cursor = self.free_tree.cursor_lower_bound(&(size, 0))?;
|
||||||
|
let ((_, offset), ()) = free_cursor.current();
|
||||||
|
self.tree.get_mut(offset)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Try to reserve a new buffer, using the provided allocation if necessary.
|
||||||
|
pub(crate) fn reserve_new(
|
||||||
|
&mut self,
|
||||||
|
debug_id: usize,
|
||||||
|
size: usize,
|
||||||
|
is_oneway: bool,
|
||||||
|
pid: Pid,
|
||||||
|
alloc: ReserveNewTreeAlloc<T>,
|
||||||
|
) -> Result<(usize, bool)> {
|
||||||
|
// Compute new value of free_oneway_space, which is set only on success.
|
||||||
|
let new_oneway_space = if is_oneway {
|
||||||
|
match self.free_oneway_space.checked_sub(size) {
|
||||||
|
Some(new_oneway_space) => new_oneway_space,
|
||||||
|
None => return Err(ENOSPC),
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
self.free_oneway_space
|
||||||
|
};
|
||||||
|
|
||||||
|
// Start detecting spammers once we have less than 20%
|
||||||
|
// of async space left (which is less than 10% of total
|
||||||
|
// buffer size).
|
||||||
|
//
|
||||||
|
// (This will short-circut, so `low_oneway_space` is
|
||||||
|
// only called when necessary.)
|
||||||
|
let oneway_spam_detected =
|
||||||
|
is_oneway && new_oneway_space < self.size / 10 && self.low_oneway_space(pid);
|
||||||
|
|
||||||
|
let (found_size, found_off, tree_node, free_tree_node) = match self.find_best_match(size) {
|
||||||
|
None => {
|
||||||
|
pr_warn!("ENOSPC from range_alloc.reserve_new - size: {}", size);
|
||||||
|
return Err(ENOSPC);
|
||||||
|
}
|
||||||
|
Some(desc) => {
|
||||||
|
let found_size = desc.size;
|
||||||
|
let found_offset = desc.offset;
|
||||||
|
|
||||||
|
// In case we need to break up the descriptor
|
||||||
|
let new_desc = Descriptor::new(found_offset + size, found_size - size);
|
||||||
|
let (tree_node, free_tree_node, desc_node_res) = alloc.initialize(new_desc);
|
||||||
|
|
||||||
|
desc.state = Some((
|
||||||
|
DescriptorState::new(is_oneway, debug_id, pid),
|
||||||
|
desc_node_res,
|
||||||
|
));
|
||||||
|
desc.size = size;
|
||||||
|
|
||||||
|
(found_size, found_offset, tree_node, free_tree_node)
|
||||||
|
}
|
||||||
|
};
|
||||||
|
self.free_oneway_space = new_oneway_space;
|
||||||
|
self.free_tree.remove(&(found_size, found_off));
|
||||||
|
|
||||||
|
if found_size != size {
|
||||||
|
self.tree.insert(tree_node);
|
||||||
|
self.free_tree.insert(free_tree_node);
|
||||||
|
}
|
||||||
|
|
||||||
|
Ok((found_off, oneway_spam_detected))
|
||||||
|
}
|
||||||
|
|
||||||
|
pub(crate) fn reservation_abort(&mut self, offset: usize) -> Result<FreedRange> {
|
||||||
|
let mut cursor = self.tree.cursor_lower_bound(&offset).ok_or_else(|| {
|
||||||
|
pr_warn!(
|
||||||
|
"EINVAL from range_alloc.reservation_abort - offset: {}",
|
||||||
|
offset
|
||||||
|
);
|
||||||
|
EINVAL
|
||||||
|
})?;
|
||||||
|
|
||||||
|
let (_, desc) = cursor.current_mut();
|
||||||
|
|
||||||
|
if desc.offset != offset {
|
||||||
|
pr_warn!(
|
||||||
|
"EINVAL from range_alloc.reservation_abort - offset: {}",
|
||||||
|
offset
|
||||||
|
);
|
||||||
|
return Err(EINVAL);
|
||||||
|
}
|
||||||
|
|
||||||
|
let (reservation, free_node_res) = desc.try_change_state(|state| match state {
|
||||||
|
Some((DescriptorState::Reserved(reservation), free_node_res)) => {
|
||||||
|
(None, Ok((reservation, free_node_res)))
|
||||||
|
}
|
||||||
|
None => {
|
||||||
|
pr_warn!(
|
||||||
|
"EINVAL from range_alloc.reservation_abort - offset: {}",
|
||||||
|
offset
|
||||||
|
);
|
||||||
|
(None, Err(EINVAL))
|
||||||
|
}
|
||||||
|
allocated => {
|
||||||
|
pr_warn!(
|
||||||
|
"EPERM from range_alloc.reservation_abort - offset: {}",
|
||||||
|
offset
|
||||||
|
);
|
||||||
|
(allocated, Err(EPERM))
|
||||||
|
}
|
||||||
|
})?;
|
||||||
|
|
||||||
|
let mut size = desc.size;
|
||||||
|
let mut offset = desc.offset;
|
||||||
|
let free_oneway_space_add = if reservation.is_oneway { size } else { 0 };
|
||||||
|
|
||||||
|
self.free_oneway_space += free_oneway_space_add;
|
||||||
|
|
||||||
|
let mut freed_range = FreedRange::interior_pages(offset, size);
|
||||||
|
// Compute how large the next free region needs to be to include one more page in
|
||||||
|
// the newly freed range.
|
||||||
|
let add_next_page_needed = match (offset + size) % PAGE_SIZE {
|
||||||
|
0 => usize::MAX,
|
||||||
|
unalign => PAGE_SIZE - unalign,
|
||||||
|
};
|
||||||
|
// Compute how large the previous free region needs to be to include one more page
|
||||||
|
// in the newly freed range.
|
||||||
|
let add_prev_page_needed = match offset % PAGE_SIZE {
|
||||||
|
0 => usize::MAX,
|
||||||
|
unalign => unalign,
|
||||||
|
};
|
||||||
|
|
||||||
|
// Merge next into current if next is free
|
||||||
|
let remove_next = match cursor.peek_next() {
|
||||||
|
Some((_, next)) if next.state.is_none() => {
|
||||||
|
if next.size >= add_next_page_needed {
|
||||||
|
freed_range.end_page_idx += 1;
|
||||||
|
}
|
||||||
|
self.free_tree.remove(&(next.size, next.offset));
|
||||||
|
size += next.size;
|
||||||
|
true
|
||||||
|
}
|
||||||
|
_ => false,
|
||||||
|
};
|
||||||
|
|
||||||
|
if remove_next {
|
||||||
|
let (_, desc) = cursor.current_mut();
|
||||||
|
desc.size = size;
|
||||||
|
cursor.remove_next();
|
||||||
|
}
|
||||||
|
|
||||||
|
// Merge current into prev if prev is free
|
||||||
|
match cursor.peek_prev_mut() {
|
||||||
|
Some((_, prev)) if prev.state.is_none() => {
|
||||||
|
if prev.size >= add_prev_page_needed {
|
||||||
|
freed_range.start_page_idx -= 1;
|
||||||
|
}
|
||||||
|
// merge previous with current, remove current
|
||||||
|
self.free_tree.remove(&(prev.size, prev.offset));
|
||||||
|
offset = prev.offset;
|
||||||
|
size += prev.size;
|
||||||
|
prev.size = size;
|
||||||
|
cursor.remove_current();
|
||||||
|
}
|
||||||
|
_ => {}
|
||||||
|
};
|
||||||
|
|
||||||
|
self.free_tree
|
||||||
|
.insert(free_node_res.into_node((size, offset), ()));
|
||||||
|
|
||||||
|
Ok(freed_range)
|
||||||
|
}
|
||||||
|
|
||||||
|
pub(crate) fn reservation_commit(&mut self, offset: usize, data: &mut Option<T>) -> Result {
|
||||||
|
let desc = self.tree.get_mut(&offset).ok_or(ENOENT)?;
|
||||||
|
|
||||||
|
desc.try_change_state(|state| match state {
|
||||||
|
Some((DescriptorState::Reserved(reservation), free_node_res)) => (
|
||||||
|
Some((
|
||||||
|
DescriptorState::Allocated(reservation.allocate(data.take())),
|
||||||
|
free_node_res,
|
||||||
|
)),
|
||||||
|
Ok(()),
|
||||||
|
),
|
||||||
|
other => (other, Err(ENOENT)),
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Takes an entry at the given offset from [`DescriptorState::Allocated`] to
|
||||||
|
/// [`DescriptorState::Reserved`].
|
||||||
|
///
|
||||||
|
/// Returns the size of the existing entry and the data associated with it.
|
||||||
|
pub(crate) fn reserve_existing(&mut self, offset: usize) -> Result<(usize, usize, Option<T>)> {
|
||||||
|
let desc = self.tree.get_mut(&offset).ok_or_else(|| {
|
||||||
|
pr_warn!(
|
||||||
|
"ENOENT from range_alloc.reserve_existing - offset: {}",
|
||||||
|
offset
|
||||||
|
);
|
||||||
|
ENOENT
|
||||||
|
})?;
|
||||||
|
|
||||||
|
let (debug_id, data) = desc.try_change_state(|state| match state {
|
||||||
|
Some((DescriptorState::Allocated(allocation), free_node_res)) => {
|
||||||
|
let (reservation, data) = allocation.deallocate();
|
||||||
|
let debug_id = reservation.debug_id;
|
||||||
|
(
|
||||||
|
Some((DescriptorState::Reserved(reservation), free_node_res)),
|
||||||
|
Ok((debug_id, data)),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
other => {
|
||||||
|
pr_warn!(
|
||||||
|
"ENOENT from range_alloc.reserve_existing - offset: {}",
|
||||||
|
offset
|
||||||
|
);
|
||||||
|
(other, Err(ENOENT))
|
||||||
|
}
|
||||||
|
})?;
|
||||||
|
|
||||||
|
Ok((desc.size, debug_id, data))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Call the provided callback at every allocated region.
|
||||||
|
///
|
||||||
|
/// This destroys the range allocator. Used only during shutdown.
|
||||||
|
pub(crate) fn take_for_each<F: Fn(usize, usize, usize, Option<T>)>(&mut self, callback: F) {
|
||||||
|
for (_, desc) in self.tree.iter_mut() {
|
||||||
|
if let Some((DescriptorState::Allocated(allocation), _)) = &mut desc.state {
|
||||||
|
callback(
|
||||||
|
desc.offset,
|
||||||
|
desc.size,
|
||||||
|
allocation.debug_id(),
|
||||||
|
allocation.take(),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Find the amount and size of buffers allocated by the current caller.
|
||||||
|
///
|
||||||
|
/// The idea is that once we cross the threshold, whoever is responsible
|
||||||
|
/// for the low async space is likely to try to send another async transaction,
|
||||||
|
/// and at some point we'll catch them in the act. This is more efficient
|
||||||
|
/// than keeping a map per pid.
|
||||||
|
fn low_oneway_space(&self, calling_pid: Pid) -> bool {
|
||||||
|
let mut total_alloc_size = 0;
|
||||||
|
let mut num_buffers = 0;
|
||||||
|
for (_, desc) in self.tree.iter() {
|
||||||
|
if let Some((state, _)) = &desc.state {
|
||||||
|
if state.is_oneway() && state.pid() == calling_pid {
|
||||||
|
total_alloc_size += desc.size;
|
||||||
|
num_buffers += 1;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Warn if this pid has more than 50 transactions, or more than 50% of
|
||||||
|
// async space (which is 25% of total buffer size). Oneway spam is only
|
||||||
|
// detected when the threshold is exceeded.
|
||||||
|
num_buffers > 50 || total_alloc_size > self.size / 4
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
type TreeDescriptorState<T> = (DescriptorState<T>, FreeNodeRes);
|
||||||
|
struct Descriptor<T> {
|
||||||
|
size: usize,
|
||||||
|
offset: usize,
|
||||||
|
state: Option<TreeDescriptorState<T>>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl<T> Descriptor<T> {
|
||||||
|
fn new(offset: usize, size: usize) -> Self {
|
||||||
|
Self {
|
||||||
|
size,
|
||||||
|
offset,
|
||||||
|
state: None,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn try_change_state<F, Data>(&mut self, f: F) -> Result<Data>
|
||||||
|
where
|
||||||
|
F: FnOnce(Option<TreeDescriptorState<T>>) -> (Option<TreeDescriptorState<T>>, Result<Data>),
|
||||||
|
{
|
||||||
|
let (new_state, result) = f(self.state.take());
|
||||||
|
self.state = new_state;
|
||||||
|
result
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// (Descriptor.size, Descriptor.offset)
|
||||||
|
type FreeKey = (usize, usize);
|
||||||
|
type FreeNodeRes = RBTreeNodeReservation<FreeKey, ()>;
|
||||||
|
|
||||||
|
/// An allocation for use by `reserve_new`.
|
||||||
|
pub(crate) struct ReserveNewTreeAlloc<T> {
|
||||||
|
tree_node_res: RBTreeNodeReservation<usize, Descriptor<T>>,
|
||||||
|
free_tree_node_res: FreeNodeRes,
|
||||||
|
desc_node_res: FreeNodeRes,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl<T> ReserveNewTreeAlloc<T> {
|
||||||
|
pub(crate) fn try_new() -> Result<Self> {
|
||||||
|
let tree_node_res = RBTreeNodeReservation::new(GFP_KERNEL)?;
|
||||||
|
let free_tree_node_res = RBTreeNodeReservation::new(GFP_KERNEL)?;
|
||||||
|
let desc_node_res = RBTreeNodeReservation::new(GFP_KERNEL)?;
|
||||||
|
Ok(Self {
|
||||||
|
tree_node_res,
|
||||||
|
free_tree_node_res,
|
||||||
|
desc_node_res,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
fn initialize(
|
||||||
|
self,
|
||||||
|
desc: Descriptor<T>,
|
||||||
|
) -> (
|
||||||
|
RBTreeNode<usize, Descriptor<T>>,
|
||||||
|
RBTreeNode<FreeKey, ()>,
|
||||||
|
FreeNodeRes,
|
||||||
|
) {
|
||||||
|
let size = desc.size;
|
||||||
|
let offset = desc.offset;
|
||||||
|
(
|
||||||
|
self.tree_node_res.into_node(offset, desc),
|
||||||
|
self.free_tree_node_res.into_node((size, offset), ()),
|
||||||
|
self.desc_node_res,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// An allocation for creating a tree from an `ArrayRangeAllocator`.
|
||||||
|
pub(crate) struct FromArrayAllocs<T> {
|
||||||
|
tree: KVec<RBTreeNodeReservation<usize, Descriptor<T>>>,
|
||||||
|
free_tree: KVec<RBTreeNodeReservation<FreeKey, ()>>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl<T> FromArrayAllocs<T> {
|
||||||
|
pub(crate) fn try_new(len: usize) -> Result<Self> {
|
||||||
|
let num_descriptors = 2 * len + 1;
|
||||||
|
|
||||||
|
let mut tree = KVec::with_capacity(num_descriptors, GFP_KERNEL)?;
|
||||||
|
for _ in 0..num_descriptors {
|
||||||
|
tree.push(RBTreeNodeReservation::new(GFP_KERNEL)?, GFP_KERNEL)?;
|
||||||
|
}
|
||||||
|
|
||||||
|
let mut free_tree = KVec::with_capacity(num_descriptors, GFP_KERNEL)?;
|
||||||
|
for _ in 0..num_descriptors {
|
||||||
|
free_tree.push(RBTreeNodeReservation::new(GFP_KERNEL)?, GFP_KERNEL)?;
|
||||||
|
}
|
||||||
|
|
||||||
|
Ok(Self { tree, free_tree })
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,23 @@
|
|||||||
|
/* SPDX-License-Identifier: GPL-2.0 */
|
||||||
|
/*
|
||||||
|
* Copyright (C) 2025 Google, Inc.
|
||||||
|
*/
|
||||||
|
|
||||||
|
#ifndef _LINUX_RUST_BINDER_H
|
||||||
|
#define _LINUX_RUST_BINDER_H
|
||||||
|
|
||||||
|
#include <uapi/linux/android/binder.h>
|
||||||
|
#include <uapi/linux/android/binderfs.h>
|
||||||
|
|
||||||
|
/*
|
||||||
|
* These symbols are exposed by `rust_binderfs.c` and exist here so that Rust
|
||||||
|
* Binder can call them.
|
||||||
|
*/
|
||||||
|
int init_rust_binderfs(void);
|
||||||
|
|
||||||
|
struct dentry;
|
||||||
|
struct inode;
|
||||||
|
struct dentry *rust_binderfs_create_proc_file(struct inode *nodp, int pid);
|
||||||
|
void rust_binderfs_remove_file(struct dentry *dentry);
|
||||||
|
|
||||||
|
#endif
|
||||||
@@ -0,0 +1,59 @@
|
|||||||
|
// SPDX-License-Identifier: GPL-2.0-only
|
||||||
|
/* rust_binder_events.c
|
||||||
|
*
|
||||||
|
* Rust Binder tracepoints.
|
||||||
|
*
|
||||||
|
* Copyright 2025 Google LLC
|
||||||
|
*/
|
||||||
|
|
||||||
|
#include "rust_binder.h"
|
||||||
|
|
||||||
|
const char * const binder_command_strings[] = {
|
||||||
|
"BC_TRANSACTION",
|
||||||
|
"BC_REPLY",
|
||||||
|
"BC_ACQUIRE_RESULT",
|
||||||
|
"BC_FREE_BUFFER",
|
||||||
|
"BC_INCREFS",
|
||||||
|
"BC_ACQUIRE",
|
||||||
|
"BC_RELEASE",
|
||||||
|
"BC_DECREFS",
|
||||||
|
"BC_INCREFS_DONE",
|
||||||
|
"BC_ACQUIRE_DONE",
|
||||||
|
"BC_ATTEMPT_ACQUIRE",
|
||||||
|
"BC_REGISTER_LOOPER",
|
||||||
|
"BC_ENTER_LOOPER",
|
||||||
|
"BC_EXIT_LOOPER",
|
||||||
|
"BC_REQUEST_DEATH_NOTIFICATION",
|
||||||
|
"BC_CLEAR_DEATH_NOTIFICATION",
|
||||||
|
"BC_DEAD_BINDER_DONE",
|
||||||
|
"BC_TRANSACTION_SG",
|
||||||
|
"BC_REPLY_SG",
|
||||||
|
};
|
||||||
|
|
||||||
|
const char * const binder_return_strings[] = {
|
||||||
|
"BR_ERROR",
|
||||||
|
"BR_OK",
|
||||||
|
"BR_TRANSACTION",
|
||||||
|
"BR_REPLY",
|
||||||
|
"BR_ACQUIRE_RESULT",
|
||||||
|
"BR_DEAD_REPLY",
|
||||||
|
"BR_TRANSACTION_COMPLETE",
|
||||||
|
"BR_INCREFS",
|
||||||
|
"BR_ACQUIRE",
|
||||||
|
"BR_RELEASE",
|
||||||
|
"BR_DECREFS",
|
||||||
|
"BR_ATTEMPT_ACQUIRE",
|
||||||
|
"BR_NOOP",
|
||||||
|
"BR_SPAWN_LOOPER",
|
||||||
|
"BR_FINISHED",
|
||||||
|
"BR_DEAD_BINDER",
|
||||||
|
"BR_CLEAR_DEATH_NOTIFICATION_DONE",
|
||||||
|
"BR_FAILED_REPLY",
|
||||||
|
"BR_FROZEN_REPLY",
|
||||||
|
"BR_ONEWAY_SPAM_SUSPECT",
|
||||||
|
"BR_TRANSACTION_PENDING_FROZEN"
|
||||||
|
};
|
||||||
|
|
||||||
|
#define CREATE_TRACE_POINTS
|
||||||
|
#define CREATE_RUST_TRACE_POINTS
|
||||||
|
#include "rust_binder_events.h"
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user