Merge tag 'drm-rust-next-2026-03-30' of https://gitlab.freedesktop.org/drm/rust/kernel into drm-next

DRM Rust changes for v7.1-rc1

- DMA:
  - Rework the DMA coherent API: introduce Coherent<T> as a generalized
    container for arbitrary types, replacing the slice-only
    CoherentAllocation<T>. Add CoherentBox for memory initialization
    before exposing a buffer to hardware (converting to Coherent when
    ready), and CoherentHandle for allocations without kernel mapping.

  - Add Coherent::init() / init_with_attrs() for one-shot initialization
    via pin-init, and from-slice constructors for both Coherent and
    CoherentBox

  - Add uaccess write_dma() for copying from DMA buffers to userspace
    and BinaryWriter support for Coherent<T>

- DRM:
  - Add GPU buddy allocator abstraction

  - Add DRM shmem GEM helper abstraction

  - Allow drm::Device to dispatch work and delayed work items to driver
    private data

  - Add impl_aref_for_gem_obj!() macro to reduce GEM refcount
    boilerplate, and introduce DriverObject::Args for constructor
    context

  - Add dma_resv_lock helper and raw_dma_resv() accessor on GEM objects

  - Clean up imports across the DRM module

- I/O:
  - Merged via a signed tag from the driver-core tree: register!() macro
    and I/O infrastructure improvements (IoCapable refactor, RelaxedMmio
    wrapper, IoLoc trait, generic accessors, write_reg /
    LocatedRegister)

- Nova (Core):
  - Fix and harden the GSP command queue: correct write pointer
    advancing, empty slot handling, and ring buffer indexing; add mutex
    locking and make Cmdq a pinned type; distinguish wait vs no-wait
    commands

  - Add support for large RPCs via continuation records, splitting
    oversized commands across multiple queue slots

  - Simplify GSP sequencer and message handling code: remove unused
    trait and Display impls, derive Debug and Zeroable where applicable,
    warn on unconsumed message data

  - Refactor Falcon firmware handling: create DMA objects lazily, add
    PIO upload support, and use the Generic Bootloader to boot FWSEC on
    Turing

  - Convert all register definitions (PMC, PBUS, PFB, GC6, FUSE, PDISP,
    Falcon) to the kernel register!() macro; add bounded_enum macro to
    define enums usable as register fields

  - Migrate all DMA usage to the new Coherent, CoherentBox, and
    CoherentHandle APIs

  - Harden firmware parsing with checked arithmetic throughout FWSEC,
    Booter, RISC-V parsing paths

  - Add debugfs support for reading GSP-RM log buffers; replace
    module_pci_driver!() with explicit module init to support
    module-level debugfs setup

  - Fix auxiliary device registration for multi-GPU systems

  - Various cleanups: import style, firmware parsing refactoring,
    framebuffer size logging

- Rust:
  - Add interop::list module providing a C linked list interface

  - Extend num::Bounded with shift operations, into_bool(), and const
    get() to support register bitfield manipulation

  - Enable the generic_arg_infer Rust feature and add EMSGSIZE error
    code

- Tyr:
  - Adopt vertical import style per kernel Rust guidelines

  - Clarify driver/device type names and use DRM device type alias
    consistently across the driver

  - Fix GPU model/version decoding in GpuInfo

- Workqueue:
  - Add ARef<T> support for work and delayed work

Signed-off-by: Dave Airlie <airlied@redhat.com>

From: "Danilo Krummrich" <dakr@kernel.org>
Link: https://patch.msgid.link/DHGH4BLT03BU.ZJH5U52WE8BY@kernel.org
This commit is contained in:
Dave Airlie
2026-04-01 07:32:05 +10:00
74 changed files with 7132 additions and 3065 deletions
-76
View File
@@ -51,82 +51,6 @@ There also have been considerations of ToPrimitive [2].
| Link: https://lore.kernel.org/all/cover.1750689857.git.y.j3ms.n@gmail.com/ [1]
| Link: https://rust-for-linux.zulipchat.com/#narrow/channel/288089-General/topic/Implement.20.60FromPrimitive.60.20trait.20.2B.20derive.20macro.20for.20nova-core/with/541971854 [2]
Generic register abstraction [REGA]
-----------------------------------
Work out how register constants and structures can be automatically generated
through generalized macros.
Example:
.. code-block:: rust
register!(BOOT0, 0x0, u32, pci::Bar<SIZE>, Fields [
MINOR_REVISION(3:0, RO),
MAJOR_REVISION(7:4, RO),
REVISION(7:0, RO), // Virtual register combining major and minor rev.
])
This could expand to something like:
.. code-block:: rust
const BOOT0_OFFSET: usize = 0x00000000;
const BOOT0_MINOR_REVISION_SHIFT: u8 = 0;
const BOOT0_MINOR_REVISION_MASK: u32 = 0x0000000f;
const BOOT0_MAJOR_REVISION_SHIFT: u8 = 4;
const BOOT0_MAJOR_REVISION_MASK: u32 = 0x000000f0;
const BOOT0_REVISION_SHIFT: u8 = BOOT0_MINOR_REVISION_SHIFT;
const BOOT0_REVISION_MASK: u32 = BOOT0_MINOR_REVISION_MASK | BOOT0_MAJOR_REVISION_MASK;
struct Boot0(u32);
impl Boot0 {
#[inline]
fn read(bar: &RevocableGuard<'_, pci::Bar<SIZE>>) -> Self {
Self(bar.readl(BOOT0_OFFSET))
}
#[inline]
fn minor_revision(&self) -> u32 {
(self.0 & BOOT0_MINOR_REVISION_MASK) >> BOOT0_MINOR_REVISION_SHIFT
}
#[inline]
fn major_revision(&self) -> u32 {
(self.0 & BOOT0_MAJOR_REVISION_MASK) >> BOOT0_MAJOR_REVISION_SHIFT
}
#[inline]
fn revision(&self) -> u32 {
(self.0 & BOOT0_REVISION_MASK) >> BOOT0_REVISION_SHIFT
}
}
Usage:
.. code-block:: rust
let bar = bar.try_access().ok_or(ENXIO)?;
let boot0 = Boot0::read(&bar);
pr_info!("Revision: {}\n", boot0.revision());
A work-in-progress implementation currently resides in
`drivers/gpu/nova-core/regs/macros.rs` and is used in nova-core. It would be
nice to improve it (possibly using proc macros) and move it to the `kernel`
crate so it can be used by other components as well.
Features desired before this happens:
* Make I/O optional I/O (for field values that are not registers),
* Support other sizes than `u32`,
* Allow visibility control for registers and individual fields,
* Use Rust slice syntax to express fields ranges.
| Complexity: Advanced
| Contact: Alexandre Courbot
Numerical operations [NUMM]
---------------------------
+17 -1
View File
@@ -7534,6 +7534,7 @@ F: include/linux/*fence.h
F: include/linux/dma-buf.h
F: include/linux/dma-buf/
F: include/linux/dma-resv.h
F: rust/helpers/dma-resv.c
K: \bdma_(?:buf|fence|resv)\b
DMA GENERIC OFFLOAD ENGINE SUBSYSTEM
@@ -8513,7 +8514,10 @@ T: git https://gitlab.freedesktop.org/drm/rust/kernel.git
F: drivers/gpu/drm/nova/
F: drivers/gpu/drm/tyr/
F: drivers/gpu/nova-core/
F: rust/helpers/gpu.c
F: rust/kernel/drm/
F: rust/kernel/gpu.rs
F: rust/kernel/gpu/
DRM DRIVERS FOR ALLWINNER A10
M: Chen-Yu Tsai <wens@kernel.org>
@@ -8931,7 +8935,7 @@ F: include/drm/ttm/
GPU BUDDY ALLOCATOR
M: Matthew Auld <matthew.auld@intel.com>
M: Arun Pravin <arunpravin.paneerselvam@amd.com>
R: Christian Koenig <christian.koenig@amd.com>
R: Joel Fernandes <joelagnelf@nvidia.com>
L: dri-devel@lists.freedesktop.org
S: Maintained
T: git https://gitlab.freedesktop.org/drm/misc/kernel.git
@@ -8940,6 +8944,9 @@ F: drivers/gpu/drm/drm_buddy.c
F: drivers/gpu/tests/gpu_buddy_test.c
F: include/drm/drm_buddy.h
F: include/linux/gpu_buddy.h
F: rust/helpers/gpu.c
F: rust/kernel/gpu.rs
F: rust/kernel/gpu/
DRM AUTOMATED TESTING
M: Helen Koike <helen.fornazier@gmail.com>
@@ -23208,6 +23215,15 @@ T: git https://github.com/Rust-for-Linux/linux.git alloc-next
F: rust/kernel/alloc.rs
F: rust/kernel/alloc/
RUST [INTEROP]
M: Joel Fernandes <joelagnelf@nvidia.com>
M: Alexandre Courbot <acourbot@nvidia.com>
L: rust-for-linux@vger.kernel.org
S: Maintained
T: git https://github.com/Rust-for-Linux/linux.git interop-next
F: rust/kernel/interop.rs
F: rust/kernel/interop/
RUST [NUM]
M: Alexandre Courbot <acourbot@nvidia.com>
R: Yury Norov <yury.norov@gmail.com>
+7
View File
@@ -268,6 +268,13 @@ config DRM_GEM_SHMEM_HELPER
help
Choose this if you need the GEM shmem helper functions
config RUST_DRM_GEM_SHMEM_HELPER
bool
depends on DRM && MMU
select DRM_GEM_SHMEM_HELPER
help
Choose this if you need the GEM shmem helper functions In Rust
config DRM_SUBALLOC_HELPER
tristate
depends on DRM
+3 -2
View File
@@ -19,8 +19,9 @@ pub(crate) struct NovaObject {}
impl gem::DriverObject for NovaObject {
type Driver = NovaDriver;
type Args = ();
fn new(_dev: &NovaDevice, _size: usize) -> impl PinInit<Self, Error> {
fn new(_dev: &NovaDevice, _size: usize, _args: Self::Args) -> impl PinInit<Self, Error> {
try_pin_init!(NovaObject {})
}
}
@@ -33,7 +34,7 @@ impl NovaObject {
}
let aligned_size = page::page_align(size).ok_or(EINVAL)?;
gem::Object::new(dev, aligned_size)
gem::Object::new(dev, aligned_size, ())
}
/// Look up a GEM object handle for a `File` and return an `ObjectRef` for it.
+55 -43
View File
@@ -1,44 +1,56 @@
// SPDX-License-Identifier: GPL-2.0 or MIT
use kernel::clk::Clk;
use kernel::clk::OptionalClk;
use kernel::device::Bound;
use kernel::device::Core;
use kernel::device::Device;
use kernel::devres::Devres;
use kernel::drm;
use kernel::drm::ioctl;
use kernel::io::poll;
use kernel::new_mutex;
use kernel::of;
use kernel::platform;
use kernel::prelude::*;
use kernel::regulator;
use kernel::regulator::Regulator;
use kernel::sizes::SZ_2M;
use kernel::sync::aref::ARef;
use kernel::sync::Arc;
use kernel::sync::Mutex;
use kernel::time;
use kernel::{
clk::{
Clk,
OptionalClk, //
},
device::{
Bound,
Core,
Device, //
},
devres::Devres,
drm,
drm::ioctl,
io::poll,
new_mutex,
of,
platform,
prelude::*,
regulator,
regulator::Regulator,
sizes::SZ_2M,
sync::{
aref::ARef,
Arc,
Mutex, //
},
time, //
};
use crate::file::File;
use crate::gem::TyrObject;
use crate::gpu;
use crate::gpu::GpuInfo;
use crate::regs;
use crate::{
file::TyrDrmFileData,
gem::TyrObject,
gpu,
gpu::GpuInfo,
regs, //
};
pub(crate) type IoMem = kernel::io::mem::IoMem<SZ_2M>;
pub(crate) struct TyrDrmDriver;
/// Convenience type alias for the DRM device type for this driver.
pub(crate) type TyrDevice = drm::Device<TyrDriver>;
pub(crate) type TyrDrmDevice = drm::Device<TyrDrmDriver>;
#[pin_data(PinnedDrop)]
pub(crate) struct TyrDriver {
_device: ARef<TyrDevice>,
pub(crate) struct TyrPlatformDriverData {
_device: ARef<TyrDrmDevice>,
}
#[pin_data(PinnedDrop)]
pub(crate) struct TyrData {
pub(crate) struct TyrDrmDeviceData {
pub(crate) pdev: ARef<platform::Device>,
#[pin]
@@ -61,9 +73,9 @@ pub(crate) struct TyrData {
// that it will be removed in a future patch.
//
// SAFETY: This will be removed in a future patch.
unsafe impl Send for TyrData {}
unsafe impl Send for TyrDrmDeviceData {}
// SAFETY: This will be removed in a future patch.
unsafe impl Sync for TyrData {}
unsafe impl Sync for TyrDrmDeviceData {}
fn issue_soft_reset(dev: &Device<Bound>, iomem: &Devres<IoMem>) -> Result {
regs::GPU_CMD.write(dev, iomem, regs::GPU_CMD_SOFT_RESET)?;
@@ -82,14 +94,14 @@ fn issue_soft_reset(dev: &Device<Bound>, iomem: &Devres<IoMem>) -> Result {
kernel::of_device_table!(
OF_TABLE,
MODULE_OF_TABLE,
<TyrDriver as platform::Driver>::IdInfo,
<TyrPlatformDriverData as platform::Driver>::IdInfo,
[
(of::DeviceId::new(c"rockchip,rk3588-mali"), ()),
(of::DeviceId::new(c"arm,mali-valhall-csf"), ())
]
);
impl platform::Driver for TyrDriver {
impl platform::Driver for TyrPlatformDriverData {
type IdInfo = ();
const OF_ID_TABLE: Option<of::IdTable<Self::IdInfo>> = Some(&OF_TABLE);
@@ -119,7 +131,7 @@ impl platform::Driver for TyrDriver {
let platform: ARef<platform::Device> = pdev.into();
let data = try_pin_init!(TyrData {
let data = try_pin_init!(TyrDrmDeviceData {
pdev: platform.clone(),
clks <- new_mutex!(Clocks {
core: core_clk,
@@ -133,10 +145,10 @@ impl platform::Driver for TyrDriver {
gpu_info,
});
let tdev: ARef<TyrDevice> = drm::Device::new(pdev.as_ref(), data)?;
drm::driver::Registration::new_foreign_owned(&tdev, pdev.as_ref(), 0)?;
let ddev: ARef<TyrDrmDevice> = drm::Device::new(pdev.as_ref(), data)?;
drm::driver::Registration::new_foreign_owned(&ddev, pdev.as_ref(), 0)?;
let driver = TyrDriver { _device: tdev };
let driver = TyrPlatformDriverData { _device: ddev };
// We need this to be dev_info!() because dev_dbg!() does not work at
// all in Rust for now, and we need to see whether probe succeeded.
@@ -146,12 +158,12 @@ impl platform::Driver for TyrDriver {
}
#[pinned_drop]
impl PinnedDrop for TyrDriver {
impl PinnedDrop for TyrPlatformDriverData {
fn drop(self: Pin<&mut Self>) {}
}
#[pinned_drop]
impl PinnedDrop for TyrData {
impl PinnedDrop for TyrDrmDeviceData {
fn drop(self: Pin<&mut Self>) {
// TODO: the type-state pattern for Clks will fix this.
let clks = self.clks.lock();
@@ -172,15 +184,15 @@ const INFO: drm::DriverInfo = drm::DriverInfo {
};
#[vtable]
impl drm::Driver for TyrDriver {
type Data = TyrData;
type File = File;
impl drm::Driver for TyrDrmDriver {
type Data = TyrDrmDeviceData;
type File = TyrDrmFileData;
type Object = drm::gem::Object<TyrObject>;
const INFO: drm::DriverInfo = INFO;
kernel::declare_drm_ioctls! {
(PANTHOR_DEV_QUERY, drm_panthor_dev_query, ioctl::RENDER_ALLOW, File::dev_query),
(PANTHOR_DEV_QUERY, drm_panthor_dev_query, ioctl::RENDER_ALLOW, TyrDrmFileData::dev_query),
}
}
+19 -15
View File
@@ -1,37 +1,41 @@
// SPDX-License-Identifier: GPL-2.0 or MIT
use kernel::drm;
use kernel::prelude::*;
use kernel::uaccess::UserSlice;
use kernel::uapi;
use kernel::{
drm,
prelude::*,
uaccess::UserSlice,
uapi, //
};
use crate::driver::TyrDevice;
use crate::TyrDriver;
use crate::driver::{
TyrDrmDevice,
TyrDrmDriver, //
};
#[pin_data]
pub(crate) struct File {}
pub(crate) struct TyrDrmFileData {}
/// Convenience type alias for our DRM `File` type
pub(crate) type DrmFile = drm::file::File<File>;
pub(crate) type TyrDrmFile = drm::file::File<TyrDrmFileData>;
impl drm::file::DriverFile for File {
type Driver = TyrDriver;
impl drm::file::DriverFile for TyrDrmFileData {
type Driver = TyrDrmDriver;
fn open(_dev: &drm::Device<Self::Driver>) -> Result<Pin<KBox<Self>>> {
KBox::try_pin_init(try_pin_init!(Self {}), GFP_KERNEL)
}
}
impl File {
impl TyrDrmFileData {
pub(crate) fn dev_query(
tdev: &TyrDevice,
ddev: &TyrDrmDevice,
devquery: &mut uapi::drm_panthor_dev_query,
_file: &DrmFile,
_file: &TyrDrmFile,
) -> Result<u32> {
if devquery.pointer == 0 {
match devquery.type_ {
uapi::drm_panthor_dev_query_type_DRM_PANTHOR_DEV_QUERY_GPU_INFO => {
devquery.size = core::mem::size_of_val(&tdev.gpu_info) as u32;
devquery.size = core::mem::size_of_val(&ddev.gpu_info) as u32;
Ok(0)
}
_ => Err(EINVAL),
@@ -45,7 +49,7 @@ impl File {
)
.writer();
writer.write(&tdev.gpu_info)?;
writer.write(&ddev.gpu_info)?;
Ok(0)
}
+12 -6
View File
@@ -1,18 +1,24 @@
// SPDX-License-Identifier: GPL-2.0 or MIT
use crate::driver::TyrDevice;
use crate::driver::TyrDriver;
use kernel::drm::gem;
use kernel::prelude::*;
use kernel::{
drm::gem,
prelude::*, //
};
use crate::driver::{
TyrDrmDevice,
TyrDrmDriver, //
};
/// GEM Object inner driver data
#[pin_data]
pub(crate) struct TyrObject {}
impl gem::DriverObject for TyrObject {
type Driver = TyrDriver;
type Driver = TyrDrmDriver;
type Args = ();
fn new(_dev: &TyrDevice, _size: usize) -> impl PinInit<Self, Error> {
fn new(_dev: &TyrDrmDevice, _size: usize, _args: ()) -> impl PinInit<Self, Error> {
try_pin_init!(TyrObject {})
}
}
+31 -25
View File
@@ -1,20 +1,28 @@
// SPDX-License-Identifier: GPL-2.0 or MIT
use core::ops::Deref;
use core::ops::DerefMut;
use kernel::bits::genmask_u32;
use kernel::device::Bound;
use kernel::device::Device;
use kernel::devres::Devres;
use kernel::io::poll;
use kernel::platform;
use kernel::prelude::*;
use kernel::time::Delta;
use kernel::transmute::AsBytes;
use kernel::uapi;
use core::ops::{
Deref,
DerefMut, //
};
use kernel::{
bits::genmask_u32,
device::{
Bound,
Device, //
},
devres::Devres,
io::poll,
platform,
prelude::*,
time::Delta,
transmute::AsBytes,
uapi, //
};
use crate::driver::IoMem;
use crate::regs;
use crate::{
driver::IoMem,
regs, //
};
/// Struct containing information that can be queried by userspace. This is read from
/// the GPU's registers.
@@ -84,13 +92,11 @@ impl GpuInfo {
}
pub(crate) fn log(&self, pdev: &platform::Device) {
let major = (self.gpu_id >> 16) & 0xff;
let minor = (self.gpu_id >> 8) & 0xff;
let status = self.gpu_id & 0xff;
let gpu_id = GpuId::from(self.gpu_id);
let model_name = if let Some(model) = GPU_MODELS
.iter()
.find(|&f| f.major == major && f.minor == minor)
.find(|&f| f.arch_major == gpu_id.arch_major && f.prod_major == gpu_id.prod_major)
{
model.name
} else {
@@ -102,9 +108,9 @@ impl GpuInfo {
"mali-{} id 0x{:x} major 0x{:x} minor 0x{:x} status 0x{:x}",
model_name,
self.gpu_id >> 16,
major,
minor,
status
gpu_id.ver_major,
gpu_id.ver_minor,
gpu_id.ver_status
);
dev_info!(
@@ -166,14 +172,14 @@ unsafe impl AsBytes for GpuInfo {}
struct GpuModels {
name: &'static str,
major: u32,
minor: u32,
arch_major: u32,
prod_major: u32,
}
const GPU_MODELS: [GpuModels; 1] = [GpuModels {
name: "g610",
major: 10,
minor: 7,
arch_major: 10,
prod_major: 7,
}];
#[allow(dead_code)]
+10 -6
View File
@@ -7,12 +7,16 @@
// does.
#![allow(dead_code)]
use kernel::bits::bit_u32;
use kernel::device::Bound;
use kernel::device::Device;
use kernel::devres::Devres;
use kernel::io::Io;
use kernel::prelude::*;
use kernel::{
bits::bit_u32,
device::{
Bound,
Device, //
},
devres::Devres,
io::Io,
prelude::*, //
};
use crate::driver::IoMem;
+2 -2
View File
@@ -5,7 +5,7 @@
//! The name "Tyr" is inspired by Norse mythology, reflecting Arm's tradition of
//! naming their GPUs after Nordic mythological figures and places.
use crate::driver::TyrDriver;
use crate::driver::TyrPlatformDriverData;
mod driver;
mod file;
@@ -14,7 +14,7 @@ mod gpu;
mod regs;
kernel::module_platform_driver! {
type: TyrDriver,
type: TyrPlatformDriverData,
name: "tyr",
authors: ["The Tyr driver authors"],
description: "Arm Mali Tyr DRM driver",
+1 -1
View File
@@ -3,8 +3,8 @@ config NOVA_CORE
depends on 64BIT
depends on PCI
depends on RUST
select RUST_FW_LOADER_ABSTRACTIONS
select AUXILIARY_BUS
select RUST_FW_LOADER_ABSTRACTIONS
default n
help
Choose this if you want to build the Nova Core driver for Nvidia
-54
View File
@@ -1,54 +0,0 @@
// SPDX-License-Identifier: GPL-2.0
//! Simple DMA object wrapper.
use core::ops::{
Deref,
DerefMut, //
};
use kernel::{
device,
dma::CoherentAllocation,
page::PAGE_SIZE,
prelude::*, //
};
pub(crate) struct DmaObject {
dma: CoherentAllocation<u8>,
}
impl DmaObject {
pub(crate) fn new(dev: &device::Device<device::Bound>, len: usize) -> Result<Self> {
let len = core::alloc::Layout::from_size_align(len, PAGE_SIZE)
.map_err(|_| EINVAL)?
.pad_to_align()
.size();
let dma = CoherentAllocation::alloc_coherent(dev, len, GFP_KERNEL | __GFP_ZERO)?;
Ok(Self { dma })
}
pub(crate) fn from_data(dev: &device::Device<device::Bound>, data: &[u8]) -> Result<Self> {
Self::new(dev, data.len()).and_then(|mut dma_obj| {
// SAFETY: We have just allocated the DMA memory, we are the only users and
// we haven't made the device aware of the handle yet.
unsafe { dma_obj.write(data, 0)? }
Ok(dma_obj)
})
}
}
impl Deref for DmaObject {
type Target = CoherentAllocation<u8>;
fn deref(&self) -> &Self::Target {
&self.dma
}
}
impl DerefMut for DmaObject {
fn deref_mut(&mut self) -> &mut Self::Target {
&mut self.dma
}
}
+14 -3
View File
@@ -14,11 +14,20 @@ use kernel::{
},
prelude::*,
sizes::SZ_16M,
sync::Arc, //
sync::{
atomic::{
Atomic,
Relaxed, //
},
Arc,
},
};
use crate::gpu::Gpu;
/// Counter for generating unique auxiliary device IDs.
static AUXILIARY_ID_COUNTER: Atomic<u32> = Atomic::new(0);
#[pin_data]
pub(crate) struct NovaCore {
#[pin]
@@ -70,7 +79,7 @@ impl pci::Driver for NovaCore {
fn probe(pdev: &pci::Device<Core>, _info: &Self::IdInfo) -> impl PinInit<Self, Error> {
pin_init::pin_init_scope(move || {
dev_dbg!(pdev.as_ref(), "Probe Nova Core GPU driver.\n");
dev_dbg!(pdev, "Probe Nova Core GPU driver.\n");
pdev.enable_device_mem()?;
pdev.set_master();
@@ -90,7 +99,9 @@ impl pci::Driver for NovaCore {
_reg <- auxiliary::Registration::new(
pdev.as_ref(),
c"nova-drm",
0, // TODO[XARR]: Once it lands, use XArray; for now we don't use the ID.
// TODO[XARR]: Use XArray or perhaps IDA for proper ID allocation/recycling. For
// now, use a simple atomic counter that never recycles IDs.
AUXILIARY_ID_COUNTER.fetch_add(1, Relaxed),
crate::MODULE_NAME
),
}))
File diff suppressed because it is too large Load Diff
+15 -12
View File
@@ -1,7 +1,14 @@
// SPDX-License-Identifier: GPL-2.0
use kernel::{
io::poll::read_poll_timeout,
io::{
poll::read_poll_timeout,
register::{
RegisterBase,
WithBase, //
},
Io,
},
prelude::*,
time::Delta, //
};
@@ -14,10 +21,7 @@ use crate::{
PFalcon2Base,
PFalconBase, //
},
regs::{
self,
macros::RegisterBase, //
},
regs,
};
/// Type specifying the `Gsp` falcon engine. Cannot be instantiated.
@@ -31,23 +35,22 @@ impl RegisterBase<PFalcon2Base> for Gsp {
const BASE: usize = 0x00111000;
}
impl FalconEngine for Gsp {
const ID: Self = Gsp(());
}
impl FalconEngine for Gsp {}
impl Falcon<Gsp> {
/// Clears the SWGEN0 bit in the Falcon's IRQ status clear register to
/// allow GSP to signal CPU for processing new messages in message queue.
pub(crate) fn clear_swgen0_intr(&self, bar: &Bar0) {
regs::NV_PFALCON_FALCON_IRQSCLR::default()
.set_swgen0(true)
.write(bar, &Gsp::ID);
bar.write(
WithBase::of::<Gsp>(),
regs::NV_PFALCON_FALCON_IRQSCLR::zeroed().with_swgen0(true),
);
}
/// Checks if GSP reload/resume has completed during the boot process.
pub(crate) fn check_reload_completed(&self, bar: &Bar0, timeout: Delta) -> Result<bool> {
read_poll_timeout(
|| Ok(regs::NV_PGC6_BSI_SECURE_SCRATCH_14::read(bar)),
|| Ok(bar.read(regs::NV_PGC6_BSI_SECURE_SCRATCH_14)),
|val| val.boot_stage_3_handoff(),
Delta::ZERO,
timeout,
+5 -1
View File
@@ -58,7 +58,11 @@ pub(crate) trait FalconHal<E: FalconEngine>: Send + Sync {
/// Reset the falcon engine.
fn reset_eng(&self, bar: &Bar0) -> Result;
/// returns the method needed to load data into Falcon memory
/// Returns the method used to load data into the falcon's memory.
///
/// The only chipsets supporting PIO are those < GA102, and PIO is the preferred method for
/// these. For anything above, the PIO registers appear to be masked to the CPU, so DMA is the
/// only usable method.
fn load_method(&self) -> LoadMethod;
}
+43 -27
View File
@@ -4,7 +4,14 @@ use core::marker::PhantomData;
use kernel::{
device,
io::poll::read_poll_timeout,
io::{
poll::read_poll_timeout,
register::{
Array,
WithBase, //
},
Io, //
},
prelude::*,
time::Delta, //
};
@@ -25,15 +32,16 @@ use crate::{
use super::FalconHal;
fn select_core_ga102<E: FalconEngine>(bar: &Bar0) -> Result {
let bcr_ctrl = regs::NV_PRISCV_RISCV_BCR_CTRL::read(bar, &E::ID);
let bcr_ctrl = bar.read(regs::NV_PRISCV_RISCV_BCR_CTRL::of::<E>());
if bcr_ctrl.core_select() != PeregrineCoreSelect::Falcon {
regs::NV_PRISCV_RISCV_BCR_CTRL::default()
.set_core_select(PeregrineCoreSelect::Falcon)
.write(bar, &E::ID);
bar.write(
WithBase::of::<E>(),
regs::NV_PRISCV_RISCV_BCR_CTRL::zeroed().with_core_select(PeregrineCoreSelect::Falcon),
);
// TIMEOUT: falcon core should take less than 10ms to report being enabled.
read_poll_timeout(
|| Ok(regs::NV_PRISCV_RISCV_BCR_CTRL::read(bar, &E::ID)),
|| Ok(bar.read(regs::NV_PRISCV_RISCV_BCR_CTRL::of::<E>())),
|r| r.valid(),
Delta::ZERO,
Delta::from_millis(10),
@@ -60,12 +68,15 @@ fn signature_reg_fuse_version_ga102(
// `ucode_idx` is guaranteed to be in the range [0..15], making the `read` calls provable valid
// at build-time.
let reg_fuse_version = if engine_id_mask & 0x0001 != 0 {
regs::NV_FUSE_OPT_FPF_SEC2_UCODE1_VERSION::read(bar, ucode_idx).data()
let reg_fuse_version: u16 = if engine_id_mask & 0x0001 != 0 {
bar.read(regs::NV_FUSE_OPT_FPF_SEC2_UCODE1_VERSION::at(ucode_idx))
.data()
} else if engine_id_mask & 0x0004 != 0 {
regs::NV_FUSE_OPT_FPF_NVDEC_UCODE1_VERSION::read(bar, ucode_idx).data()
bar.read(regs::NV_FUSE_OPT_FPF_NVDEC_UCODE1_VERSION::at(ucode_idx))
.data()
} else if engine_id_mask & 0x0400 != 0 {
regs::NV_FUSE_OPT_FPF_GSP_UCODE1_VERSION::read(bar, ucode_idx).data()
bar.read(regs::NV_FUSE_OPT_FPF_GSP_UCODE1_VERSION::at(ucode_idx))
.data()
} else {
dev_err!(dev, "unexpected engine_id_mask {:#x}\n", engine_id_mask);
return Err(EINVAL);
@@ -76,18 +87,23 @@ fn signature_reg_fuse_version_ga102(
}
fn program_brom_ga102<E: FalconEngine>(bar: &Bar0, params: &FalconBromParams) -> Result {
regs::NV_PFALCON2_FALCON_BROM_PARAADDR::default()
.set_value(params.pkc_data_offset)
.write(bar, &E::ID, 0);
regs::NV_PFALCON2_FALCON_BROM_ENGIDMASK::default()
.set_value(u32::from(params.engine_id_mask))
.write(bar, &E::ID);
regs::NV_PFALCON2_FALCON_BROM_CURR_UCODE_ID::default()
.set_ucode_id(params.ucode_id)
.write(bar, &E::ID);
regs::NV_PFALCON2_FALCON_MOD_SEL::default()
.set_algo(FalconModSelAlgo::Rsa3k)
.write(bar, &E::ID);
bar.write(
WithBase::of::<E>().at(0),
regs::NV_PFALCON2_FALCON_BROM_PARAADDR::zeroed().with_value(params.pkc_data_offset),
);
bar.write(
WithBase::of::<E>(),
regs::NV_PFALCON2_FALCON_BROM_ENGIDMASK::zeroed()
.with_value(u32::from(params.engine_id_mask)),
);
bar.write(
WithBase::of::<E>(),
regs::NV_PFALCON2_FALCON_BROM_CURR_UCODE_ID::zeroed().with_ucode_id(params.ucode_id),
);
bar.write(
WithBase::of::<E>(),
regs::NV_PFALCON2_FALCON_MOD_SEL::zeroed().with_algo(FalconModSelAlgo::Rsa3k),
);
Ok(())
}
@@ -120,14 +136,14 @@ impl<E: FalconEngine> FalconHal<E> for Ga102<E> {
}
fn is_riscv_active(&self, bar: &Bar0) -> bool {
let cpuctl = regs::NV_PRISCV_RISCV_CPUCTL::read(bar, &E::ID);
cpuctl.active_stat()
bar.read(regs::NV_PRISCV_RISCV_CPUCTL::of::<E>())
.active_stat()
}
fn reset_wait_mem_scrubbing(&self, bar: &Bar0) -> Result {
// TIMEOUT: memory scrubbing should complete in less than 20ms.
read_poll_timeout(
|| Ok(regs::NV_PFALCON_FALCON_HWCFG2::read(bar, &E::ID)),
|| Ok(bar.read(regs::NV_PFALCON_FALCON_HWCFG2::of::<E>())),
|r| r.mem_scrubbing_done(),
Delta::ZERO,
Delta::from_millis(20),
@@ -136,12 +152,12 @@ impl<E: FalconEngine> FalconHal<E> for Ga102<E> {
}
fn reset_eng(&self, bar: &Bar0) -> Result {
let _ = regs::NV_PFALCON_FALCON_HWCFG2::read(bar, &E::ID);
let _ = bar.read(regs::NV_PFALCON_FALCON_HWCFG2::of::<E>());
// According to OpenRM's `kflcnPreResetWait_GA102` documentation, HW sometimes does not set
// RESET_READY so a non-failing timeout is used.
let _ = read_poll_timeout(
|| Ok(regs::NV_PFALCON_FALCON_HWCFG2::read(bar, &E::ID)),
|| Ok(bar.read(regs::NV_PFALCON_FALCON_HWCFG2::of::<E>())),
|r| r.reset_ready(),
Delta::ZERO,
Delta::from_micros(150),
+8 -4
View File
@@ -3,7 +3,11 @@
use core::marker::PhantomData;
use kernel::{
io::poll::read_poll_timeout,
io::{
poll::read_poll_timeout,
register::WithBase,
Io, //
},
prelude::*,
time::Delta, //
};
@@ -49,14 +53,14 @@ impl<E: FalconEngine> FalconHal<E> for Tu102<E> {
}
fn is_riscv_active(&self, bar: &Bar0) -> bool {
let cpuctl = regs::NV_PRISCV_RISCV_CORE_SWITCH_RISCV_STATUS::read(bar, &E::ID);
cpuctl.active_stat()
bar.read(regs::NV_PRISCV_RISCV_CORE_SWITCH_RISCV_STATUS::of::<E>())
.active_stat()
}
fn reset_wait_mem_scrubbing(&self, bar: &Bar0) -> Result {
// TIMEOUT: memory scrubbing should complete in less than 10ms.
read_poll_timeout(
|| Ok(regs::NV_PFALCON_FALCON_DMACTL::read(bar, &E::ID)),
|| Ok(bar.read(regs::NV_PFALCON_FALCON_DMACTL::of::<E>())),
|r| r.mem_scrubbing_done(),
Delta::ZERO,
Delta::from_millis(10),
+7 -10
View File
@@ -1,12 +1,11 @@
// SPDX-License-Identifier: GPL-2.0
use crate::{
falcon::{
FalconEngine,
PFalcon2Base,
PFalconBase, //
},
regs::macros::RegisterBase,
use kernel::io::register::RegisterBase;
use crate::falcon::{
FalconEngine,
PFalcon2Base,
PFalconBase, //
};
/// Type specifying the `Sec2` falcon engine. Cannot be instantiated.
@@ -20,6 +19,4 @@ impl RegisterBase<PFalcon2Base> for Sec2 {
const BASE: usize = 0x00841000;
}
impl FalconEngine for Sec2 {
const ID: Self = Sec2(());
}
impl FalconEngine for Sec2 {}
+80 -21
View File
@@ -1,9 +1,15 @@
// SPDX-License-Identifier: GPL-2.0
use core::ops::Range;
use core::ops::{
Deref,
Range, //
};
use kernel::{
device,
dma::CoherentHandle,
fmt,
io::Io,
prelude::*,
ptr::{
Alignable,
@@ -14,7 +20,6 @@ use kernel::{
};
use crate::{
dma::DmaObject,
driver::Bar0,
firmware::gsp::GspFirmware,
gpu::Chipset,
@@ -48,7 +53,7 @@ pub(crate) struct SysmemFlush {
chipset: Chipset,
device: ARef<device::Device>,
/// Keep the page alive as long as we need it.
page: DmaObject,
page: CoherentHandle,
}
impl SysmemFlush {
@@ -58,7 +63,7 @@ impl SysmemFlush {
bar: &Bar0,
chipset: Chipset,
) -> Result<Self> {
let page = DmaObject::new(dev, kernel::page::PAGE_SIZE)?;
let page = CoherentHandle::alloc(dev, kernel::page::PAGE_SIZE, GFP_KERNEL)?;
hal::fb_hal(chipset).write_sysmem_flush_page(bar, page.dma_handle())?;
@@ -94,26 +99,77 @@ impl SysmemFlush {
}
}
pub(crate) struct FbRange(Range<u64>);
impl FbRange {
pub(crate) fn len(&self) -> u64 {
self.0.end - self.0.start
}
}
impl From<Range<u64>> for FbRange {
fn from(range: Range<u64>) -> Self {
Self(range)
}
}
impl Deref for FbRange {
type Target = Range<u64>;
fn deref(&self) -> &Self::Target {
&self.0
}
}
impl fmt::Debug for FbRange {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
// Use alternate format ({:#?}) to include size, compact format ({:?}) for just the range.
if f.alternate() {
let size = self.len();
if size < usize_as_u64(SZ_1M) {
let size_kib = size / usize_as_u64(SZ_1K);
f.write_fmt(fmt!(
"{:#x}..{:#x} ({} KiB)",
self.0.start,
self.0.end,
size_kib
))
} else {
let size_mib = size / usize_as_u64(SZ_1M);
f.write_fmt(fmt!(
"{:#x}..{:#x} ({} MiB)",
self.0.start,
self.0.end,
size_mib
))
}
} else {
f.write_fmt(fmt!("{:#x}..{:#x}", self.0.start, self.0.end))
}
}
}
/// Layout of the GPU framebuffer memory.
///
/// Contains ranges of GPU memory reserved for a given purpose during the GSP boot process.
#[derive(Debug)]
pub(crate) struct FbLayout {
/// Range of the framebuffer. Starts at `0`.
pub(crate) fb: Range<u64>,
pub(crate) fb: FbRange,
/// VGA workspace, small area of reserved memory at the end of the framebuffer.
pub(crate) vga_workspace: Range<u64>,
pub(crate) vga_workspace: FbRange,
/// FRTS range.
pub(crate) frts: Range<u64>,
pub(crate) frts: FbRange,
/// Memory area containing the GSP bootloader image.
pub(crate) boot: Range<u64>,
pub(crate) boot: FbRange,
/// Memory area containing the GSP firmware image.
pub(crate) elf: Range<u64>,
pub(crate) elf: FbRange,
/// WPR2 heap.
pub(crate) wpr2_heap: Range<u64>,
pub(crate) wpr2_heap: FbRange,
/// WPR2 region range, starting with an instance of `GspFwWprMeta`.
pub(crate) wpr2: Range<u64>,
pub(crate) heap: Range<u64>,
pub(crate) wpr2: FbRange,
pub(crate) heap: FbRange,
pub(crate) vf_partition_count: u8,
}
@@ -125,7 +181,7 @@ impl FbLayout {
let fb = {
let fb_size = hal.vidmem_size(bar);
0..fb_size
FbRange(0..fb_size)
};
let vga_workspace = {
@@ -134,7 +190,10 @@ impl FbLayout {
let base = fb.end - NV_PRAMIN_SIZE;
if hal.supports_display(bar) {
match regs::NV_PDISP_VGA_WORKSPACE_BASE::read(bar).vga_workspace_addr() {
match bar
.read(regs::NV_PDISP_VGA_WORKSPACE_BASE)
.vga_workspace_addr()
{
Some(addr) => {
if addr < base {
const VBIOS_WORKSPACE_SIZE: u64 = usize_as_u64(SZ_128K);
@@ -152,7 +211,7 @@ impl FbLayout {
}
};
vga_base..fb.end
FbRange(vga_base..fb.end)
};
let frts = {
@@ -160,7 +219,7 @@ impl FbLayout {
const FRTS_SIZE: u64 = usize_as_u64(SZ_1M);
let frts_base = vga_workspace.start.align_down(FRTS_DOWN_ALIGN) - FRTS_SIZE;
frts_base..frts_base + FRTS_SIZE
FbRange(frts_base..frts_base + FRTS_SIZE)
};
let boot = {
@@ -168,7 +227,7 @@ impl FbLayout {
let bootloader_size = u64::from_safe_cast(gsp_fw.bootloader.ucode.size());
let bootloader_base = (frts.start - bootloader_size).align_down(BOOTLOADER_DOWN_ALIGN);
bootloader_base..bootloader_base + bootloader_size
FbRange(bootloader_base..bootloader_base + bootloader_size)
};
let elf = {
@@ -176,7 +235,7 @@ impl FbLayout {
let elf_size = u64::from_safe_cast(gsp_fw.size);
let elf_addr = (boot.start - elf_size).align_down(ELF_DOWN_ALIGN);
elf_addr..elf_addr + elf_size
FbRange(elf_addr..elf_addr + elf_size)
};
let wpr2_heap = {
@@ -185,7 +244,7 @@ impl FbLayout {
gsp::LibosParams::from_chipset(chipset).wpr_heap_size(chipset, fb.end);
let wpr2_heap_addr = (elf.start - wpr2_heap_size).align_down(WPR2_HEAP_DOWN_ALIGN);
wpr2_heap_addr..(elf.start).align_down(WPR2_HEAP_DOWN_ALIGN)
FbRange(wpr2_heap_addr..(elf.start).align_down(WPR2_HEAP_DOWN_ALIGN))
};
let wpr2 = {
@@ -193,13 +252,13 @@ impl FbLayout {
let wpr2_addr = (wpr2_heap.start - u64::from_safe_cast(size_of::<gsp::GspFwWprMeta>()))
.align_down(WPR2_DOWN_ALIGN);
wpr2_addr..frts.end
FbRange(wpr2_addr..frts.end)
};
let heap = {
const HEAP_SIZE: u64 = usize_as_u64(SZ_1M);
wpr2.start - HEAP_SIZE..wpr2.start
FbRange(wpr2.start - HEAP_SIZE..wpr2.start)
};
Ok(Self {

Some files were not shown because too many files have changed in this diff Show More