t2linux/kernel
0
0
Fork 0
mirror of https://github.com/t2linux/kernel.git synced 2026-04-30 13:48:59 -07:00
Code Issues Packages Projects Releases Wiki Activity

Merge tag 'iommu-updates-v5.14' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu

Browse Source 
Pull iommu updates from Joerg Roedel:

 - SMMU Updates from Will Deacon:

     - SMMUv3:
        - Support stalling faults for platform devices
        - Decrease defaults sizes for the event and PRI queues
     - SMMUv2:
        - Support for a new '->probe_finalize' hook, needed by Nvidia
        - Even more Qualcomm compatible strings
        - Avoid Adreno TTBR1 quirk for DB820C platform

 - Intel VT-d updates from Lu Baolu:

     - Convert Intel IOMMU to use sva_lib helpers in iommu core
     - ftrace and debugfs supports for page fault handling
     - Support asynchronous nested capabilities
     - Various misc cleanups

 - Support for new VIOT ACPI table to make the VirtIO IOMMU
   available on x86

 - Add the amd_iommu=force_enable command line option to enable
   the IOMMU on platforms where they are known to cause problems

 - Support for version 2 of the Rockchip IOMMU

 - Various smaller fixes, cleanups and refactorings

* tag 'iommu-updates-v5.14' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu: (66 commits)
  iommu/virtio: Enable x86 support
  iommu/dma: Pass address limit rather than size to iommu_setup_dma_ops()
  ACPI: Add driver for the VIOT table
  ACPI: Move IOMMU setup code out of IORT
  ACPI: arm64: Move DMA setup operations out of IORT
  iommu/vt-d: Fix dereference of pointer info before it is null checked
  iommu: Update "iommu.strict" documentation
  iommu/arm-smmu: Check smmu->impl pointer before dereferencing
  iommu/arm-smmu-v3: Remove unnecessary oom message
  iommu/arm-smmu: Fix arm_smmu_device refcount leak in address translation
  iommu/arm-smmu: Fix arm_smmu_device refcount leak when arm_smmu_rpm_get fails
  iommu/vt-d: Fix linker error on 32-bit
  iommu/vt-d: No need to typecast
  iommu/vt-d: Define counter explicitly as unsigned int
  iommu/vt-d: Remove unnecessary braces
  iommu/vt-d: Removed unused iommu_count in dmar domain
  iommu/vt-d: Use bitfields for DMAR capabilities
  iommu/vt-d: Use DEVICE_ATTR_RO macro
  iommu/vt-d: Fix out-bounds-warning in intel/svm.c
  iommu/vt-d: Add PRQ handling latency sampling
  ...
This commit is contained in:
Linus Torvalds
2021-07-02 13:22:47 -07:00
parent 35e43538af 2b9d8e3e9a
commit cd3eb7efaa
56 changed files with 2175 additions and 784 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Documentation/admin-guide/kernel-parameters.txt
+8 -1
View File
@@ -301,6 +301,9 @@
allowed anymore to lift isolation
requirements as needed. This option
does not override iommu=pt
force_enable - Force enable the IOMMU on platforms known
to be buggy with IOMMU enabled. Use this
option with care.
amd_iommu_dump= [HW,X86-64]
Enable AMD IOMMU driver option to dump the ACPI table
@@ -2031,7 +2034,7 @@
forcing Dual Address Cycle for PCI cards supporting
greater than 32-bit addressing.
iommu.strict= [ARM64] Configure TLB invalidation behaviour
iommu.strict= [ARM64, X86] Configure TLB invalidation behaviour
Format: { "0" | "1" }
0 - Lazy mode.
Request that DMA unmap operations use deferred
@@ -2042,6 +2045,10 @@
1 - Strict mode (default).
DMA unmap operations invalidate IOMMU hardware TLBs
synchronously.
Note: on x86, the default behaviour depends on the
equivalent driver-specific parameters, but a strict
mode explicitly specified by either method takes
precedence.
iommu.passthrough=
[ARM64, X86] Configure DMA to bypass the IOMMU by default.
Documentation/devicetree/bindings/iommu/iommu.txt
+18
View File
@@ -92,6 +92,24 @@ Optional properties:
tagging DMA transactions with an address space identifier. By default,
this is 0, which means that the device only has one address space.
- dma-can-stall: When present, the master can wait for a transaction to
complete for an indefinite amount of time. Upon translation fault some
IOMMUs, instead of aborting the translation immediately, may first
notify the driver and keep the transaction in flight. This allows the OS
to inspect the fault and, for example, make physical pages resident
before updating the mappings and completing the transaction. Such IOMMU
accepts a limited number of simultaneous stalled transactions before
having to either put back-pressure on the master, or abort new faulting
transactions.
Firmware has to opt-in stalling, because most buses and masters don't
support it. In particular it isn't compatible with PCI, where
transactions have to complete before a time limit. More generally it
won't work in systems and masters that haven't been designed for
stalling. For example the OS, in order to handle a stalled transaction,
may attempt to retrieve pages from secondary storage in a stalled
domain, leading to a deadlock.
Notes:
======
Documentation/devicetree/bindings/iommu/rockchip,iommu.txt
-38
View File
@@ -1,38 +0,0 @@
Rockchip IOMMU
==============
A Rockchip DRM iommu translates io virtual addresses to physical addresses for
its master device. Each slave device is bound to a single master device, and
shares its clocks, power domain and irq.
Required properties:
- compatible : Should be "rockchip,iommu"
- reg : Address space for the configuration registers
- interrupts : Interrupt specifier for the IOMMU instance
- interrupt-names : Interrupt name for the IOMMU instance
- #iommu-cells : Should be <0>. This indicates the iommu is a
"single-master" device, and needs no additional information
to associate with its master device. See:
Documentation/devicetree/bindings/iommu/iommu.txt
- clocks : A list of clocks required for the IOMMU to be accessible by
the host CPU.
- clock-names : Should contain the following:
"iface" - Main peripheral bus clock (PCLK/HCL) (required)
"aclk" - AXI bus clock (required)
Optional properties:
- rockchip,disable-mmu-reset : Don't use the mmu reset operation.
Some mmu instances may produce unexpected results
when the reset operation is used.
Example:
vopl_mmu: iommu@ff940300 {
compatible = "rockchip,iommu";
reg = <0xff940300 0x100>;
interrupts = <GIC_SPI 16 IRQ_TYPE_LEVEL_HIGH>;
interrupt-names = "vopl_mmu";
clocks = <&cru ACLK_VOP1>, <&cru HCLK_VOP1>;
clock-names = "aclk", "iface";
#iommu-cells = <0>;
};
Documentation/devicetree/bindings/iommu/rockchip,iommu.yaml
+85
View File
@@ -0,0 +1,85 @@
# SPDX-License-Identifier: GPL-2.0-only
%YAML 1.2
---
$id: http://devicetree.org/schemas/iommu/rockchip,iommu.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
title: Rockchip IOMMU
maintainers:
- Heiko Stuebner <heiko@sntech.de>
description: |+
A Rockchip DRM iommu translates io virtual addresses to physical addresses for
its master device. Each slave device is bound to a single master device and
shares its clocks, power domain and irq.
For information on assigning IOMMU controller to its peripheral devices,
see generic IOMMU bindings.
properties:
compatible:
enum:
- rockchip,iommu
- rockchip,rk3568-iommu
reg:
items:
- description: configuration registers for MMU instance 0
- description: configuration registers for MMU instance 1
minItems: 1
maxItems: 2
interrupts:
items:
- description: interruption for MMU instance 0
- description: interruption for MMU instance 1
minItems: 1
maxItems: 2
clocks:
items:
- description: Core clock
- description: Interface clock
clock-names:
items:
- const: aclk
- const: iface
"#iommu-cells":
const: 0
power-domains:
maxItems: 1
rockchip,disable-mmu-reset:
$ref: /schemas/types.yaml#/definitions/flag
description: |
Do not use the mmu reset operation.
Some mmu instances may produce unexpected results
when the reset operation is used.
required:
- compatible
- reg
- interrupts
- clocks
- clock-names
- "#iommu-cells"
additionalProperties: false
examples:
- |
#include <dt-bindings/clock/rk3399-cru.h>
#include <dt-bindings/interrupt-controller/arm-gic.h>
vopl_mmu: iommu@ff940300 {
compatible = "rockchip,iommu";
reg = <0xff940300 0x100>;
interrupts = <GIC_SPI 16 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&cru ACLK_VOP1>, <&cru HCLK_VOP1>;
clock-names = "aclk", "iface";
#iommu-cells = <0>;
};
MAINTAINERS
+8
View File
@@ -431,6 +431,14 @@ W: https://01.org/linux-acpi
B: https://bugzilla.kernel.org
F: drivers/acpi/acpi_video.c
ACPI VIOT DRIVER
M: Jean-Philippe Brucker <jean-philippe@linaro.org>
L: linux-acpi@vger.kernel.org
L: iommu@lists.linux-foundation.org
S: Maintained
F: drivers/acpi/viot.c
F: include/linux/acpi_viot.h
ACPI WMI DRIVER
L: platform-driver-x86@vger.kernel.org
S: Orphan
arch/arm64/boot/dts/qcom/msm8996.dtsi
+1 -1
View File
@@ -1136,7 +1136,7 @@
};
adreno_smmu: iommu@b40000 {
compatible = "qcom,msm8996-smmu-v2", "qcom,smmu-v2";
compatible = "qcom,msm8996-smmu-v2", "qcom,adreno-smmu", "qcom,smmu-v2";
reg = <0x00b40000 0x10000>;
#global-interrupts = <1>;
arch/arm64/mm/dma-mapping.c
+1 -1
View File
@@ -50,7 +50,7 @@ void arch_setup_dma_ops(struct device *dev, u64 dma_base, u64 size,
dev->dma_coherent = coherent;
if (iommu)
iommu_setup_dma_ops(dev, dma_base, size);
iommu_setup_dma_ops(dev, dma_base, dma_base + size - 1);
#ifdef CONFIG_XEN
if (xen_swiotlb_detect())
drivers/acpi/Kconfig
+3
View File
@@ -526,6 +526,9 @@ endif
source "drivers/acpi/pmic/Kconfig"
config ACPI_VIOT
bool
endif # ACPI
config X86_PM_TIMER
drivers/acpi/Makefile
+2
View File
@@ -124,3 +124,5 @@ video-objs += acpi_video.o video_detect.o
obj-y += dptf/
obj-$(CONFIG_ARM64) += arm64/
obj-$(CONFIG_ACPI_VIOT) += viot.o
drivers/acpi/arm64/Makefile
+1
View File
@@ -1,3 +1,4 @@
# SPDX-License-Identifier: GPL-2.0-only
obj-$(CONFIG_ACPI_IORT) += iort.o
obj-$(CONFIG_ACPI_GTDT) += gtdt.o
obj-y += dma.o
drivers/acpi/arm64/dma.c
+50
View File
@@ -0,0 +1,50 @@
// SPDX-License-Identifier: GPL-2.0-only
#include <linux/acpi.h>
#include <linux/acpi_iort.h>
#include <linux/device.h>
#include <linux/dma-direct.h>
void acpi_arch_dma_setup(struct device *dev, u64 *dma_addr, u64 *dma_size)
{
int ret;
u64 end, mask;
u64 dmaaddr = 0, size = 0, offset = 0;
/*
* If @dev is expected to be DMA-capable then the bus code that created
* it should have initialised its dma_mask pointer by this point. For
* now, we'll continue the legacy behaviour of coercing it to the
* coherent mask if not, but we'll no longer do so quietly.
*/
if (!dev->dma_mask) {
dev_warn(dev, "DMA mask not set\n");
dev->dma_mask = &dev->coherent_dma_mask;
}
if (dev->coherent_dma_mask)
size = max(dev->coherent_dma_mask, dev->coherent_dma_mask + 1);
else
size = 1ULL << 32;
ret = acpi_dma_get_range(dev, &dmaaddr, &offset, &size);
if (ret == -ENODEV)
ret = iort_dma_get_ranges(dev, &size);
if (!ret) {
/*
* Limit coherent and dma mask based on size retrieved from
* firmware.
*/
end = dmaaddr + size - 1;
mask = DMA_BIT_MASK(ilog2(end) + 1);
dev->bus_dma_limit = end;
dev->coherent_dma_mask = min(dev->coherent_dma_mask, mask);
*dev->dma_mask = min(*dev->dma_mask, mask);
}
*dma_addr = dmaaddr;
*dma_size = size;
ret = dma_direct_set_offset(dev, dmaaddr + offset, dmaaddr, size);
dev_dbg(dev, "dma_offset(%#08llx)%s\n", offset, ret ? " failed!" : "");
}
drivers/acpi/arm64/iort.c
+19 -113
View File
@@ -806,23 +806,6 @@ static struct acpi_iort_node *iort_get_msi_resv_iommu(struct device *dev)
return NULL;
}
static inline const struct iommu_ops *iort_fwspec_iommu_ops(struct device *dev)
{
struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev);
return (fwspec && fwspec->ops) ? fwspec->ops : NULL;
}
static inline int iort_add_device_replay(struct device *dev)
{
int err = 0;
if (dev->bus && !device_iommu_mapped(dev))
err = iommu_probe_device(dev);
return err;
}
/**
* iort_iommu_msi_get_resv_regions - Reserved region driver helper
* @dev: Device from iommu_get_resv_regions()
@@ -900,18 +883,6 @@ static inline bool iort_iommu_driver_enabled(u8 type)
}
}
static int arm_smmu_iort_xlate(struct device *dev, u32 streamid,
struct fwnode_handle *fwnode,
const struct iommu_ops *ops)
{
int ret = iommu_fwspec_init(dev, fwnode, ops);
if (!ret)
ret = iommu_fwspec_add_ids(dev, &streamid, 1);
return ret;
}
static bool iort_pci_rc_supports_ats(struct acpi_iort_node *node)
{
struct acpi_iort_root_complex *pci_rc;
@@ -946,7 +917,7 @@ static int iort_iommu_xlate(struct device *dev, struct acpi_iort_node *node,
return iort_iommu_driver_enabled(node->type) ?
-EPROBE_DEFER : -ENODEV;
return arm_smmu_iort_xlate(dev, streamid, iort_fwnode, ops);
return acpi_iommu_fwspec_init(dev, streamid, iort_fwnode, ops);
}
struct iort_pci_alias_info {
@@ -968,13 +939,15 @@ static int iort_pci_iommu_init(struct pci_dev *pdev, u16 alias, void *data)
static void iort_named_component_init(struct device *dev,
struct acpi_iort_node *node)
{
struct property_entry props[2] = {};
struct property_entry props[3] = {};
struct acpi_iort_named_component *nc;
nc = (struct acpi_iort_named_component *)node->node_data;
props[0] = PROPERTY_ENTRY_U32("pasid-num-bits",
FIELD_GET(ACPI_IORT_NC_PASID_BITS,
nc->node_flags));
if (nc->node_flags & ACPI_IORT_NC_STALL_SUPPORTED)
props[1] = PROPERTY_ENTRY_BOOL("dma-can-stall");
if (device_create_managed_software_node(dev, props, NULL))
dev_warn(dev, "Could not add device properties\n");
@@ -1020,24 +993,13 @@ static int iort_nc_iommu_map_id(struct device *dev,
* @dev: device to configure
* @id_in: optional input id const value pointer
*
* Returns: iommu_ops pointer on configuration success
* NULL on configuration failure
* Returns: 0 on success, <0 on failure
*/
const struct iommu_ops *iort_iommu_configure_id(struct device *dev,
const u32 *id_in)
int iort_iommu_configure_id(struct device *dev, const u32 *id_in)
{
struct acpi_iort_node *node;
const struct iommu_ops *ops;
int err = -ENODEV;
/*
* If we already translated the fwspec there
* is nothing left to do, return the iommu_ops.
*/
ops = iort_fwspec_iommu_ops(dev);
if (ops)
return ops;
if (dev_is_pci(dev)) {
struct iommu_fwspec *fwspec;
struct pci_bus *bus = to_pci_dev(dev)->bus;
@@ -1046,7 +1008,7 @@ const struct iommu_ops *iort_iommu_configure_id(struct device *dev,
node = iort_scan_node(ACPI_IORT_NODE_PCI_ROOT_COMPLEX,
iort_match_node_callback, &bus->dev);
if (!node)
return NULL;
return -ENODEV;
info.node = node;
err = pci_for_each_dma_alias(to_pci_dev(dev),
@@ -1059,7 +1021,7 @@ const struct iommu_ops *iort_iommu_configure_id(struct device *dev,
node = iort_scan_node(ACPI_IORT_NODE_NAMED_COMPONENT,
iort_match_node_callback, dev);
if (!node)
return NULL;
return -ENODEV;
err = id_in ? iort_nc_iommu_map_id(dev, node, id_in) :
iort_nc_iommu_map(dev, node);
@@ -1068,32 +1030,14 @@ const struct iommu_ops *iort_iommu_configure_id(struct device *dev,
iort_named_component_init(dev, node);
}
/*
* If we have reason to believe the IOMMU driver missed the initial
* add_device callback for dev, replay it to get things in order.
*/
if (!err) {
ops = iort_fwspec_iommu_ops(dev);
err = iort_add_device_replay(dev);
}
/* Ignore all other errors apart from EPROBE_DEFER */
if (err == -EPROBE_DEFER) {
ops = ERR_PTR(err);
} else if (err) {
dev_dbg(dev, "Adding to IOMMU failed: %d\n", err);
ops = NULL;
}
return ops;
return err;
}
#else
int iort_iommu_msi_get_resv_regions(struct device *dev, struct list_head *head)
{ return 0; }
const struct iommu_ops *iort_iommu_configure_id(struct device *dev,
const u32 *input_id)
{ return NULL; }
int iort_iommu_configure_id(struct device *dev, const u32 *input_id)
{ return -ENODEV; }
#endif
static int nc_dma_get_range(struct device *dev, u64 *size)
@@ -1144,56 +1088,18 @@ static int rc_dma_get_range(struct device *dev, u64 *size)
}
/**
* iort_dma_setup() - Set-up device DMA parameters.
* iort_dma_get_ranges() - Look up DMA addressing limit for the device
* @dev: device to lookup
* @size: DMA range size result pointer
*
* @dev: device to configure
* @dma_addr: device DMA address result pointer
* @dma_size: DMA range size result pointer
* Return: 0 on success, an error otherwise.
*/
void iort_dma_setup(struct device *dev, u64 *dma_addr, u64 *dma_size)
int iort_dma_get_ranges(struct device *dev, u64 *size)
{
u64 end, mask, dmaaddr = 0, size = 0, offset = 0;
int ret;
/*
* If @dev is expected to be DMA-capable then the bus code that created
* it should have initialised its dma_mask pointer by this point. For
* now, we'll continue the legacy behaviour of coercing it to the
* coherent mask if not, but we'll no longer do so quietly.
*/
if (!dev->dma_mask) {
dev_warn(dev, "DMA mask not set\n");
dev->dma_mask = &dev->coherent_dma_mask;
}
if (dev->coherent_dma_mask)
size = max(dev->coherent_dma_mask, dev->coherent_dma_mask + 1);
if (dev_is_pci(dev))
return rc_dma_get_range(dev, size);
else
size = 1ULL << 32;
ret = acpi_dma_get_range(dev, &dmaaddr, &offset, &size);
if (ret == -ENODEV)
ret = dev_is_pci(dev) ? rc_dma_get_range(dev, &size)
: nc_dma_get_range(dev, &size);
if (!ret) {
/*
* Limit coherent and dma mask based on size retrieved from
* firmware.
*/
end = dmaaddr + size - 1;
mask = DMA_BIT_MASK(ilog2(end) + 1);
dev->bus_dma_limit = end;
dev->coherent_dma_mask = min(dev->coherent_dma_mask, mask);
*dev->dma_mask = min(*dev->dma_mask, mask);
}
*dma_addr = dmaaddr;
*dma_size = size;
ret = dma_direct_set_offset(dev, dmaaddr + offset, dmaaddr, size);
dev_dbg(dev, "dma_offset(%#08llx)%s\n", offset, ret ? " failed!" : "");
return nc_dma_get_range(dev, size);
}
static void __init acpi_iort_register_irq(int hwirq, const char *name,
drivers/acpi/bus.c
+2
View File
@@ -27,6 +27,7 @@
#include <linux/dmi.h>
#endif
#include <linux/acpi_iort.h>
#include <linux/acpi_viot.h>
#include <linux/pci.h>
#include <acpi/apei.h>
#include <linux/suspend.h>
@@ -1335,6 +1336,7 @@ static int __init acpi_init(void)
acpi_wakeup_device_init();
acpi_debugger_init();
acpi_setup_sb_notify_handler();
acpi_viot_init();
return 0;
}
drivers/acpi/scan.c
+76 -2
View File
@@ -11,6 +11,8 @@
#include <linux/kernel.h>
#include <linux/acpi.h>
#include <linux/acpi_iort.h>
#include <linux/acpi_viot.h>
#include <linux/iommu.h>
#include <linux/signal.h>
#include <linux/kthread.h>
#include <linux/dmi.h>
@@ -1526,6 +1528,78 @@ int acpi_dma_get_range(struct device *dev, u64 *dma_addr, u64 *offset,
return ret >= 0 ? 0 : ret;
}
#ifdef CONFIG_IOMMU_API
int acpi_iommu_fwspec_init(struct device *dev, u32 id,
struct fwnode_handle *fwnode,
const struct iommu_ops *ops)
{
int ret = iommu_fwspec_init(dev, fwnode, ops);
if (!ret)
ret = iommu_fwspec_add_ids(dev, &id, 1);
return ret;
}
static inline const struct iommu_ops *acpi_iommu_fwspec_ops(struct device *dev)
{
struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev);
return fwspec ? fwspec->ops : NULL;
}
static const struct iommu_ops *acpi_iommu_configure_id(struct device *dev,
const u32 *id_in)
{
int err;
const struct iommu_ops *ops;
/*
* If we already translated the fwspec there is nothing left to do,
* return the iommu_ops.
*/
ops = acpi_iommu_fwspec_ops(dev);
if (ops)
return ops;
err = iort_iommu_configure_id(dev, id_in);
if (err && err != -EPROBE_DEFER)
err = viot_iommu_configure(dev);
/*
* If we have reason to believe the IOMMU driver missed the initial
* iommu_probe_device() call for dev, replay it to get things in order.
*/
if (!err && dev->bus && !device_iommu_mapped(dev))
err = iommu_probe_device(dev);
/* Ignore all other errors apart from EPROBE_DEFER */
if (err == -EPROBE_DEFER) {
return ERR_PTR(err);
} else if (err) {
dev_dbg(dev, "Adding to IOMMU failed: %d\n", err);
return NULL;
}
return acpi_iommu_fwspec_ops(dev);
}
#else /* !CONFIG_IOMMU_API */
int acpi_iommu_fwspec_init(struct device *dev, u32 id,
struct fwnode_handle *fwnode,
const struct iommu_ops *ops)
{
return -ENODEV;
}
static const struct iommu_ops *acpi_iommu_configure_id(struct device *dev,
const u32 *id_in)
{
return NULL;
}
#endif /* !CONFIG_IOMMU_API */
/**
* acpi_dma_configure_id - Set-up DMA configuration for the device.
* @dev: The pointer to the device
@@ -1543,9 +1617,9 @@ int acpi_dma_configure_id(struct device *dev, enum dev_dma_attr attr,
return 0;
}
iort_dma_setup(dev, &dma_addr, &size);
acpi_arch_dma_setup(dev, &dma_addr, &size);
iommu = iort_iommu_configure_id(dev, input_id);
iommu = acpi_iommu_configure_id(dev, input_id);
if (PTR_ERR(iommu) == -EPROBE_DEFER)
return -EPROBE_DEFER;
drivers/acpi/viot.c
+366
View File
@@ -0,0 +1,366 @@
// SPDX-License-Identifier: GPL-2.0
/*
* Virtual I/O topology
*
* The Virtual I/O Translation Table (VIOT) describes the topology of
* para-virtual IOMMUs and the endpoints they manage. The OS uses it to
* initialize devices in the right order, preventing endpoints from issuing DMA
* before their IOMMU is ready.
*
* When binding a driver to a device, before calling the device driver's probe()
* method, the driver infrastructure calls dma_configure(). At that point the
* VIOT driver looks for an IOMMU associated to the device in the VIOT table.
* If an IOMMU exists and has been initialized, the VIOT driver initializes the
* device's IOMMU fwspec, allowing the DMA infrastructure to invoke the IOMMU
* ops when the device driver configures DMA mappings. If an IOMMU exists and
* hasn't yet been initialized, VIOT returns -EPROBE_DEFER to postpone probing
* the device until the IOMMU is available.
*/
#define pr_fmt(fmt) "ACPI: VIOT: " fmt
#include <linux/acpi_viot.h>
#include <linux/dma-iommu.h>
#include <linux/fwnode.h>
#include <linux/iommu.h>
#include <linux/list.h>
#include <linux/pci.h>
#include <linux/platform_device.h>
struct viot_iommu {
/* Node offset within the table */
unsigned int offset;
struct fwnode_handle *fwnode;
struct list_head list;
};
struct viot_endpoint {
union {
/* PCI range */
struct {
u16 segment_start;
u16 segment_end;
u16 bdf_start;
u16 bdf_end;
};
/* MMIO */
u64 address;
};
u32 endpoint_id;
struct viot_iommu *viommu;
struct list_head list;
};
static struct acpi_table_viot *viot;
static LIST_HEAD(viot_iommus);
static LIST_HEAD(viot_pci_ranges);
static LIST_HEAD(viot_mmio_endpoints);
static int __init viot_check_bounds(const struct acpi_viot_header *hdr)
{
struct acpi_viot_header *start, *end, *hdr_end;
start = ACPI_ADD_PTR(struct acpi_viot_header, viot,
max_t(size_t, sizeof(*viot), viot->node_offset));
end = ACPI_ADD_PTR(struct acpi_viot_header, viot, viot->header.length);
hdr_end = ACPI_ADD_PTR(struct acpi_viot_header, hdr, sizeof(*hdr));
if (hdr < start || hdr_end > end) {
pr_err(FW_BUG "Node pointer overflows\n");
return -EOVERFLOW;
}
if (hdr->length < sizeof(*hdr)) {
pr_err(FW_BUG "Empty node\n");
return -EINVAL;
}
return 0;
}
static int __init viot_get_pci_iommu_fwnode(struct viot_iommu *viommu,
u16 segment, u16 bdf)
{
struct pci_dev *pdev;
struct fwnode_handle *fwnode;
pdev = pci_get_domain_bus_and_slot(segment, PCI_BUS_NUM(bdf),
bdf & 0xff);
if (!pdev) {
pr_err("Could not find PCI IOMMU\n");
return -ENODEV;
}
fwnode = pdev->dev.fwnode;
if (!fwnode) {
/*
* PCI devices aren't necessarily described by ACPI. Create a
* fwnode so the IOMMU subsystem can identify this device.
*/
fwnode = acpi_alloc_fwnode_static();
if (!fwnode) {
pci_dev_put(pdev);
return -ENOMEM;
}
set_primary_fwnode(&pdev->dev, fwnode);
}
viommu->fwnode = pdev->dev.fwnode;
pci_dev_put(pdev);
return 0;
}
static int __init viot_get_mmio_iommu_fwnode(struct viot_iommu *viommu,
u64 address)
{
struct acpi_device *adev;
struct resource res = {
.start = address,
.end = address,
.flags = IORESOURCE_MEM,
};
adev = acpi_resource_consumer(&res);
if (!adev) {
pr_err("Could not find MMIO IOMMU\n");
return -EINVAL;
}
viommu->fwnode = &adev->fwnode;
return 0;
}
static struct viot_iommu * __init viot_get_iommu(unsigned int offset)
{
int ret;
struct viot_iommu *viommu;
struct acpi_viot_header *hdr = ACPI_ADD_PTR(struct acpi_viot_header,
viot, offset);
union {
struct acpi_viot_virtio_iommu_pci pci;
struct acpi_viot_virtio_iommu_mmio mmio;
} *node = (void *)hdr;
list_for_each_entry(viommu, &viot_iommus, list)
if (viommu->offset == offset)
return viommu;
if (viot_check_bounds(hdr))
return NULL;
viommu = kzalloc(sizeof(*viommu), GFP_KERNEL);
if (!viommu)
return NULL;
viommu->offset = offset;
switch (hdr->type) {
case ACPI_VIOT_NODE_VIRTIO_IOMMU_PCI:
if (hdr->length < sizeof(node->pci))
goto err_free;
ret = viot_get_pci_iommu_fwnode(viommu, node->pci.segment,
node->pci.bdf);
break;
case ACPI_VIOT_NODE_VIRTIO_IOMMU_MMIO:
if (hdr->length < sizeof(node->mmio))
goto err_free;
ret = viot_get_mmio_iommu_fwnode(viommu,
node->mmio.base_address);
break;
default:
ret = -EINVAL;
}
if (ret)
goto err_free;
list_add(&viommu->list, &viot_iommus);
return viommu;
err_free:
kfree(viommu);
return NULL;
}
static int __init viot_parse_node(const struct acpi_viot_header *hdr)
{
int ret = -EINVAL;
struct list_head *list;
struct viot_endpoint *ep;
union {
struct acpi_viot_mmio mmio;
struct acpi_viot_pci_range pci;
} *node = (void *)hdr;
if (viot_check_bounds(hdr))
return -EINVAL;
if (hdr->type == ACPI_VIOT_NODE_VIRTIO_IOMMU_PCI ||
hdr->type == ACPI_VIOT_NODE_VIRTIO_IOMMU_MMIO)
return 0;
ep = kzalloc(sizeof(*ep), GFP_KERNEL);
if (!ep)
return -ENOMEM;
switch (hdr->type) {
case ACPI_VIOT_NODE_PCI_RANGE:
if (hdr->length < sizeof(node->pci)) {
pr_err(FW_BUG "Invalid PCI node size\n");
goto err_free;
}
ep->segment_start = node->pci.segment_start;
ep->segment_end = node->pci.segment_end;
ep->bdf_start = node->pci.bdf_start;
ep->bdf_end = node->pci.bdf_end;
ep->endpoint_id = node->pci.endpoint_start;
ep->viommu = viot_get_iommu(node->pci.output_node);
list = &viot_pci_ranges;
break;
case ACPI_VIOT_NODE_MMIO:
if (hdr->length < sizeof(node->mmio)) {
pr_err(FW_BUG "Invalid MMIO node size\n");
goto err_free;
}
ep->address = node->mmio.base_address;
ep->endpoint_id = node->mmio.endpoint;
ep->viommu = viot_get_iommu(node->mmio.output_node);
list = &viot_mmio_endpoints;
break;
default:
pr_warn("Unsupported node %x\n", hdr->type);
ret = 0;
goto err_free;
}
if (!ep->viommu) {
pr_warn("No IOMMU node found\n");
/*
* A future version of the table may use the node for other
* purposes. Keep parsing.
*/
ret = 0;
goto err_free;
}
list_add(&ep->list, list);
return 0;
err_free:
kfree(ep);
return ret;
}
/**
* acpi_viot_init - Parse the VIOT table
*
* Parse the VIOT table, prepare the list of endpoints to be used during DMA
* setup of devices.
*/
void __init acpi_viot_init(void)
{
int i;
acpi_status status;
struct acpi_table_header *hdr;
struct acpi_viot_header *node;
status = acpi_get_table(ACPI_SIG_VIOT, 0, &hdr);
if (ACPI_FAILURE(status)) {
if (status != AE_NOT_FOUND) {
const char *msg = acpi_format_exception(status);
pr_err("Failed to get table, %s\n", msg);
}
return;
}
viot = (void *)hdr;
node = ACPI_ADD_PTR(struct acpi_viot_header, viot, viot->node_offset);
for (i = 0; i < viot->node_count; i++) {
if (viot_parse_node(node))
return;
node = ACPI_ADD_PTR(struct acpi_viot_header, node,
node->length);
}
acpi_put_table(hdr);
}
static int viot_dev_iommu_init(struct device *dev, struct viot_iommu *viommu,
u32 epid)
{
const struct iommu_ops *ops;
if (!viommu)
return -ENODEV;
/* We're not translating ourself */
if (viommu->fwnode == dev->fwnode)
return -EINVAL;
ops = iommu_ops_from_fwnode(viommu->fwnode);
if (!ops)
return IS_ENABLED(CONFIG_VIRTIO_IOMMU) ?
-EPROBE_DEFER : -ENODEV;
return acpi_iommu_fwspec_init(dev, epid, viommu->fwnode, ops);
}
static int viot_pci_dev_iommu_init(struct pci_dev *pdev, u16 dev_id, void *data)
{
u32 epid;
struct viot_endpoint *ep;
u32 domain_nr = pci_domain_nr(pdev->bus);
list_for_each_entry(ep, &viot_pci_ranges, list) {
if (domain_nr >= ep->segment_start &&
domain_nr <= ep->segment_end &&
dev_id >= ep->bdf_start &&
dev_id <= ep->bdf_end) {
epid = ((domain_nr - ep->segment_start) << 16) +
dev_id - ep->bdf_start + ep->endpoint_id;
/*
* If we found a PCI range managed by the viommu, we're
* the one that has to request ACS.
*/
pci_request_acs();
return viot_dev_iommu_init(&pdev->dev, ep->viommu,
epid);
}
}
return -ENODEV;
}
static int viot_mmio_dev_iommu_init(struct platform_device *pdev)
{
struct resource *mem;
struct viot_endpoint *ep;
mem = platform_get_resource(pdev, IORESOURCE_MEM, 0);
if (!mem)
return -ENODEV;
list_for_each_entry(ep, &viot_mmio_endpoints, list) {
if (ep->address == mem->start)
return viot_dev_iommu_init(&pdev->dev, ep->viommu,
ep->endpoint_id);
}
return -ENODEV;
}
/**
* viot_iommu_configure - Setup IOMMU ops for an endpoint described by VIOT
* @dev: the endpoint
*
* Return: 0 on success, <0 on failure
*/
int viot_iommu_configure(struct device *dev)
{
if (dev_is_pci(dev))
return pci_for_each_dma_alias(to_pci_dev(dev),
viot_pci_dev_iommu_init, NULL);
else if (dev_is_platform(dev))
return viot_mmio_dev_iommu_init(to_platform_device(dev));
return -ENODEV;
}
drivers/iommu/Kconfig
+3 -1
View File
@@ -400,9 +400,11 @@ config HYPERV_IOMMU
config VIRTIO_IOMMU
tristate "Virtio IOMMU driver"
depends on VIRTIO
depends on ARM64
depends on (ARM64 || X86)
select IOMMU_API
select IOMMU_DMA
select INTERVAL_TREE
select ACPI_VIOT if ACPI
help
Para-virtualised IOMMU driver with virtio.
drivers/iommu/amd/amd_iommu.h
-2
View File
@@ -11,8 +11,6 @@
#include "amd_iommu_types.h"
extern int amd_iommu_init_dma_ops(void);
extern int amd_iommu_init_passthrough(void);
extern irqreturn_t amd_iommu_int_thread(int irq, void *data);
extern irqreturn_t amd_iommu_int_handler(int irq, void *data);
extern void amd_iommu_apply_erratum_63(u16 devid);
drivers/iommu/amd/init.c
+11 -9
View File
@@ -153,7 +153,8 @@ int amd_iommu_guest_ir = AMD_IOMMU_GUEST_IR_VAPIC;
static int amd_iommu_xt_mode = IRQ_REMAP_XAPIC_MODE;
static bool amd_iommu_detected;
static bool __initdata amd_iommu_disabled;
static bool amd_iommu_disabled __initdata;
static bool amd_iommu_force_enable __initdata;
static int amd_iommu_target_ivhd_type;
u16 amd_iommu_last_bdf; /* largest PCI device id we have
@@ -231,7 +232,6 @@ enum iommu_init_state {
IOMMU_ENABLED,
IOMMU_PCI_INIT,
IOMMU_INTERRUPTS_EN,
IOMMU_DMA_OPS,
IOMMU_INITIALIZED,
IOMMU_NOT_FOUND,
IOMMU_INIT_ERROR,
@@ -1908,8 +1908,8 @@ static void print_iommu_info(void)
pci_info(pdev, "Found IOMMU cap 0x%x\n", iommu->cap_ptr);
if (iommu->cap & (1 << IOMMU_CAP_EFR)) {
pci_info(pdev, "Extended features (%#llx):",
iommu->features);
pr_info("Extended features (%#llx):", iommu->features);
for (i = 0; i < ARRAY_SIZE(feat_str); ++i) {
if (iommu_feature(iommu, (1ULL << i)))
pr_cont(" %s", feat_str[i]);
@@ -2817,7 +2817,7 @@ out:
return ret;
}
static bool detect_ivrs(void)
static bool __init detect_ivrs(void)
{
struct acpi_table_header *ivrs_base;
acpi_status status;
@@ -2834,6 +2834,9 @@ static bool detect_ivrs(void)
acpi_put_table(ivrs_base);
if (amd_iommu_force_enable)
goto out;
/* Don't use IOMMU if there is Stoney Ridge graphics */
for (i = 0; i < 32; i++) {
u32 pci_id;
@@ -2845,6 +2848,7 @@ static bool detect_ivrs(void)
}
}
out:
/* Make sure ACS will be enabled during PCI probe */
pci_request_acs();
@@ -2895,10 +2899,6 @@ static int __init state_next(void)
init_state = ret ? IOMMU_INIT_ERROR : IOMMU_INTERRUPTS_EN;
break;
case IOMMU_INTERRUPTS_EN:
ret = amd_iommu_init_dma_ops();
init_state = ret ? IOMMU_INIT_ERROR : IOMMU_DMA_OPS;
break;
case IOMMU_DMA_OPS:
init_state = IOMMU_INITIALIZED;
break;
case IOMMU_INITIALIZED:
@@ -3100,6 +3100,8 @@ static int __init parse_amd_iommu_options(char *str)
for (; *str; ++str) {
if (strncmp(str, "fullflush", 9) == 0)
amd_iommu_unmap_flush = true;
if (strncmp(str, "force_enable", 12) == 0)
amd_iommu_force_enable = true;
if (strncmp(str, "off", 3) == 0)
amd_iommu_disabled = true;
if (strncmp(str, "force_isolation", 15) == 0)
drivers/iommu/amd/iommu.c
+14 -19
View File
@@ -30,7 +30,6 @@
#include <linux/msi.h>
#include <linux/irqdomain.h>
#include <linux/percpu.h>
#include <linux/iova.h>
#include <linux/io-pgtable.h>
#include <asm/irq_remapping.h>
#include <asm/io_apic.h>
@@ -1713,7 +1712,7 @@ static void amd_iommu_probe_finalize(struct device *dev)
/* Domains are initialized for this device - have a look what we ended up with */
domain = iommu_get_domain_for_dev(dev);
if (domain->type == IOMMU_DOMAIN_DMA)
iommu_setup_dma_ops(dev, IOVA_START_PFN << PAGE_SHIFT, 0);
iommu_setup_dma_ops(dev, 0, U64_MAX);
else
set_dma_ops(dev, NULL);
}
@@ -1773,13 +1772,22 @@ void amd_iommu_domain_update(struct protection_domain *domain)
amd_iommu_domain_flush_complete(domain);
}
static void __init amd_iommu_init_dma_ops(void)
{
swiotlb = (iommu_default_passthrough() || sme_me_mask) ? 1 : 0;
if (amd_iommu_unmap_flush)
pr_info("IO/TLB flush on unmap enabled\n");
else
pr_info("Lazy IO/TLB flushing enabled\n");
iommu_set_dma_strict(amd_iommu_unmap_flush);
}
int __init amd_iommu_init_api(void)
{
int ret, err = 0;
int err;
ret = iova_cache_get();
if (ret)
return ret;
amd_iommu_init_dma_ops();
err = bus_set_iommu(&pci_bus_type, &amd_iommu_ops);
if (err)
@@ -1796,19 +1804,6 @@ int __init amd_iommu_init_api(void)
return 0;
}
int __init amd_iommu_init_dma_ops(void)
{
swiotlb = (iommu_default_passthrough() || sme_me_mask) ? 1 : 0;
if (amd_iommu_unmap_flush)
pr_info("IO/TLB flush on unmap enabled\n");
else
pr_info("Lazy IO/TLB flushing enabled\n");
iommu_set_dma_strict(amd_iommu_unmap_flush);
return 0;
}
/*****************************************************************************
*
* The following functions belong to the exported interface of AMD IOMMU
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
+53 -6
View File
@@ -435,9 +435,13 @@ bool arm_smmu_sva_supported(struct arm_smmu_device *smmu)
return true;
}
static bool arm_smmu_iopf_supported(struct arm_smmu_master *master)
bool arm_smmu_master_iopf_supported(struct arm_smmu_master *master)
{
return false;
/* We're not keeping track of SIDs in fault events */
if (master->num_streams != 1)
return false;
return master->stall_enabled;
}
bool arm_smmu_master_sva_supported(struct arm_smmu_master *master)
@@ -445,8 +449,8 @@ bool arm_smmu_master_sva_supported(struct arm_smmu_master *master)
if (!(master->smmu->features & ARM_SMMU_FEAT_SVA))
return false;
/* SSID and IOPF support are mandatory for the moment */
return master->ssid_bits && arm_smmu_iopf_supported(master);
/* SSID support is mandatory for the moment */
return master->ssid_bits;
}
bool arm_smmu_master_sva_enabled(struct arm_smmu_master *master)
@@ -459,13 +463,55 @@ bool arm_smmu_master_sva_enabled(struct arm_smmu_master *master)
return enabled;
}
static int arm_smmu_master_sva_enable_iopf(struct arm_smmu_master *master)
{
int ret;
struct device *dev = master->dev;
/*
* Drivers for devices supporting PRI or stall should enable IOPF first.
* Others have device-specific fault handlers and don't need IOPF.
*/
if (!arm_smmu_master_iopf_supported(master))
return 0;
if (!master->iopf_enabled)
return -EINVAL;
ret = iopf_queue_add_device(master->smmu->evtq.iopf, dev);
if (ret)
return ret;
ret = iommu_register_device_fault_handler(dev, iommu_queue_iopf, dev);
if (ret) {
iopf_queue_remove_device(master->smmu->evtq.iopf, dev);
return ret;
}
return 0;
}
static void arm_smmu_master_sva_disable_iopf(struct arm_smmu_master *master)
{
struct device *dev = master->dev;
if (!master->iopf_enabled)
return;
iommu_unregister_device_fault_handler(dev);
iopf_queue_remove_device(master->smmu->evtq.iopf, dev);
}
int arm_smmu_master_enable_sva(struct arm_smmu_master *master)
{
int ret;
mutex_lock(&sva_lock);
master->sva_enabled = true;
ret = arm_smmu_master_sva_enable_iopf(master);
if (!ret)
master->sva_enabled = true;
mutex_unlock(&sva_lock);
return 0;
return ret;
}
int arm_smmu_master_disable_sva(struct arm_smmu_master *master)
@@ -476,6 +522,7 @@ int arm_smmu_master_disable_sva(struct arm_smmu_master *master)
mutex_unlock(&sva_lock);
return -EBUSY;
}
arm_smmu_master_sva_disable_iopf(master);
master->sva_enabled = false;
mutex_unlock(&sva_lock);

Some files were not shown because too many files have changed in this diff Show More