Pull x86 SEV updates from Borislav Petkov:
- Remove redundant GHCB initialization guards in the SEV page state and
SVSM call paths now that the GHCB helpers handle early-boot fallback
internally
- Skip SNP initialization in the CCP driver immediately when the
preparation step fails rather than proceeding to an operation that
will certainly fail
- Abort SNP preparation and return an error when not all CPUs are
online, since the firmware enforces that every CPU enables SNP and
will fail init if not
- Simplify the VMM communication exception entry path by replacing
separate kernel and user mode macros with a single handler that
dispatches based on the current privilege level
* tag 'x86_sev_for_v7.2_rc1' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip:
x86/sev: Remove redundant ghcbs_initialized checks around __sev_{get,put}_ghcb()
crypto/ccp: Skip SNP_INIT if preparation fails
x86/sev: Do not initialize SNP if missing CPUs
x86/entry: Zap the #VC entry user and kernel macros
After
3645eb7e39 ("x86/fred: Fix early boot failures on SEV-ES/SNP guests"),
__sev_{get,put}_ghcb() handle the early-boot GHCB fallback internally, making
the ghcbs_initialized guards in __set_pages_state() and
svsm_perform_call_protocol() redundant.
Remove them.
Also initialize state->ghcb to NULL in the early-boot path of
__sev_get_ghcb() so that the ghcb_state is well-defined for all callers,
even though __sev_put_ghcb() currently returns early before reading it.
No functional change intended.
Suggested-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Nikunj A Dadhania <nikunj@amd.com>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
Link: https://patch.msgid.link/20260518102230.3394603-1-nikunj@amd.com
Modify all CPUID call sites which implicitly include any of the CPUID
headers to explicitly include them instead.
For KVM's reverse_cpuid.h, just include <asm/cpuid/types.h> since it
references the CPUID_EAX..EDX symbols without using the CPUID APIs.
Note, this allows removing the inclusion of <asm/cpuid/api.h> from within
<asm/processor.h> next. That allows the CPUID API headers to include
<asm/processor.h> without introducing a circular dependency.
Signed-off-by: Ahmed S. Darwish <darwi@linutronix.de>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Link: https://lore.kernel.org/20260327021645.555257-1-darwi@linutronix.de
Drop the separate kernel and user macros in favor of calling a single #VC
C handler which multiplexes between the kernel and user #VC entry points
by looking at CS's RPL.
Zap unused DEFINE_IDTENTRY_VC while at it.
There should be no functionality change resulting from this - just code
simplification.
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Reviewed-by: Nikunj A. Dadhania <nikunj@amd.com>
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
Acked-by: Joerg Roedel <joerg.roedel@amd.com>
Tested-by: Nikunj A. Dadhania <nikunj@amd.com>
Link: https://patch.msgid.link/20260420164352.32129-1-bp@kernel.org
Pull x86 TDX updates from Dave Hansen:
"The only real thing of note here is printing the TDX module version.
This is a little silly on its own, but the upcoming TDX module update
code needs the same TDX module call. This shrinks that set a wee bit.
There's also few minor macro cleanups and a tweak to the GetQuote ABI
to make it easier for userspace to detect zero-length (failed) quotes"
* tag 'x86_tdx_for_7.1-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
virt: tdx-guest: Return error for GetQuote failures
KVM/TDX: Rename KVM_SUPPORTED_TD_ATTRS to KVM_SUPPORTED_TDX_TD_ATTRS
x86/tdx: Rename TDX_ATTR_* to TDX_TD_ATTR_*
KVM/TDX: Remove redundant definitions of TDX_TD_ATTR_*
x86/tdx: Fix the typo in TDX_ATTR_MIGRTABLE
x86/virt/tdx: Print TDX module version during init
x86/virt/tdx: Retrieve TDX module version
FRED-enabled SEV-(ES,SNP) guests fail to boot due to the following issues
in the early boot sequence:
* FRED does not have a #VC exception handler in the dispatch logic
* Early FRED #VC exceptions attempt to use uninitialized per-CPU GHCBs
instead of boot_ghcb
Add X86_TRAP_VC case to fred_hwexc() with a new exc_vmm_communication()
function that provides the unified entry point FRED requires, dispatching
to existing user/kernel handlers based on privilege level. The function is
already declared via DECLARE_IDTENTRY_VC().
Fix early GHCB access by falling back to boot_ghcb in
__sev_{get,put}_ghcb() when per-CPU GHCBs are not yet initialized.
Fixes: 14619d912b ("x86/fred: FRED entry/exit and dispatch code")
Signed-off-by: Nikunj A Dadhania <nikunj@amd.com>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
Cc: <stable@kernel.org> # 6.12+
Link: https://patch.msgid.link/20260318075654.1792916-4-nikunj@amd.com
The TD scoped TDCS attributes are defined by bit positions. In the guest
side of the TDX code, the 'tdx_attributes' string array holds pretty
print names for these attributes, which are generated via macros and
defines. Today these pretty print names are only used to print the
attribute names to dmesg.
Unfortunately there is a typo in the define for the migratable bit.
Change the defines TDX_ATTR_MIGRTABLE* to TDX_ATTR_MIGRATABLE*. Update
the sole user, the tdx_attributes array, to use the fixed name.
Since these defines control the string printed to dmesg, the change is
user visible. But the risk of breakage is almost zero since it is not
exposed in any interface expected to be consumed programmatically.
Fixes: 564ea84c8c ("x86/tdx: Dump attributes and TD_CTLS on boot")
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Acked-by: Sean Christopherson <seanjc@google.com>
Link: https://patch.msgid.link/20260303030335.766779-2-xiaoyao.li@intel.com
The SEV-SNP IBPB-on-Entry feature does not require a guest-side
implementation. It was added in Zen5 h/w, after the first SNP Zen
implementation, and thus was not accounted for when the initial set of SNP
features were added to the kernel.
In its abundant precaution, commit
8c29f01654 ("x86/sev: Add SEV-SNP guest feature negotiation support")
included SEV_STATUS' IBPB-on-Entry bit as a reserved bit, thereby masking
guests from using the feature.
Allow guests to make use of IBPB-on-Entry when supported by the hypervisor, as
the bit is now architecturally defined and safe to expose.
Fixes: 8c29f01654 ("x86/sev: Add SEV-SNP guest feature negotiation support")
Signed-off-by: Kim Phillips <kim.phillips@amd.com>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Reviewed-by: Nikunj A Dadhania <nikunj@amd.com>
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
Cc: stable@kernel.org
Link: https://patch.msgid.link/20260203222405.4065706-2-kim.phillips@amd.com
This was done entirely with mindless brute force, using
git grep -l '\<k[vmz]*alloc_objs*(.*, GFP_KERNEL)' |
xargs sed -i 's/\(alloc_objs*(.*\), GFP_KERNEL)/\1)/'
to convert the new alloc_obj() users that had a simple GFP_KERNEL
argument to just drop that argument.
Note that due to the extreme simplicity of the scripting, any slightly
more complex cases spread over multiple lines would not be triggered:
they definitely exist, but this covers the vast bulk of the cases, and
the resulting diff is also then easier to check automatically.
For the same reason the 'flex' versions will be done as a separate
conversion.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
This is the result of running the Coccinelle script from
scripts/coccinelle/api/kmalloc_objs.cocci. The script is designed to
avoid scalar types (which need careful case-by-case checking), and
instead replace kmalloc-family calls that allocate struct or union
object instances:
Single allocations: kmalloc(sizeof(TYPE), ...)
are replaced with: kmalloc_obj(TYPE, ...)
Array allocations: kmalloc_array(COUNT, sizeof(TYPE), ...)
are replaced with: kmalloc_objs(TYPE, COUNT, ...)
Flex array allocations: kmalloc(struct_size(PTR, FAM, COUNT), ...)
are replaced with: kmalloc_flex(*PTR, FAM, COUNT, ...)
(where TYPE may also be *VAR)
The resulting allocations no longer return "void *", instead returning
"TYPE *".
Signed-off-by: Kees Cook <kees@kernel.org>
Pull x86 SEV updates from Borislav Petkov:
- Make the SEV internal header really internal and carve out the
SVSM-specific code into a separate compilation unit, along with other
cleanups and fixups
[ TLA translation service: 'SEV' is AMD's 'Secure Encrypted
Virtualization' and SVSM is an ETLA ('Enhanced TLA') for 'Secure
VM Service Module'.
Some of us have trouble keeping track of this all and need all the
help we can get ]
* tag 'x86_sev_for_v7.0_rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
x86/sev: Don't emit BSS_DECRYPTED section unless it is in use
x86/sev: Use kfree_sensitive() when freeing a SNP message descriptor
x86/sev: Rename sev_es_ghcb_handle_msr() to __vc_handle_msr()
x86/sev: Carve out the SVSM code into a separate compilation unit
x86/sev: Add internal header guards
x86/sev: Move the internal header
With Debian clang version 19.1.7 (3+build5) there are calls to
kasan_check_write() from __sev_es_nmi_complete(), which violates noinstr. Fix
it by disabling GCOV for the noinstr object, as has been done for previous
such instrumentation issues.
Note that this file already disables __SANITIZE_ADDRESS__ and
__SANITIZE_THREAD__, thus calls like kasan_check_write() ought to be nops
regardless of GCOV. This has been fixed in other patches. However, to avoid
any other accidental instrumentation showing up, (and since, in principle GCOV
is instrumentation and hence should be disabled for noinstr code anyway),
disable GCOV overall as well.
Signed-off-by: Brendan Jackman <jackmanb@google.com>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Acked-by: Marco Elver <elver@google.com>
Link: https://patch.msgid.link/20251216-gcov-inline-noinstr-v3-3-10244d154451@google.com
Move the internal header out of the usual include/asm/ include path
because having an "internal" header there doesn't really make it
internal - quite the opposite - that's the normal arch include path.
So move where it belongs and make it really internal.
No functional changes.
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Link: https://lore.kernel.org/r/20251204145716.GDaTGhTEHNOtSdTkEe@fat_crate.local
When a guest issues a CPUID instruction for Fn0000000D_x01, the hypervisor may
be intercepting the CPUID instruction and need to access the guest XSS value.
For SEV-ES, the XSS value is encrypted and needs to be included in the GHCB to
be visible to the hypervisor.
Signed-off-by: John Allen <john.allen@amd.com>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
Link: https://patch.msgid.link/all/20250924200852.4452-3-john.allen@amd.com/
This function is going away so replace the callsites with the equivalent
functionality. Add a new SAVIC-specific termination reason. If more
granularity is needed there, it will be revisited in the future.
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
In order to be able to have tight control over which code may execute
from the early 1:1 mapping of memory, but still link vmlinux as a single
executable, prefix all symbol references in startup code with __pi_, and
invoke it from outside using the __pi_ prefix.
Use objtool to check that no absolute symbol references are present in
the startup code, as these cannot be used from code running from the 1:1
mapping.
Note that this also requires disabling the latent-entropy GCC plugin, as
the global symbol references that it injects would require explicit
exports, and given that the startup code rarely executes more than once,
it is not a useful source of entropy anyway.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Link: https://lore.kernel.org/20250828102202.1849035-43-ardb+git@google.com
Rename sev-nmi.c to noinstr.c, and move the get/put GHCB routines into it too,
which are also annotated as 'noinstr' and suffer from the same problem as the
NMI code, i.e., that GCC may ignore the __no_sanitize_address__ function
attribute implied by 'noinstr' and insert KASAN instrumentation anyway.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Link: https://lore.kernel.org/20250828102202.1849035-37-ardb+git@google.com
Provide PIC aliases for data objects that are shared between the SEV startup
code and the SEV code that executes later. This is needed so that the confined
startup code is permitted to access them.
This requires some of these variables to be moved into a source file that is
not part of the startup code, as the PIC alias is already implied, and
exporting variables in the opposite direction is not supported.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Link: https://lore.kernel.org/20250828102202.1849035-36-ardb+git@google.com
Linus Torvalds
Nikunj A Dadhania
Ahmed S. Darwish
Borislav Petkov (AMD)
Xiaoyao Li
Kim Phillips
Kees Cook
Brendan Jackman
John Allen
Ingo Molnar
Ard Biesheuvel