This feature is controlled by an MSR; MSRs are per-CPU.
The Intel SDM doesn't document CPUID faulting, at least as of the Dec 2024
revision; despite the deleted comment in ring0/kernel_amd64.go, there is no
Vol. 3 Table 2-43, and every table in Vol. 4 ("Model-Specific Registers") lists
bit 31 in MSR_PLATFORM_INFO as "reserved". The only documentation seems to be
that cited by Linux's e9ea1e7f53b85 ("x86/arch_prctl: Add
ARCH_[GET|SET]_CPUID"): "Intel Virtualization Technology FlexMigration
Application Note" 323850-004, 2012. This document positions CPUID faulting as
an alternative way to support cross-CPU migration for VMs that don't use VMX;
consequently it does not clarify if CPUID faulting is effective in guest ("VMX
non-root") mode, or if the CPUID VM exit takes precedence. If the former is the
case then CPUID faulting is probably faster than setting app CPUID with
KVM_SET_CPUID2, and vice versa. But regardless, this is much simpler.
PiperOrigin-RevId: 733113944
Top-Byte-Ignore (TBI) is a feature on all ARMv8.0 CPUs that causes the top byte
of virtual addresses to be ignored on loads and stores. Instead, bit 55 is
extended over bits 56-63 before address translation. This feature allows use of
the (ignored) top byte as a tag or for other in-band metadata.
In Linux, brk()/mmap()/mremap() syscalls don't untag addresses. More details
are in dcde237319e6 ("mm: Avoid creating virtual address aliases in
brk()/mmap()/mremap()")
PiperOrigin-RevId: 715885990
Now we don't need to trigger a second fault to figure out whether it was write
or read access.
Fixes#11008
Co-developed-by: Jamie Liu <jamieliu@google.com>
PiperOrigin-RevId: 697677262
On x86_64, we prefer not to map the entire sentry address space into
the VM due to memory overhead. It is about 3MB for a 40-bit address
space and about 250MB for 46-bit address spaces (modern CPUs).
If the entire address space isn't mapped into the VM, we need to
trap mmap system calls and map sentry memory regions on demand. This
introduces some overhead for mmap system calls, but considering that
mmap isn't called frequently, it seems better than the memory and
startup time overhead introduced by mapping the entire address
space.
* native mmap:
BenchmarkHostMMap-12 266972 4297 ns/op
* mmpa with the seccomp trap:
BenchmarkHostMMap-12 174373 6855 ns/op
PiperOrigin-RevId: 687418707
The user-space bluepill introduced the issue. With that change, we use a
per-vcpu stack to run bluepillHandler. BluepillHanlder releases a vcpu before
calling sigreturn, so it is still running on the vcpu stack. The race looks
like this: one thread releases a vcpu, another thread takes it and starts
using its stack, the first thread calls sigreturn with a corrupted signal
frame.

PiperOrigin-RevId: 684992950
futex wait needs to be called surrounded by syscall.Entersyscall() and
syscall.Exitsyscall(), which RawSyscall does not do. It was a mistake
to substitute it.
PiperOrigin-RevId: 681629844
These are not as important as the instances that have already been replaced,
because everything builds without these replacements. However, for consistency
and for the sake of a couple of less jumps, let's replace these too.
PiperOrigin-RevId: 681135990
These are already used in platform/kvm and pkg/sigframe. Maybe there will be
more in the future that are outside KVM, we don't want to keep redefining
these.
PiperOrigin-RevId: 680737814
Debug build functions use more stack space than normal, such that the
KVM-nosplit function call chain doesn't fit. This patch replaces calls into
unix.RawSyscall* functions with variants that do not grow the stack, and inlines
some functions in ring0/pagetables in order to reduce stack usage. Additionally
seccompMmapHandler is not used during debug builds anymore for making it fit
into the nosplit stack size requirements.
PiperOrigin-RevId: 679774881
RFLAGS cannot be used because the IF flag (interrupt flag) is cleared when
a goroutine suspended on the host side is resumed in the VM. This means it
can start switching into the vm when it is already in the VM. It only works
because the rt_sigprocmask syscall is triggered before constructing a signal
frame, returning execution back to the host side, and then it switches into
the VM again.
Fixes: 374a11a7cd ("platform/kvm: rewriting bluepill()...")
PiperOrigin-RevId: 673621750
Implement bluepill() without relying on a signal handler trampoline by
directly switching stacks and execution contexts. Here, it is implemented
just for x86_64. The arm64 part will be implemented separately.
Here are benchmark results before and after this change: Before:
BenchmarkWorldSwitchToUserRoundtrip-8 255940 4407 ns/op
After:
BenchmarkWorldSwitchToUserRoundtrip-8 307773 3765 ns/op
Suggested-by: Jamie Liu <jamieliu@google.com>
PiperOrigin-RevId: 664989586
This turns the uint64 metric constructor arguments into a struct, making it
more explicit as to what each part means. It also allows the creation of
non-cumulative uint64 (gauge) metrics, and adds methods to decrement or set
them.
PiperOrigin-RevId: 647134245
