This feature is controlled by an MSR; MSRs are per-CPU.
The Intel SDM doesn't document CPUID faulting, at least as of the Dec 2024
revision; despite the deleted comment in ring0/kernel_amd64.go, there is no
Vol. 3 Table 2-43, and every table in Vol. 4 ("Model-Specific Registers") lists
bit 31 in MSR_PLATFORM_INFO as "reserved". The only documentation seems to be
that cited by Linux's e9ea1e7f53b85 ("x86/arch_prctl: Add
ARCH_[GET|SET]_CPUID"): "Intel Virtualization Technology FlexMigration
Application Note" 323850-004, 2012. This document positions CPUID faulting as
an alternative way to support cross-CPU migration for VMs that don't use VMX;
consequently it does not clarify if CPUID faulting is effective in guest ("VMX
non-root") mode, or if the CPUID VM exit takes precedence. If the former is the
case then CPUID faulting is probably faster than setting app CPUID with
KVM_SET_CPUID2, and vice versa. But regardless, this is much simpler.
PiperOrigin-RevId: 733113944
Currently ring0.PhysicalAddressBits will be set in ring0.Init() even
if it has already been initialized elsewhere. For example, when
compiling runsc with cgo enabled, function init() of the package
kvm(machine_cgo.go) sets this variable. And this value would be updated
by ring0.Init() when creating a new KVM context. If two values are
inconsistent, the sandbox would panic during later memory region
setting.
This patch fixes it by checking the PhysicalAddressBits in ring0.Init()
and setting the value only if it is uninitialized.
Signed-off-by: Tianyu Zhou <albert.zty@antgroup.com>
Debug build functions use more stack space than normal, such that the
KVM-nosplit function call chain doesn't fit. This patch replaces calls into
unix.RawSyscall* functions with variants that do not grow the stack, and inlines
some functions in ring0/pagetables in order to reduce stack usage. Additionally
seccompMmapHandler is not used during debug builds anymore for making it fit
into the nosplit stack size requirements.
PiperOrigin-RevId: 679774881
```
CR4.PGE (bit-7) enables Paging Global Extensions (PGE). PGE determines whether
moves to CR3 flush all of the PTE's from the TLB, or only those whose G-bit
(global bit) is not set. Likewise, for task switches which implicitly set CR3,
CR4.PGE controls TLB flushing in the same manner.
```
In case of gvisor, it improves performance of switching to/from the Sentry:
│ kvm │ kvm.pge │
│ wall_ns/op │ wall_ns/op vs base │
Getpid 1.774k ± 0% 1.607k ± 1% -9.36% (p=0.000 n=10)
GetpidOpt 1.623k ± 1% 1.452k ± 1% -10.54% (p=0.000 n=10)
geomean 1.696k 1.528k -9.95%
│ kvm │ kvm.pge │
│ cpu_ns/op │ cpu_ns/op vs base │
Getpid 1.749k ± 0% 1.583k ± 1% -9.52% (p=0.000 n=10)
GetpidOpt 1.600k ± 0% 1.414k ± 2% -11.62% (p=0.000 n=10)
geomean 1.673k 1.496k -10.58%
It plays only on hosts without PCID.
PiperOrigin-RevId: 671587163
SETREGSET/GETREGSET expect AMX portions of fpstate to always be used.
For this reason we need to allocate enough memory for this to happen,
even if we never populate the AMX portions within initX86FPState.
PiperOrigin-RevId: 599702181
For now we are going to completely disable using AMX, so we will
always subtract extended state size reserved from AMX from the rest
of the extended state size, and hardcode the AMX XCR0 bits to be
always off.
Fixes#9750.
PiperOrigin-RevId: 599302059
This caches the GS value in the ring0.CPU struct, in order to avoid a
full pipeline flush in the switch path.
Fixes#2010
Co-authored-by: Adin Scannell <adin@scannell.ca>
Co-authored-by: Lai Jiangshan <jiangshan.ljs@antfin.com>
PiperOrigin-RevId: 578919437
It is an idea of running codespell as part of our presubmit checks.
Before enabling it for new changes, let's fix what it has found.
Signed-off-by: Andrei Vagin <avagin@gmail.com>
This moves the initialization of `cpuid.hostFeatureSet` out of package-level
`init` and instead moves it to an explicit `cpuid.Initialize()` function.
On AMD64, this saves about 512KiB of heap memory that would otherwise always
be live.
PiperOrigin-RevId: 514896323
For all trivial and zero frame-sized functions, we now require an explicit
NOFRAME annotation as the heuristic has changed. See go.dev/cl/466316.
Most of these functions do not *require* NOFRAME, but this change encodes
the existing behavior in order to avoid accidental bugs or regressions.
PiperOrigin-RevId: 513946210
This allows fact information to be validated in the underlying source files,
but requires us to explicitly maintain this in appropriate version-tagged, and
architecture-tagged files. This is more explicit and safer.
This mechanism uses a special regular expression for matching a +checkconst
stanza to validate constant values, sizes and offsets. This applies to both Go
source files and assembly files.
PiperOrigin-RevId: 511867507
Unexported constants can end up unaddressable in objectpath and thus their
object facts get lost on fact import. Work around this by adding a
package-level fact which is a map of all constants to their value.
Remove the Value facts because it is confusing to have two ways to access the
same thing (especially when one only works sometimes).
PiperOrigin-RevId: 505153453
It is similar with 36a17a814b ("Go 1.17 support for the KVM platform") that
fixed the same problem for x86. Go 1.18 adds the support of the register-based
calling convention on aarch64.
Signed-off-by: Andrei Vagin <avagin@gmail.com>
There must be no space in "//go:nosplit". This probably never
caused problems because the function is so simple it is almost
certainly inlined.
PiperOrigin-RevId: 464823961
Jamie LiuandgVisor bot
Tianyu Zhou
Andrei VaginandgVisor bot
Konstantin BogomolovandgVisor bot
Jing ChenandgVisor bot
Andrei Vagin
Etienne PerotandgVisor bot
Adin ScannellandgVisor bot
Michael PrattandgVisor bot
Kevin KrakauerandgVisor bot