From 86059124df31971e07da40395a2c4225f004dc40 Mon Sep 17 00:00:00 2001 From: Etienne Perot Date: Fri, 5 Aug 2022 12:38:08 -0700 Subject: [PATCH] `gvisor.dev`: Update documentation around platforms. - Explain the consequences of using KVM in a nested virtualization environment. - Mention GKE Sandbox proprietary platform. - Fix command to check if the user is part of the `kvm` group. - Small fixes here and there. PiperOrigin-RevId: 465625190 --- g3doc/architecture_guide/platforms.md | 21 ++++++++++++++++----- g3doc/user_guide/platforms.md | 15 +++++++++------ 2 files changed, 25 insertions(+), 11 deletions(-) diff --git a/g3doc/architecture_guide/platforms.md b/g3doc/architecture_guide/platforms.md index e19c77236..68518e6ba 100644 --- a/g3doc/architecture_guide/platforms.md +++ b/g3doc/architecture_guide/platforms.md @@ -4,7 +4,7 @@ gVisor requires a platform to implement interception of syscalls, basic context switching, and memory mapping functionality. Internally, gVisor uses an -abstraction sensibly called [Platform][platform]. A simplified version of this +abstraction sensibly called [`Platform`][platform]. A simplified version of this interface looks like: ```golang @@ -29,14 +29,15 @@ various trade-offs, generally around performance and hardware requirements. ## Implementations The choice of platform depends on the context in which `runsc` is executing. In -general, virtualized platforms may be limited to platforms that do not require -hardware virtualized support (since the hardware is already in use): +general, when running on bare-metal (not inside a VM), the KVM platform will +provide the best performance. The `ptrace` platform is a better choice when +running inside a VM, or on a machine without virtualization support. ![Platforms](platforms.png "Platform examples.") ### ptrace -The ptrace platform uses [PTRACE_SYSEMU][ptrace] to execute user code without +The ptrace platform uses [`PTRACE_SYSEMU`][ptrace] to execute user code without allowing it to execute host system calls. This platform can run anywhere that `ptrace` works (even VMs without nested virtualization), which is ubiquitous. @@ -46,12 +47,21 @@ call-heavy applications may pay a [performance penalty](./performance.md). ### KVM The KVM platform uses the kernel's [KVM][kvm] functionality to allow the Sentry -to act as both guest OS and VMM. The KVM platform can run on bare-metal or in a +to act as both guest OS and VMM. The KVM platform can run on bare-metal, or in a VM with nested virtualization enabled. While there is no virtualized hardware layer -- the sandbox retains a process model -- gVisor leverages virtualization extensions available on modern processors in order to improve isolation and performance of address space switches. +Note that while running within a nested VM is feasible with the KVM platform, +the `ptrace` platform will often provide better performance in such a setup, due +to the overhead of nested virtualization. + +### GKE Sandbox + +[GKE Sandbox] uses a custom gVisor platform implementation which provides better +performance than `ptrace` and KVM. + ## Changing Platforms See [Changing Platforms](../user_guide/platforms.md). @@ -59,3 +69,4 @@ See [Changing Platforms](../user_guide/platforms.md). [kvm]: https://www.kernel.org/doc/Documentation/virtual/kvm/api.txt [platform]: https://cs.opensource.google/gvisor/gvisor/+/release-20190304.1:pkg/sentry/platform/platform.go;l=33 [ptrace]: http://man7.org/linux/man-pages/man2/ptrace.2.html +[GKE Sandbox]: https://cloud.google.com/kubernetes-engine/docs/concepts/sandbox-pods diff --git a/g3doc/user_guide/platforms.md b/g3doc/user_guide/platforms.md index 5186e2c9a..acbcd5547 100644 --- a/g3doc/user_guide/platforms.md +++ b/g3doc/user_guide/platforms.md @@ -11,21 +11,24 @@ If you intend to run the KVM platform, you will also to have KVM installed on your system. If you are running a Debian based system like Debian or Ubuntu you can usually do this by ensuring the module is loaded, and your user has permissions to access the `/dev/kvm` device. Usually, it means that the user is -in the kvm group. +in the `kvm` group. -```bash +```shell # Check that /dev/kvm is owned by the kvm group $ ls -l /dev/kvm crw-rw----+ 1 root kvm 10, 232 Jul 26 00:04 /dev/kvm # Make sure that the current user is part of the kvm group -$ $ groups | grep -q kvm | echo ok +$ groups | grep -qw kvm && echo ok ok ``` -If you are using a virtual machine you will need to make sure that nested -virtualization is configured. Here are links to documents on how to set up -nested virtualization in several popular environments: +**For best performance, use the KVM platform on bare-metal machines only**. If +you have to run gVisor within a virtual machine, the `ptrace` platform will +often yield better performance than KVM. If you still want to use KVM within a +virtual machine, you will need to make sure that nested virtualization is +configured. Here are links to documents on how to set up nested virtualization +in several popular environments: * Google Cloud: [Enabling Nested Virtualization for VM Instances][nested-gcp] * Microsoft Azure: