From b3b36167452c2db1268b4b655817379308df26f6 Mon Sep 17 00:00:00 2001 From: Zach Koopmans Date: Tue, 5 Mar 2024 14:08:28 -0800 Subject: [PATCH] Add security documentation to nvproxy page in gvisor.dev PiperOrigin-RevId: 612963488 --- g3doc/user_guide/gpu.md | 106 +++++++++++++++++++++++++++++++++++++++- 1 file changed, 104 insertions(+), 2 deletions(-) diff --git a/g3doc/user_guide/gpu.md b/g3doc/user_guide/gpu.md index ab4e7be0a..07d4a9e7c 100644 --- a/g3doc/user_guide/gpu.md +++ b/g3doc/user_guide/gpu.md @@ -16,8 +16,6 @@ NVIDIA's driver on the host. It provides access to NVIDIA GPU-specific devices to the sandboxed application. The CUDA application can run unmodified inside the sandbox and interact transparently with these devices. - - ## Environments The `runsc` flag `--nvproxy` must be specified to enable GPU support. gVisor @@ -154,3 +152,107 @@ commands might still be unimplemented. Please to describe about your use case. If a missing `ioctl` implementation is the problem, then the [debug logs](/docs/user_guide/debugging/) will contain warnings with prefix `nvproxy: unknown *`. + +## Security + +While CUDA support enables important use cases for gVisor, it is important for +users to understand the security model around the use of GPUs in sandboxes. In +short, while gVisor will protect the host from the sandboxed application, +**NVIDIA driver updates must be part of any security plan with or without +gVisor**. + +First, a short discussion on +[gvisor's security model](../architecture_guide/security.md). gVisor protects +the host from sandboxed applications by providing several layers of defense. The +layers most relevant to this discussion are the redirection of application +syscalls to the gVisor sandbox and use of +[seccomp-bpf](https://www.kernel.org/doc/html/v4.19/userspace-api/seccomp_filter.html) +on gVisor sandboxes. + +gVisor uses a "platform" to tell the host kernel to reroute system calls to the +sandbox process, known as the sentry. The sentry implements a syscall table, +which services all application syscalls. The Sentry *may* make syscalls to the +host kernel if it needs them to fulfill the application syscall, but it doesn't +merely pass an application syscall to the host kernel. + +On sandbox boot, seccomp filters are applied to the sandbox. Seccomp filters +applied to the sandbox constrain the set of syscalls that it can make to the +host kernel, blocking access to most host kernel vulnerabilities even if the +sandbox becomes compromised. + +For example, [CVE-2022-0185](https://nvd.nist.gov/vuln/detail/CVE-2022-0185) is +mitigated because gVisor itself handles the syscalls required to use namespaces +and capabilities, so the application is using gVisor's implementation, not the +host kernel's. For a compromised sandbox, the syscalls required to exploit the +vulnerability are blocked by seccomp filters. + +In addition, seccomp-bpf filters can filter by argument names allowing us to +allowlist granularly by `ioctl(2)` arguments. `ioctl(2)` is a source of many +bugs in any kernel due to the complexity of its implementation. As of writing, +gVisor does +[allowlist some `ioctl`s](https://github.com/google/gvisor/blob/ccc3c2cbd26d3514885bd665b0a110150a6e8c53/runsc/boot/filter/config/config_main.go#L111) +by argument for things like terminal support. + +For example, [CVE-2024-21626](https://nvd.nist.gov/vuln/detail/CVE-2024-21626) +is mitigated by gVisor because the application would use gVisor's implementation +of `ioctl(2)`. For a compromised sentry, `ioctl(2)` calls with the needed +arguments are not in the seccomp filter allowlist, blocking the attacker from +making the call. gVisor also mitigates similar vulnerabilities that come with +device drivers +([CVE-2023-33107](https://nvd.nist.gov/vuln/detail/CVE-2023-33107)). + +### nvproxy Security + +Recall that "nvproxy" allows applications to directly interact with supported +ioctls defined in the NVIDIA driver. + +gVisor's seccomp filter rules are modified such that `ioctl(2)` calls can be +made +[*only for supported ioctls*](https://github.com/google/gvisor/blob/be9169a6ce095a08b99940a97db3f58e5c5bd2ce/pkg/sentry/devices/nvproxy/seccomp_filters.go#L1). +The allowlisted rules aligned with each +[driver version](https://github.com/google/gvisor/blob/c087777e37a186e38206209c41178e92ef1bbe81/pkg/sentry/devices/nvproxy/version.go#L152). +This approach is similar to the allowlisted ioctls for terminal support +described above. This allows gVisor to retain the vast majority of its +protection for the host while allowing access to GPUs. All of the above CVEs +remain mitigated even when "nvproxy" is used. + +However, gVisor is much less effective at mitigating vulnerabilities within the +NVIDIA GPU drivers themselves, *because* gVisor passes through calls to be +handled by the kernel module. If there is a vulnerability in a given driver for +a given GPU `ioctl` (read feature) that gVisor passes through, then gVisor will +also be vulnerable. If the vulnerability is in an unimplemented feature, gVisor +will block the required calls with seccomp filters. + +In addition, gVisor doesn't introduce any additional hardware-level isolation +beyond that which is configured by by the NVIDIA kernel-mode driver. There is no +validation of things like DMA buffers. The only checks are done in seccomp-bpf +rules to ensure `ioctl(2)` calls are made on supported and allowlisted `ioctl`s. + +Therefore, **it is imperative that users update NVIDIA drivers in a timely +manner with or without gVisor**. To see the latest drivers gVisor supports, you +can run the following with your runsc release: + +``` +$ runsc nvproxy list-supported-drivers +``` + +Alternatively you can view the +[source code](https://github.com/google/gvisor/blob/be9169a6ce095a08b99940a97db3f58e5c5bd2ce/pkg/sentry/devices/nvproxy/version.go#L1) +or download it and run: + +``` +$ make run TARGETS=runsc:runsc ARGS="nvproxy list-supported-drivers" +``` + +### So, if you don't protect against all the things, why even? + +While gVisor doesn't protect against *all* NVIDIA driver vulnerabilities, it +*does* protect against a large set of general vulnerabilities in Linux. +Applications don't just use GPUs, they use them as a part of a larger +application that may include third party libraries. For example, Tensorflow +[suffers from the same kind of vulnerabilities](https://nvd.nist.gov/vuln/detail/CVE-2022-29216) +that every application does. Designing and implementing an application with +security in mind is hard and in the emerging AI space, security is often +overlooked in favor of getting to market fast. There are also many services that +allow users to run external users' code on the vendor's infrastructure. gVisor +is well suited as part of a larger security plan for these and other use cases.