Merge pull request #2513 from amscanne:website-integrated

PiperOrigin-RevId: 311184385

BUILD

@@ -1,9 +1,48 @@
 load("//tools:defs.bzl", "build_test", "gazelle", "go_path")
+load("//website:defs.bzl", "doc")
 
 package(licenses = ["notice"])
 
 exports_files(["LICENSE"])
 
+doc(
+    name = "contributing",
+    src = "CONTRIBUTING.md",
+    category = "Project",
+    permalink = "/contributing/",
+    visibility = ["//website:__pkg__"],
+    weight = "20",
+)
+
+doc(
+    name = "security",
+    src = "SECURITY.md",
+    category = "Project",
+    permalink = "/security/",
+    visibility = ["//website:__pkg__"],
+    weight = "30",
+)
+
+doc(
+    name = "governance",
+    src = "GOVERNANCE.md",
+    category = "Project",
+    permalink = "/community/governance/",
+    subcategory = "Community",
+    visibility = ["//website:__pkg__"],
+    weight = "91",
+)
+
+doc(
+    name = "code_of_conduct",
+    src = "CODE_OF_CONDUCT.md",
+    category = "Project",
+    permalink = "/community/code_of_conduct/",
+    subcategory = "Community",
+    visibility = ["//website:__pkg__"],
+    weight = "99",
+)
+
 # The sandbox filegroup is used for sandbox-internal dependencies.
 package_group(
     name = "sandbox",

CODE_OF_CONDUCT.md

@@ -87,6 +87,5 @@ harassment or threats to anyone's safety, we may take action without notice.
 
 ## Attribution
 
-This Code of Conduct is adapted from the Contributor Covenant, version 1.4,
-available at
-https://www.contributor-covenant.org/version/1/4/code-of-conduct.html
+This Code of Conduct is adapted from the
+[Contributor Covenant, version 1.4](https://www.contributor-covenant.org/version/1/4/code-of-conduct.html).

CONTRIBUTING.md

@@ -39,8 +39,8 @@ Dependencies can be added by using `go mod get`. In order to keep the
 
 All Go code should conform to the [Go style guidelines][gostyle]. C++ code
 should conform to the [Google C++ Style Guide][cppstyle] and the guidelines
-described for [tests][teststyle]. Note that code may be automatically formatted
-per the guidelines when merged.
+described for tests. Note that code may be automatically formatted per the
+guidelines when merged.
 
 As a secure runtime, we need to maintain the safety of all of code included in
 gVisor. The following rules help mitigate issues.
@@ -131,4 +131,3 @@ one above, the
 [github]: https://github.com/google/gvisor/compare
 [gvisor-dev-list]: https://groups.google.com/forum/#!forum/gvisor-dev
 [gostyle]: https://github.com/golang/go/wiki/CodeReviewComments
-[teststyle]: ./test/

GOVERNANCE.md

@@ -0,0 +1,113 @@
+# Governance
+
+## Projects
+
+A *project* is the primary unit of collaboration. Each project may have its own
+repository and contribution process.
+
+All projects are covered by the [Code of Conduct](CODE_OF_CONDUCT.md), and
+should include an up-to-date copy in the project repository or a link here.
+
+## Contributors
+
+Anyone can be a *contributor* to a project, provided they have signed relevant
+Contributor License Agreements (CLAs) and follow the project's contribution
+guidelines. Contributions will be reviewed by a maintainer, and must pass all
+applicable tests.
+
+Reviews check for code quality and style, including documentation, and enforce
+other policies. Contributions may be rejected for reasons unrelated to the code
+in question. For example, a change may be too complex to maintain or duplicate
+existing functionality.
+
+Note that contributions are not limited to code alone. Bugs, documentation,
+experience reports or public advocacy are all valuable ways to contribute to a
+project and build trust in the community.
+
+## Maintainers
+
+Each project has one or more *maintainers*. Maintainers set technical direction,
+facilitate contributions and exercise overall stewardship.
+
+Maintainers have write access to the project repository. Maintainers review and
+approve changes. They can also assign issues and add additional reviewers.
+
+Note that some repositories may not allow direct commit access, which is
+reserved for administrators or automated processes. In this case, maintainers
+have approval rights, and a separate process exists for merging a change.
+
+Maintainers are responsible for upholding the code of conduct in interactions
+via project communication channels. If comments or exchanges are in violation,
+they may remove them at their discretion.
+
+### Repositories requiring synchronization
+
+For some projects initiated by Google, the infrastructure which synchronizes and
+merges internal and external changes requires that merges are performed by a
+Google employee. In such cases, Google will initiate a rotation to merge changes
+once they pass tests and are approved by a maintainer. This does not preclude
+non-Google contributors from becoming maintainers, in which case the maintainer
+holds approval rights and the merge is an automated process. In some cases,
+Google-internal tests may fail and have to be fixed: the Google employee will
+work with the submitter to achieve this.
+
+### Becoming a maintainer
+
+The list of maintainers is defined by the list of people with commit access or
+approval authority on a repository, typically via a Gerrit group or a GitHub
+team.
+
+Existing maintainers may elevate a contributor to maintainer status on evidence
+of previous contributions and established trust. This decision is based on lazy
+consensus from existing maintainers. While contributors may ask maintainers to
+make this decision, existing maintainers will also pro-actively identify
+contributors who have demonstrated a sustained track record of technical
+leadership and direct contributions.
+
+## Special Interest Groups (SIGs)
+
+From time-to-time, a SIG may be formed in order to solve larger, more complex
+problems across one or more projects. There are many avenues for collaboration
+outside a SIG, but a SIG can provide structure for collaboration on a single
+topic.
+
+Each group will be established by a charter, and governed by the Code of
+Conduct. Some resources may be provided to the group, such as mailing lists or
+meeting space, and archives will be public.
+
+## Security disclosure
+
+Projects may maintain security mailing lists for vulnerability reports and
+internal project audits may occasionally reveal security issues. Access to these
+lists and audits will be limited to project *maintainers*; individual
+maintainers should opt to participate in these lists based on need and
+expertise. Once maintainers become aware of a potential security issue, they
+will assess the scope and potential impact. If reported externally, maintainers
+will determine a reasonable embargo period with the reporter.
+
+During the embargo period, the maintainers will prioritize a fix for the
+security issue. They may choose to disclose the issue to additional trusted
+contributors in order to facilitate a fix, subjecting them to the embargo, or
+notify affected users in order to give them an advanced opportunity to mitigate
+the issue. The inclusion of specific users in this disclosure is left to the
+discretion of the maintainers and contributors involved, and depends on the
+scale of known project use and exposure.
+
+Once a fix is widely available or the embargo period ends, the maintainers will
+make technical details about the vulnerability and associated fixes available.
+
+## Mailing lists
+
+There are four key mailing lists that span projects.
+
+*   [gvisor-users](mailto:gvisor-users@googlegroups.com): general purpose user
+    list.
+*   [gvisor-dev](mailto:gvisor-dev@googlegroups.com): general purpose
+    development list.
+*   [gvisor-security](mailto:gvisor-security@googlegroups.com): private security
+    list. Access to this list is restricted to maintainers of the core gVisor
+    project, subject to the security disclosure policy described above.
+*   [gvisor-syzkaller](mailto:gvisor-syzkaller@googlegroups.com): private
+    syzkaller bug tracking list. Access to this list is not limited to
+    maintainers, but will be granted to those who can credibly contribute to
+    fixes.

Makefile

@@ -108,7 +108,7 @@ runsc: ## Builds the runsc binary.
 .PHONY: runsc
 
 smoke-test: ## Runs a simple smoke test after build runsc.
-	@$(MAKE) run DOCKER_RUN_OPTIONS="" ARGS="--alsologtostderr --network none --debug --TESTONLY-unsafe-nonroot=true --rootless do true"
+	@$(MAKE) run DOCKER_PRIVILEGED="" ARGS="--alsologtostderr --network none --debug --TESTONLY-unsafe-nonroot=true --rootless do true"
 .PHONY: smoke-tests
 
 unit-tests: ## Runs all unit tests in pkg runsc and tools.
@@ -119,6 +119,38 @@ tests: ## Runs all local ptrace system call tests.
 	@$(MAKE) test OPTIONS="--test_tag_filter runsc_ptrace test/syscalls/..."
 .PHONY: tests
 
+##
+## Website & documentation helpers.
+##
+##   The website is built from repository documentation and wrappers, using
+##   using a locally-defined Docker image (see images/jekyll). The following
+##   variables may be set when using website-push:
+##     WEBSITE_IMAGE   - The name of the container image.
+##     WEBSITE_SERVICE - The backend service.
+##     WEBSITE_PROJECT - The project id to use.
+##     WEBSITE_REGION  - The region to deploy to.
+##
+WEBSITE_IMAGE   := gcr.io/gvisordev/gvisordev
+WEBSITE_SERVICE := gvisordev
+WEBSITE_PROJECT := gvisordev
+WEBSITE_REGION  := us-central1
+
+website-build: load-jekyll ## Build the site image locally.
+	@$(MAKE) run TARGETS="//website:website"
+.PHONY: website-build
+
+website-server: website-build ## Run a local server for development.
+	@docker run -i -p 8080:8080 gvisor.dev/images/website
+.PHONY: website-server
+
+website-push: website-build ## Push a new image and update the service.
+	@docker tag gvisor.dev/images/website $(WEBSITE_IMAGE) && docker push $(WEBSITE_IMAGE)
+.PHONY: website-push
+
+website-deploy: website-push ## Deploy a new version of the website.
+	@gcloud run deploy $(WEBSITE_SERVICE) --platform=managed --region=$(WEBSITE_REGION) --project=$(WEBSITE_PROJECT) --image=$(WEBSITE_IMAGE)
+.PHONY: website-push
+
 ##
 ## Development helpers and tooling.
 ##

SECURITY.md

@@ -5,7 +5,7 @@ the [gvisor-security mailing list][gvisor-security-list]. You should receive a
 prompt response, typically within 48 hours.
 
 Policies for security list access, vulnerability embargo, and vulnerability
-disclosure are outlined in the [community][community] repository.
+disclosure are outlined in the [governance policy](GOVERNANCE.md).
 
 [community]: https://gvisor.googlesource.com/community
 [gvisor-security-list]: https://groups.google.com/forum/#!forum/gvisor-security

WORKSPACE

@@ -380,15 +380,15 @@ go_repository(
 go_repository(
     name = "org_uber_go_atomic",
     importpath = "go.uber.org/atomic",
-    version = "v1.6.0",
     sum = "h1:Ezj3JGmsOnG1MoRWQkPBsKLe9DwWD9QeXzTRzzldNVk=",
+    version = "v1.6.0",
 )
 
 go_repository(
     name = "org_uber_go_multierr",
     importpath = "go.uber.org/multierr",
-    version = "v1.5.0",
     sum = "h1:KCa4XfM8CWFCpxXRGok+Q0SS/0XBhMDbHHGABQLvD2A=",
+    version = "v1.5.0",
 )
 
 # BigQuery Dependencies for Benchmarks

g3doc/BUILD

@@ -0,0 +1,31 @@
+load("//website:defs.bzl", "doc")
+
+package(
+    default_visibility = ["//website:__pkg__"],
+    licenses = ["notice"],
+)
+
+doc(
+    name = "index",
+    src = "README.md",
+    category = "Project",
+    permalink = "/docs/",
+    weight = "0",
+)
+
+doc(
+    name = "roadmap",
+    src = "roadmap.md",
+    category = "Project",
+    permalink = "/roadmap/",
+    weight = "10",
+)
+
+doc(
+    name = "community",
+    src = "community.md",
+    category = "Project",
+    permalink = "/community/",
+    subcategory = "Community",
+    weight = "95",
+)

g3doc/README.md

@@ -1,2 +1,27 @@
-The gVisor logo files are licensed under CC BY-SA 4.0 (Creative Commons
-Attribution-ShareAlike 4.0 International).
+# What is gVisor?
+
+gVisor is a user-space kernel, written in Go, that implements a substantial
+portion of the [Linux system call interface][linux]. It provides an additional
+layer of isolation between running applications and the host operating system.
+
+gVisor includes an [Open Container Initiative (OCI)][oci] runtime called `runsc`
+that makes it easy to work with existing container tooling. The `runsc` runtime
+integrates with Docker and Kubernetes, making it simple to run sandboxed
+containers.
+
+gVisor takes a distinct approach to container sandboxing and makes a different
+set of technical trade-offs compared to existing sandbox technologies, thus
+providing new tools and ideas for the container security landscape.
+
+gVisor can be used with Docker, Kubernetes, or directly using `runsc`. Use the
+links below to see detailed instructions for each of them:
+
+*   [Docker](./user_guide/quick_start/docker/): The quickest and easiest way to
+    get started.
+*   [Kubernetes](./user_guide/quick_start/kubernetes/): Isolate Pods in your K8s
+    cluster with gVisor.
+*   [OCI Quick Start](./user_guide/quick_start/oci/): Expert mode. Customize
+    gVisor for your environment.
+
+[linux]: https://en.wikipedia.org/wiki/Linux_kernel_interfaces
+[oci]: https://www.opencontainers.org

g3doc/architecture_guide/BUILD

@@ -0,0 +1,64 @@
+load("//website:defs.bzl", "doc")
+
+package(
+    default_visibility = ["//website:__pkg__"],
+    licenses = ["notice"],
+)
+
+doc(
+    name = "index",
+    src = "README.md",
+    category = "Architecture Guide",
+    data = [
+        "Layers.png",
+        "Layers.svg",
+        "Machine-Virtualization.png",
+        "Machine-Virtualization.svg",
+        "Rule-Based-Execution.png",
+        "Rule-Based-Execution.svg",
+        "Sentry-Gofer.png",
+        "Sentry-Gofer.svg",
+    ],
+    permalink = "/docs/architecture_guide/",
+    weight = "0",
+)
+
+doc(
+    name = "platforms",
+    src = "platforms.md",
+    category = "Architecture Guide",
+    data = [
+        "Sentry-Gofer.png",
+        "Sentry-Gofer.svg",
+    ],
+    permalink = "/docs/architecture_guide/platforms/",
+    weight = "40",
+)
+
+doc(
+    name = "resources",
+    src = "resources.md",
+    category = "Architecture Guide",
+    permalink = "/docs/architecture_guide/resources/",
+    weight = "30",
+)
+
+doc(
+    name = "security",
+    src = "security.md",
+    category = "Architecture Guide",
+    data = [
+        "Layers.png",
+        "Layers.svg",
+    ],
+    permalink = "/docs/architecture_guide/security/",
+    weight = "10",
+)
+
+doc(
+    name = "performance",
+    src = "performance.md",
+    category = "Architecture Guide",
+    permalink = "/docs/architecture_guide/performance/",
+    weight = "20",
+)

g3doc/architecture_guide/README.md

@@ -0,0 +1,80 @@
+# Overview
+
+gVisor provides a virtualized environment in order to sandbox untrusted
+containers. The system interfaces normally implemented by the host kernel are
+moved into a distinct, per-sandbox user space kernel in order to minimize the
+risk of an exploit. gVisor does not introduce large fixed overheads however, and
+still retains a process-like model with respect to resource utilization.
+
+## How is this different?
+
+Two other approaches are commonly taken to provide stronger isolation than
+native containers.
+
+**Machine-level virtualization**, such as [KVM][kvm] and [Xen][xen], exposes
+virtualized hardware to a guest kernel via a Virtual Machine Monitor (VMM). This
+virtualized hardware is generally enlightened (paravirtualized) and additional
+mechanisms can be used to improve the visibility between the guest and host
+(e.g. balloon drivers, paravirtualized spinlocks). Running containers in
+distinct virtual machines can provide great isolation, compatibility and
+performance (though nested virtualization may bring challenges in this area),
+but for containers it often requires additional proxies and agents, and may
+require a larger resource footprint and slower start-up times.
+
+![Machine-level virtualization](Machine-Virtualization.png "Machine-level virtualization")
+
+**Rule-based execution**, such as [seccomp][seccomp], [SELinux][selinux] and
+[AppArmor][apparmor], allows the specification of a fine-grained security policy
+for an application or container. These schemes typically rely on hooks
+implemented inside the host kernel to enforce the rules. If the surface can be
+made small enough (i.e. a sufficiently complete policy defined), then this is an
+excellent way to sandbox applications and maintain native performance. However,
+in practice it can be extremely difficult (if not impossible) to reliably define
+a policy for arbitrary, previously unknown applications, making this approach
+challenging to apply universally.
+
+![Rule-based execution](Rule-Based-Execution.png "Rule-based execution")
+
+Rule-based execution is often combined with additional layers for
+defense-in-depth.
+
+**gVisor** provides a third isolation mechanism, distinct from those above.
+
+gVisor intercepts application system calls and acts as the guest kernel, without
+the need for translation through virtualized hardware. gVisor may be thought of
+as either a merged guest kernel and VMM, or as seccomp on steroids. This
+architecture allows it to provide a flexible resource footprint (i.e. one based
+on threads and memory mappings, not fixed guest physical resources) while also
+lowering the fixed costs of virtualization. However, this comes at the price of
+reduced application compatibility and higher per-system call overhead.
+
+![gVisor](Layers.png "gVisor")
+
+On top of this, gVisor employs rule-based execution to provide defense-in-depth
+(details below).
+
+gVisor's approach is similar to [User Mode Linux (UML)][uml], although UML
+virtualizes hardware internally and thus provides a fixed resource footprint.
+
+Each of the above approaches may excel in distinct scenarios. For example,
+machine-level virtualization will face challenges achieving high density, while
+gVisor may provide poor performance for system call heavy workloads.
+
+### Why Go?
+
+gVisor is written in [Go][golang] in order to avoid security pitfalls that can
+plague kernels. With Go, there are strong types, built-in bounds checks, no
+uninitialized variables, no use-after-free, no stack overflow, and a built-in
+race detector. (The use of Go has its challenges too, and isn't free.)
+
+### What about Gofers?
+
+<a name="gofer"></a> <!-- For deep linking. -->
+
+[apparmor]: https://wiki.ubuntu.com/AppArmor
+[golang]: https://golang.org
+[kvm]: https://www.linux-kvm.org
+[seccomp]: https://www.kernel.org/doc/Documentation/prctl/seccomp_filter.txt
+[selinux]: https://selinuxproject.org
+[uml]: http://user-mode-linux.sourceforge.net/
+[xen]: https://www.xenproject.org

g3doc/architecture_guide/performance.md

@@ -0,0 +1,274 @@
+# Performance Guide
+
+[TOC]
+
+gVisor is designed to provide a secure, virtualized environment while preserving
+key benefits of containerization, such as small fixed overheads and a dynamic
+resource footprint. For containerized infrastructure, this can provide a
+turn-key solution for sandboxing untrusted workloads: there are no changes to
+the fundamental resource model.
+
+gVisor imposes runtime costs over native containers. These costs come in two
+forms: additional cycles and memory usage, which may manifest as increased
+latency, reduced throughput or density, or not at all. In general, these costs
+come from two different sources.
+
+First, the existence of the [Sentry](../) means that additional memory will be
+required, and application system calls must traverse additional layers of
+software. The design emphasizes [security](../security/) and therefore we chose
+to use a language for the Sentry that provides benefits in this domain but may
+not yet offer the raw performance of other choices. Costs imposed by these
+design choices are **structural costs**.
+
+Second, as gVisor is an independent implementation of the system call surface,
+many of the subsystems or specific calls are not as optimized as more mature
+implementations. A good example here is the network stack, which is continuing
+to evolve but does not support all the advanced recovery mechanisms offered by
+other stacks and is less CPU efficient. This is an **implementation cost** and
+is distinct from **structural costs**. Improvements here are ongoing and driven
+by the workloads that matter to gVisor users and contributors.
+
+This page provides a guide for understanding baseline performance, and calls out
+distint **structural costs** and **implementation costs**, highlighting where
+improvements are possible and not possible.
+
+While we include a variety of workloads here, it’s worth emphasizing that gVisor
+may not be an appropriate solution for every workload, for reasons other than
+performance. For example, a sandbox may provide minimal benefit for a trusted
+database, since _user data would already be inside the sandbox_ and there is no
+need for an attacker to break out in the first place.
+
+## Methodology
+
+All data below was generated using the [benchmark tools][benchmark-tools]
+repository, and the machines under test are uniform [Google Compute Engine][gce]
+Virtual Machines (VMs) with the following specifications:
+
+    Machine type: n1-standard-4 (broadwell)
+    Image: Debian GNU/Linux 9 (stretch) 4.19.0-0
+    BootDisk: 2048GB SSD persistent disk
+
+Through this document, `runsc` is used to indicate the runtime provided by
+gVisor. When relevant, we use the name `runsc-platform` to describe a specific
+[platform choice](../platforms/).
+
+**Except where specified, all tests below are conducted with the `ptrace`
+platform. The `ptrace` platform works everywhere and does not require hardware
+virtualization or kernel modifications but suffers from the highest structural
+costs by far. This platform is used to provide a clear understanding of the
+performance model, but in no way represents an ideal scenario. In the future,
+this guide will be extended to bare metal environments and include additional
+platforms.**
+
+## Memory access
+
+gVisor does not introduce any additional costs with respect to raw memory
+accesses. Page faults and other Operating System (OS) mechanisms are translated
+through the Sentry, but once mappings are installed and available to the
+application, there is no additional overhead.
+
+{% include graph.html id="sysbench-memory"
+url="/performance/sysbench-memory.csv" title="perf.py sysbench.memory
+--runtime=runc --runtime=runsc" %}
+
+The above figure demonstrates the memory transfer rate as measured by
+`sysbench`.
+
+## Memory usage
+
+The Sentry provides an additional layer of indirection, and it requires memory
+in order to store state associated with the application. This memory generally
+consists of a fixed component, plus an amount that varies with the usage of
+operating system resources (e.g. how many sockets or files are opened).
+
+For many use cases, fixed memory overheads are a primary concern. This may be
+because sandboxed containers handle a low volume of requests, and it is
+therefore important to achieve high densities for efficiency.
+
+{% include graph.html id="density" url="/performance/density.csv" title="perf.py
+density --runtime=runc --runtime=runsc" log="true" y_min="100000" %}
+
+The above figure demonstrates these costs based on three sample applications.
+This test is the result of running many instances of a container (50, or 5 in
+the case of redis) and calculating available memory on the host before and
+afterwards, and dividing the difference by the number of containers. This
+technique is used for measuring memory usage over the `usage_in_bytes` value of
+the container cgroup because we found that some container runtimes, other than
+`runc` and `runsc`, do not use an individual container cgroup.
+
+The first application is an instance of `sleep`: a trivial application that does
+nothing. The second application is a synthetic `node` application which imports
+a number of modules and listens for requests. The third application is a similar
+synthetic `ruby` application which does the same. Finally, we include an
+instance of `redis` storing approximately 1GB of data. In all cases, the sandbox
+itself is responsible for a small, mostly fixed amount of memory overhead.
+
+## CPU performance
+
+gVisor does not perform emulation or otherwise interfere with the raw execution
+of CPU instructions by the application. Therefore, there is no runtime cost
+imposed for CPU operations.
+
+{% include graph.html id="sysbench-cpu" url="/performance/sysbench-cpu.csv"
+title="perf.py sysbench.cpu --runtime=runc --runtime=runsc" %}
+
+The above figure demonstrates the `sysbench` measurement of CPU events per
+second. Events per second is based on a CPU-bound loop that calculates all prime
+numbers in a specified range. We note that `runsc` does not impose a performance
+penalty, as the code is executing natively in both cases.
+
+This has important consequences for classes of workloads that are often
+CPU-bound, such as data processing or machine learning. In these cases, `runsc`
+will similarly impose minimal runtime overhead.
+
+{% include graph.html id="tensorflow" url="/performance/tensorflow.csv"
+title="perf.py tensorflow --runtime=runc --runtime=runsc" %}
+
+For example, the above figure shows a sample TensorFlow workload, the
+[convolutional neural network example][cnn]. The time indicated includes the
+full start-up and run time for the workload, which trains a model.
+
+## System calls
+
+Some **structural costs** of gVisor are heavily influenced by the
+[platform choice](../platforms/), which implements system call interception.
+Today, gVisor supports a variety of platforms. These platforms present distinct
+performance, compatibility and security trade-offs. For example, the KVM
+platform has low overhead system call interception but runs poorly with nested
+virtualization.
+
+{% include graph.html id="syscall" url="/performance/syscall.csv" title="perf.py
+syscall --runtime=runc --runtime=runsc-ptrace --runtime=runsc-kvm" y_min="100"
+log="true" %}
+
+The above figure demonstrates the time required for a raw system call on various
+platforms. The test is implemented by a custom binary which performs a large
+number of system calls and calculates the average time required.
+
+This cost will principally impact applications that are system call bound, which
+tend to be high-performance data stores and static network services. In general,
+the impact of system call interception will be lower the more work an
+application does.
+
+{% include graph.html id="redis" url="/performance/redis.csv" title="perf.py
+redis --runtime=runc --runtime=runsc" %}
+
+For example, `redis` is an application that performs relatively little work in
+userspace: in general it reads from a connected socket, reads or modifies some
+data, and writes a result back to the socket. The above figure shows the results
+of running [comprehensive set of benchmarks][redis-benchmark]. We can see that
+small operations impose a large overhead, while larger operations, such as
+`LRANGE`, where more work is done in the application, have a smaller relative
+overhead.
+
+Some of these costs above are **structural costs**, and `redis` is likely to
+remain a challenging performance scenario. However, optimizing the
+[platform](../platforms/) will also have a dramatic impact.
+
+## Start-up time
+
+For many use cases, the ability to spin-up containers quickly and efficiently is
+important. A sandbox may be short-lived and perform minimal user work (e.g. a
+function invocation).
+
+{% include graph.html id="startup" url="/performance/startup.csv" title="perf.py
+startup --runtime=runc --runtime=runsc" %}
+
+The above figure indicates how total time required to start a container through
+[Docker][docker]. This benchmark uses three different applications. First, an
+alpine Linux-container that executes `true`. Second, a `node` application that
+loads a number of modules and binds an HTTP server. The time is measured by a
+successful request to the bound port. Finally, a `ruby` application that
+similarly loads a number of modules and binds an HTTP server.
+
+> Note: most of the time overhead above is associated Docker itself. This is
+> evident with the empty `runc` benchmark. To avoid these costs with `runsc`,
+> you may also consider using `runsc do` mode or invoking the
+> [OCI runtime](../../user_guide/quick_start/oci/) directly.
+
+## Network
+
+Networking is mostly bound by **implementation costs**, and gVisor's network
+stack is improving quickly.
+
+While typically not an important metric in practice for common sandbox use
+cases, nevertheless `iperf` is a common microbenchmark used to measure raw
+throughput.
+
+{% include graph.html id="iperf" url="/performance/iperf.csv" title="perf.py
+iperf --runtime=runc --runtime=runsc" %}
+
+The above figure shows the result of an `iperf` test between two instances. For
+the upload case, the specified runtime is used for the `iperf` client, and in
+the download case, the specified runtime is the server. A native runtime is
+always used for the other endpoint in the test.
+
+{% include graph.html id="applications" metric="requests_per_second"
+url="/performance/applications.csv" title="perf.py http.(node|ruby)
+--connections=25 --runtime=runc --runtime=runsc" %}
+
+The above figure shows the result of simple `node` and `ruby` web services that
+render a template upon receiving a request. Because these synthetic benchmarks
+do minimal work per request, must like the `redis` case, they suffer from high
+overheads. In practice, the more work an application does the smaller the impact
+of **structural costs** become.
+
+## File system
+
+Some aspects of file system performance are also reflective of **implementation
+costs**, and an area where gVisor's implementation is improving quickly.
+
+In terms of raw disk I/O, gVisor does not introduce significant fundamental
+overhead. For general file operations, gVisor introduces a small fixed overhead
+for data that transitions across the sandbox boundary. This manifests as
+**structural costs** in some cases, since these operations must be routed
+through the [Gofer](../) as a result of our [security model](../security/), but
+in most cases are dominated by **implementation costs**, due to an internal
+[Virtual File System][vfs] (VFS) implementation that needs improvement.
+
+{% include graph.html id="fio-bw" url="/performance/fio.csv" title="perf.py fio
+--engine=sync --runtime=runc --runtime=runsc" log="true" %}
+
+The above figures demonstrate the results of `fio` for reads and writes to and
+from the disk. In this case, the disk quickly becomes the bottleneck and
+dominates other costs.
+
+{% include graph.html id="fio-tmpfs-bw" url="/performance/fio-tmpfs.csv"
+title="perf.py fio --engine=sync --runtime=runc --tmpfs=True --runtime=runsc"
+log="true" %}
+
+The above figure shows the raw I/O performance of using a `tmpfs` mount which is
+sandbox-internal in the case of `runsc`. Generally these operations are
+similarly bound to the cost of copying around data in-memory, and we don't see
+the cost of VFS operations.
+
+{% include graph.html id="httpd100k" metric="transfer_rate"
+url="/performance/httpd100k.csv" title="perf.py http.httpd --connections=1
+--connections=5 --connections=10 --connections=25 --runtime=runc
+--runtime=runsc" %}
+
+The high costs of VFS operations can manifest in benchmarks that execute many
+such operations in the hot path for serving requests, for example. The above
+figure shows the result of using gVisor to serve small pieces of static content
+with predictably poor results. This workload represents `apache` serving a
+single file sized 100k from the container image to a client running
+[ApacheBench][ab] with varying levels of concurrency. The high overhead comes
+principally from the VFS implementation that needs improvement, with several
+internal serialization points (since all requests are reading the same file).
+Note that some of some of network stack performance issues also impact this
+benchmark.
+
+{% include graph.html id="ffmpeg" url="/performance/ffmpeg.csv" title="perf.py
+media.ffmpeg --runtime=runc --runtime=runsc" %}
+
+For benchmarks that are bound by raw disk I/O and a mix of compute, file system
+operations are less of an issue. The above figure shows the total time required
+for an `ffmpeg` container to start, load and transcode a 27MB input video.
+
+[ab]: https://en.wikipedia.org/wiki/ApacheBench
+[benchmark-tools]: https://github.com/google/gvisor/tree/master/benchmarks
+[gce]: https://cloud.google.com/compute/
+[cnn]: https://github.com/aymericdamien/TensorFlow-Examples/blob/master/examples/3_NeuralNetworks/convolutional_network.py
+[docker]: https://docker.io
+[redis-benchmark]: https://redis.io/topics/benchmarks
+[vfs]: https://en.wikipedia.org/wiki/Virtual_file_system