26 Commits

Author SHA1 Message Date
gVisor bot 079c1a937b Merge pull request #9551 from amysaq2023:support-external-stack
PiperOrigin-RevId: 677933413
2024-09-23 14:12:32 -07:00
Anqi Shen 56f2530dad Support plugin network stack
This commit supports a third-party network stack as a plugin stack for
gVisor.

The overall plugin package structure is the following:

- pkg/sentry/socket/plugin:
  Interfaces for initializing plugin network stack. It will be used
  in network setting up during sandbox creating.

- pkg/sentry/socket/plugin/stack:
  Glue layer for plugin stack's socket and stack ops with sentry. It
  will also register plugin stack operations if imported.

- pkg/sentry/socket/plugin/cgo:
  Interfaces defined in C for plugin network stack to support.

To build target runsc-plugin-stack, which imports
pkg/sentry/socket/plugin/stack package and enables CGO:

bazel build --config=plugin-tldk runsc:runsc-plugin-stack

(i.e. --config=plugin-tldk indicates that using TLDK as plugin stack)

By using runsc-plugin-stack binary and setting "--network=plugin" in
runtimeArgs, user can use third-party network stack instead of
netstack embedded in gVisor to get better network performance.

Redis benchmark with following setups:
1. KVM platform
2. 4 physical cores for target pod
3. target pod as redis server

Runc:
$redis-benchmark -h [target ip] -n 100000 -t get,set -q
SET: 115207.38 requests per second, p50=0.215 msec
GET: 92336.11 requests per second, p50=0.279 msec

$redis-benchmark -h [target ip] -n 100000 -t get,set -q
SET: 113895.21 requests per second, p50=0.247 msec
GET: 96899.23 requests per second, p50=0.271 msec

$redis-benchmark -h [target ip] -n 100000 -t get,set -q
SET: 126582.27 requests per second, p50=0.199 msec
GET: 95969.28 requests per second, p50=0.271 msec

Runsc with plugin stack:
$redis-benchmark -h [target ip] -n 100000 -t get,set -q
SET: 123915.74 requests per second, p50=0.343 msec
GET: 115473.45 requests per second, p50=0.335 msec

$redis-benchmark -h [target ip] -n 100000 -t get,set -q
SET: 120918.98 requests per second, p50=0.351 msec
GET: 117647.05 requests per second, p50=0.351 msec

$redis-benchmark -h [target ip] -n 100000 -t get,set -q
SET: 119904.08 requests per second, p50=0.367 msec
GET: 112739.57 requests per second, p50=0.375 msec

Runsc with netstack:
$redis-benchmark -h [target ip] -n 100000 -t get,set -q
SET: 59952.04 requests per second, p50=0.759 msec
GET: 61162.08 requests per second, p50=0.631 msec

$redis-benchmark -h [target ip] -n 100000 -t get,set -q
SET: 52219.32 requests per second, p50=0.719 msec
GET: 58719.91 requests per second, p50=0.663 msec

$redis-benchmark -h [target ip] -n 100000 -t get,set -q
SET: 59952.04 requests per second, p50=0.751 msec
GET: 60827.25 requests per second, p50=0.751 msec

Updates https://github.com/google/gvisor/issues/9266

Co-developed-by: Tianyu Zhou <wentong.zty@antgroup.com>
Signed-off-by: Anqi Shen <amy.saq@antgroup.com>
2024-07-12 09:10:13 +00:00
Jing Chen cf5c4c9cbf Replace reflect.DeepEqual with [slices/maps].Equal.
They are faster on slice/map comparisons.

PiperOrigin-RevId: 633080355
2024-05-12 21:20:18 -07:00
Kevin Krakauer 3f8ecf023c netstack: don't allocate interfaces when copying data in and out
In a redis-benchmark PING_INLINE test, this reduces allocations by 32%.

PiperOrigin-RevId: 618248114
2024-03-22 12:01:02 -07:00
prof awk 4d30f2c9ef use new clear builtin to clear bufs 2023-11-27 19:43:25 +02:00
Adin Scannell 1ceb814544 Add default_applicable_licenses rules to packages.
PiperOrigin-RevId: 513581243
2023-03-02 10:50:04 -08:00
Ayush Ranjan f6ed4523dc Reformat codebase.
PiperOrigin-RevId: 449358041
2022-05-17 17:48:35 -07:00
Kevin Krakauer ec44093c97 introduce atomicbitops 32-bit types
Part of a series of changes that will end with prohibiting use of sync/atomic
(u)int32 functions. See cl/440484071 for more details.

PiperOrigin-RevId: 442673296
2022-04-18 17:41:53 -07:00
Zach Koopmans ce58d71fd5 [syserror] Remove pkg syserror.
Removes package syserror and moves still relevant code to either linuxerr
or to syserr (to be later removed).

Internal errors are converted from random types to *errors.Error types used
in linuxerr. Internal errors are in linuxerr/internal.go.

PiperOrigin-RevId: 390724202
2021-08-13 17:16:52 -07:00
Zach Koopmans e3fdd15932 [syserror] Update syserror to linuxerr for more errors.
Update the following from syserror to the linuxerr equivalent:
EEXIST
EFAULT
ENOTDIR
ENOTTY
EOPNOTSUPP
ERANGE
ESRCH

PiperOrigin-RevId: 384329869
2021-07-12 15:26:20 -07:00
Zach Koopmans 590b8d3e99 [syserror] Update several syserror errors to linuxerr equivalents.
Update/remove most syserror errors to linuxerr equivalents. For list
of removed errors, see //pkg/syserror/syserror.go.

PiperOrigin-RevId: 382574582
2021-07-01 12:05:19 -07:00
Zach Koopmans 54b71221c0 [syserror] Change syserror to linuxerr for E2BIG, EADDRINUSE, and EINVAL
Remove three syserror entries duplicated in linuxerr. Because of the
linuxerr.Equals method, this is a mere change of return values from
syserror to linuxerr definitions.

Done with only these three errnos as CLs removing all grow to a significantly
large size.

PiperOrigin-RevId: 382173835
2021-06-29 15:08:46 -07:00
Zach Koopmans e1dc1c78e7 [syserror] Add conversions to linuxerr with temporary Equals method.
Add Equals method to compare syserror and unix.Errno errors to linuxerr errors.
This will facilitate removal of syserror definitions in a followup, and
finding needed conversions from unix.Errno to linuxerr.

PiperOrigin-RevId: 380909667
2021-06-22 15:53:32 -07:00
Ian Lewis 5c9e846223 Remove usermem dependency from marshal
Both marshal and usermem are depended on by many packages and a dependency on
marshal can often create circular dependencies. marshal should consider adding
internal dependencies carefully moving forward.

Fixes #6160

PiperOrigin-RevId: 379199882
2021-06-13 19:23:55 -07:00
Zach Koopmans 8a2f7e716d [syserror] Split usermem package
Split usermem package to help remove syserror dependency in go_marshal.
New hostarch package contains code not dependent on syserror.

PiperOrigin-RevId: 365651233
2021-03-29 13:30:21 -07:00
Ayush Ranjan a9441aea27 [op] Replace syscall package usage with golang.org/x/sys/unix in pkg/.
The syscall package has been deprecated in favor of golang.org/x/sys.

Note that syscall is still used in the following places:
- pkg/sentry/socket/hostinet/stack.go: some netlink related functionalities
  are not yet available in golang.org/x/sys.
- syscall.Stat_t is still used in some places because os.FileInfo.Sys() still
  returns it and not unix.Stat_t.

Updates #214

PiperOrigin-RevId: 360701387
2021-03-03 10:25:58 -08:00
Jamie Liu f051ec6463 Add gohacks.Slice/StringHeader.
See https://github.com/golang/go/issues/19367 for rationale. Note that the
upstream decision arrived at in that thread, while useful for some of our use
cases, doesn't account for all of our SliceHeader use cases (we often use
SliceHeader to extract pointers from slices in a way that avoids bounds
checking and/or handles nil slices correctly) and also doesn't exist yet.

PiperOrigin-RevId: 358071574
2021-02-17 17:41:10 -08:00
Tamir Duberstein 6c0e1d9cfe Define tcpip.Payloader in terms of io.Reader
Fixes #1509.

PiperOrigin-RevId: 353295589
2021-01-22 12:26:09 -08:00
Adin Scannell 4cba3904f4 Remove existing nogo exceptions.
PiperOrigin-RevId: 347047550
2020-12-11 12:06:49 -08:00
Rahat Mahmood 387501219e Replace remaining uses of reflection-based marshalling.
- Rewrite arch.Stack.{Push,Pop}. For the most part, stack now
  implements marshal.CopyContext and can be used as the target of
  marshal operations. Stack.Push had some extra logic for
  automatically null-terminating slices. This was only used for two
  specific types of slices, and is now handled explicitly.

- Delete usermem.CopyObject{In,Out}.

- Replace most remaining uses of the encoding/binary package with
  go-marshal. Most of these were using the binary package to compute
  the size of a struct, which go-marshal can directly replace. ~3 uses
  of the binary package remain. These aren't reasonably replaceable by
  go-marshal: for example one use is to construct the syscall
  trampoline for systrap.

- Fill out remaining convenience wrappers in the primitive package.

PiperOrigin-RevId: 334502375
2020-09-29 18:08:07 -07:00
Michael Pratt 129018ab3d Consistent precondition formatting
Our "Preconditions:" blocks are very useful to determine the input invariants,
but they are bit inconsistent throughout the codebase, which makes them harder
to read (particularly cases with 5+ conditions in a single paragraph).

I've reformatted all of the cases to fit in simple rules:

1. Cases with a single condition are placed on a single line.
2. Cases with multiple conditions are placed in a bulleted list.

This format has been added to the style guide.

I've also mentioned "Postconditions:", though those are much less frequently
used, and all uses already match this style.

PiperOrigin-RevId: 327687465
2020-08-20 13:32:24 -07:00
Dean Deng 05c89af6ed Implement mmap for host fs in vfs2.
In VFS1, both fs/host and fs/gofer used the same utils for host file mappings.
Refactor parts of fsimpl/gofer to create similar utils to share with
fsimpl/host (memory accounting code moved to fsutil, page rounding arithmetic
moved to usermem).

Updates #1476.

PiperOrigin-RevId: 312345090
2020-05-19 13:46:42 -07:00
Jon Budd 6a4d17a31d Remove obsolete TODOs for b/38173783
The comments in the ticket indicate that this behavior
is fine and that the ticket should be closed, so we shouldn't
need pointers to the ticket.

PiperOrigin-RevId: 306266071
2020-04-13 11:02:14 -07:00
Michael Pratt db7917556a Fix 386 build tags
The build tag for 32-bit x86 is 386, not i386.

Updates #2298

PiperOrigin-RevId: 304206373
2020-04-01 10:00:03 -07:00
Jamie Liu 471b15b212 Port most syscalls to VFS2.
pipe and pipe2 aren't ported, pending a slight rework of pipe FDs for VFS2.
mount and umount2 aren't ported out of temporary laziness. access and faccessat
need additional FSImpl methods to implement properly, but are stubbed to
prevent googletest from CHECK-failing. Other syscalls require additional
plumbing.

Updates #1623

PiperOrigin-RevId: 297188448
2020-02-25 13:37:34 -08:00