60 Commits

Author SHA1 Message Date
Koichi Shiraishi 0cf77c02f8 all: remove use io/ioutil deprecated package & fix some deprecated thing
Signed-off-by: Koichi Shiraishi <zchee.io@gmail.com>
2024-10-10 20:36:24 +09:00
Andrei Vagin ec18cb188f buildkite: run intergration tests with the network plugin 2024-09-25 21:35:17 -07:00
Etienne Perot a689c11a76 Integrate GPU ioctl sniffer in GPU tests.
This wraps all GPU tests' command line with the nvproxy ioctl sniffer.

This has multiple functions:

- Verifying that the application does not call ioctls unsupported by
  nvproxy. This is controlled by a `AllowIncompatibleIoctl` option, which
  is initially set to `true` in all tests to mirror current behavior, but
  should be flipped as we verify that they do not call unsupported ioctls.
- Verifying that the sniffer itself works transparently for a wide range
  of applications.
- Later down the line, enforcing that the application only calls ioctls
  that are part of GPU capabilities that it has a need for. This is
  controlled by a capability string which is currently only used to set
  the `NVIDIA_DRIVER_CAPABILITIES` environment variable.

Updates issue #10856

PiperOrigin-RevId: 672714520
2024-09-09 16:34:19 -07:00
Lucas Manning e0029cc0ff A few fixes to run tests on 6.2+ kernels
* disable broken socket_inet_loopback test // Lucas Manning
* disable lseek tests on anon inodes. They have been broken by
commit e7478158e1378325907edfdd960eca98a1be405b
Date:   Wed Jun 29 15:06:57 2022 +0200
    fs: clear or set FMODE_LSEEK based on llseek function

* don't check memory.kmem.limit_in_bytes, it has been deprecated and dropped:
commit 86327e8eb94c52eca4f93cfece2e29d1bf52acbf
Date:   Tue Jul 4 13:52:40 2023 +0200
    memcg: drop kmem.limit_in_bytes
commit 58056f77502f3567b760c9a8fc8d2e9081515b2d
Date:   Fri Nov 5 13:37:44 2021 -0700
    memcg, kmem: further deprecate kmem.limit_in_bytes
commit 0158115f702b0ba208ab0b5adf44cae99b3ebcc7
Date:   Mon Sep 23 15:37:22 2019 -0700
    memcg, kmem: deprecate kmem.limit_in_bytes
PiperOrigin-RevId: 562915457
2023-09-05 15:51:21 -07:00
Ayush Ranjan 039acda6e4 Preserve sandbox process env vars only for --TESTONLY-unsafe-nonroot.
Earlier setCapsAndCallSelf() and callSelfAsNobody() were unconditionally
clearing the sandbox process's env vars. But in some cases we want to preserve
it. For example, when running runsc-race, we want to set GLIBC_TUNABLES for the
sandbox process. Note that runsc-race requires --TESTONLY-unsafe-nonroot=true.

Right now, when --TESTONLY-unsafe-nonroot is set, boot process does not end up
calling setCapsAndCallSelf() or callSelfAsNobody(). So the env vars are
retained as desired. But with directfs, those methods are called in spite of
--TESTONLY-unsafe-nonroot. So this change is required to preserve env vars with
--directfs && --TESTONLY-unsafe-nonroot.

This change also adds a test to verify that the sandbox process is running with
no leaked environment variables.

PiperOrigin-RevId: 533170470
2023-05-18 10:24:09 -07:00
Zach Koopmans 9c8109c2e7 Enable portforward test after fix.
PiperOrigin-RevId: 526688386
2023-04-24 10:42:03 -07:00
Zach Koopmans 2e45ce14a4 Fix flakes in portforward test.
Attempt to fix flaky port forward test using "netstat" in the container
to check the a process is listening on the port before attempting to
connect.

Also, re-enable the test in a seperate job, but w/ a soft fail. This
will be removed later in favor of adding port-forward to integration
tests to remove the high overhead of starting a new job (scheduling,
compiling runsc, downloading images, etc).

PiperOrigin-RevId: 526135859
2023-04-21 14:28:31 -07:00
Ayush Ranjan 307a1961d8 Skip TestPortForwardLocalMode.
This test is blocking submits.

PiperOrigin-RevId: 523497354
2023-04-11 14:03:32 -07:00
Zach Koopmans 460a9fcc20 Add portforward stream mode.
Port forward stream mode will be used for k8s. Stream mode takes a
UNIX socket and donates a connected FD to the sandbox in question,
and returns. No proxy process is kept around to do the transfer
and the UDS will be returned to containerd as the address on which
to send i/o to the forwarded port.

PiperOrigin-RevId: 521823250
2023-04-04 11:46:21 -07:00
Zach Koopmans f92957314c Add portforward command to runsc
Add portforward comand so that we can use runsc to forward connections
to container ports. This will eventually be supported in k8s.

PiperOrigin-RevId: 520739913
2023-03-30 14:16:19 -07:00
Adin Scannell 1ceb814544 Add default_applicable_licenses rules to packages.
PiperOrigin-RevId: 513581243
2023-03-02 10:50:04 -08:00
Andrei Vagin 5572ab2f7d Bump go version to 1.20
Signed-off-by: Andrei Vagin <avagin@gmail.com>
2023-02-10 08:32:17 -08:00
Kevin Krakauer d8aa09e04c convert uses of interface{} to any
Done via:
  find . -name "*.go" | xargs sed -i -E 's/interface\{\}/any/g'

PiperOrigin-RevId: 487033228
2022-11-08 13:14:06 -08:00
Lucas Manning e0aa478022 Ignore systemd "scope already exists" errors.
This copies the behavior of runc. See  https://github.com/opencontainers/runc/blob/4a51b047036cf16d4f124548c2a7ff24b5640bad/libcontainer/cgroups/systemd/common.go#L150

PiperOrigin-RevId: 471110557
2022-08-30 15:53:26 -07:00
Etienne Perot de960bb645 Make root_test and containerd tests work on BuildKite on COS.
When running in COS in a container, things are different when accessing
process or cgroup data.

This CL uses the `HOST_PROCFS_MOUNTPOINT` and `HOST_CGROUPFS_MOUNTPOINT`
environment variables as alternate roots for `/proc` and `/sys/fs/cgroup` in
`root_test` and in other related places.

Also includes some minor refactors:

- Factor out code to get the parent PID of a child PID to its own function
- Fix some typos

PiperOrigin-RevId: 457102457
2022-06-24 15:12:55 -07:00
Andrei Vagin bb1a83085b buildkite: allow to run containerd tests on arm64
k8s.gcr.io/busybox:latest doesn't have the arm variant, so
let's use the upstream busybox and create /etc/recolv.conf symlink.

PiperOrigin-RevId: 450059917
2022-05-20 14:11:04 -07:00
Ayush Ranjan f6ed4523dc Reformat codebase.
PiperOrigin-RevId: 449358041
2022-05-17 17:48:35 -07:00
Lucas Manning bf86207401 Add systemd-cgroup support to runsc.
This change adds systemd-cgroup support to cgroupv2 and modifies existing
cgroup tests for systemd support.

Fixes #193

PiperOrigin-RevId: 435457762
2022-03-17 15:03:44 -07:00
Zach Koopmans 6e9962945c Automated rollback of changelist 431455723
PiperOrigin-RevId: 431504004
2022-02-28 12:53:28 -08:00
Zach Koopmans 5240904367 Fix issue with parsing flags in root_test.
PiperOrigin-RevId: 431455723
2022-02-28 09:42:39 -08:00
Adin Scannell 7ecf36cc84 Validate flags during install.
First, this change moves the internal config API to use flag.FlagSet, which
allows more flexibility and fixes many test usages.

Second, the runtime flags are validated during install. The platform is opened
and a warning issued if this fails, but this is not fatal.

This change requires moving the Makefile to --test_env, since the attribute
is not properly supported by test targets. Therefore, the targets can use args
while the Makefile must pass in configuration via --test_env.

PiperOrigin-RevId: 428048274
2022-02-11 11:24:01 -08:00
Andrei Vagin 6e7404427c runsc: Fix the data race
==================
WARNING: DATA RACE
Write at 0x00c000115be0 by main goroutine:
  gvisor.dev/gvisor/runsc/sandbox.(*Sandbox).waitForStopped()
      runsc/sandbox/sandbox.go:1275 +0x24b
  gvisor.dev/gvisor/runsc/sandbox.(*Sandbox).Wait()
      runsc/sandbox/sandbox.go:823 +0xa2c
  gvisor.dev/gvisor/runsc/container.(*Container).Wait()
      runsc/container/container.go:524 +0x131
  gvisor.dev/gvisor/runsc/cmd.startContainerAndWait()
      runsc/cmd/do.go:398 +0xa18
  gvisor.dev/gvisor/runsc/cmd.(*Do).Execute()
      runsc/cmd/do.go:156 +0xc16
  github.com/google/subcommands.(*Commander).Execute()
      external/com_github_google_subcommands/subcommands.go:200 +0x66b
  github.com/google/subcommands.Execute()
      external/com_github_google_subcommands/subcommands.go:481 +0x2096
  gvisor.dev/gvisor/runsc/cli.Main()
      runsc/cli/main.go:243 +0x1fb4
  main.main()
      runsc/main.go:23 +0x56

Previous read at 0x00c000115be0 by goroutine 9:
  gvisor.dev/gvisor/runsc/sandbox.(*Sandbox).connError()
      runsc/sandbox/sandbox.go:387 +0x1e9
  gvisor.dev/gvisor/runsc/sandbox.(*Sandbox).sandboxConnect()
      runsc/sandbox/sandbox.go:381 +0x291
  gvisor.dev/gvisor/runsc/sandbox.(*Sandbox).SignalProcess()
      runsc/sandbox/sandbox.go:932 +0x135
  gvisor.dev/gvisor/runsc/container.(*Container).ForwardSignals.func1()
      runsc/container/container.go:591 +0x267
  gvisor.dev/gvisor/pkg/sentry/sighandling.handleSignals()
      pkg/sentry/sighandling/sighandling.go:63 +0x4f8

Goroutine 9 (running) created at:
  gvisor.dev/gvisor/pkg/sentry/sighandling.StartSignalForwarding()
      pkg/sentry/sighandling/sighandling.go:96 +0x284
  gvisor.dev/gvisor/runsc/container.(*Container).ForwardSignals()
      runsc/container/container.go:589 +0x21d
  gvisor.dev/gvisor/runsc/cmd.startContainerAndWait()
      runsc/cmd/do.go:395 +0x9d0
  gvisor.dev/gvisor/runsc/cmd.(*Do).Execute()
      runsc/cmd/do.go:156 +0xc16
  github.com/google/subcommands.(*Commander).Execute()
      external/com_github_google_subcommands/subcommands.go:200 +0x66b
  github.com/google/subcommands.Execute()
      external/com_github_google_subcommands/subcommands.go:481 +0x2096
  gvisor.dev/gvisor/runsc/cli.Main()
      runsc/cli/main.go:243 +0x1fb4
  main.main()
      runsc/main.go:23 +0x56
==================
Found 1 data race(s)

Signed-off-by: Andrei Vagin <avagin@gmail.com>
2021-12-23 13:03:58 -08:00
gVisor bot fa4e2fff8a Merge pull request #6821 from dqminh:feature/cgroupv2
PiperOrigin-RevId: 413032543
2021-11-29 18:37:42 -08:00
Daniel Dao 881a271ff7 runsc: Add cgroup v2 implementation
Adds support for cgroupv2 based on the common cgroup interface.

The cgroupv2 implementation mostly mirrors the structure of cgroupv1,
with many helper functions derived from containerd/cgroups and opencontainers/runc
implementations.  We implemented the following controllers: cpu, cpuset, memory,
io, pids, hugetlb.

In order to avoid upgrading containerd dependency (to get oom poller
implementation), we copied the oom poller implementation for cgroupv2
into shim/oom_v2.go. This requires containerd/cgroups dependency to have
cgroupv2 support which we already have.

Signed-off-by: Daniel Dao <dqminh89@gmail.com>
2021-11-26 10:14:08 +00:00
Fabricio Voznika 9262ea47a5 Add support for containerd 1.5
"cri.runtimeoptions.v1" moved to "runtimeoptions.v1" and containerd
configuration format version 2 is required.

Updates #6449

PiperOrigin-RevId: 405474653
2021-10-25 13:17:25 -07:00