- Move ktime.VariableTimer to kernel.timekeeperTcpipTimer, its only use case.
This allows timekeeperTcpipTimer to use concrete types kernel.timekeeperClock
and ktime.SampledTimer instead of ktime.Clock and ktime.Timer, saving a tiny
amount of memory (interface values consist of two pointers) and CPU (for
interface method calls).
- Fix a bug where timekeeperTcpipTimer expiration can cancel a racing call to
timekeeperTcpipTimer.Reset() (see use of new field
timekeeperTcpipTimer.resets).
- Define Listener.NotifyTimer directly on timekeeperTcpipTimer (dropping
ktime.functionNotifier), and move goroutine spawning from the anonymous
function in ktime.AfterFunc() into timekeeperTcpipTimer.NotifyTimer(). This
slightly simplifies the control flow and saves an allocation for the
anonymous function object.
- Use monotonicClock rather than realtimeClock. It doesn't make sense for
time-of-day clock adjustments to affect netstack timeouts, and this is
consistent with tcpip.stdClock => time.AfterFunc => runtime.timer.
PiperOrigin-RevId: 695504159
gVisor currently implements CPU clocks as follows:
- A per-sentry "CPU clock ticker goroutine"
(task_sched.go:Kernel.runCPUClockTicker()) periodically advances
Kernel.cpuClock, causing it to serve as a very coarse but inexpensive
monotonic wall clock (that happens to be suspended when no tasks are
running).
- Task goroutines observe the most recent value of Kernel.cpuClock when
changing state (Task.gosched.Timestamp), and use it to compute the number of
CPU clock ticks that have elapsed in a given state. Thus, task CPU clocks are
approximately based on the wall time during which they were marked as
running.
- ITIMER_VIRTUAL, ITIMER_PROF, and RLIMIT_CPU are checked by the CPU clock
ticker goroutine after advancing Kernel.cpuClock. POSIX interval timers and
timerfds check CPU clocks (taskClock/tgClock) in ktime.SampledTimer
goroutines.
This has three major problems:
- ktime.SampledTimer goroutines for CPU clock timers run concurrently with the
CPU clock ticker, and are not informed as to when corresponding tasks start
or stop running (due to overhead on the task execution critical path), so
they can't determine when CPU clocks have/will advance; instead, they simply
poll CPU clocks on a period equal to that of the represented timer, resulting
in significant overhead for CPU-clock-based POSIX interval timers and
timerfds.
- For the same reason, CPU clock interval timers and timerfds may expire much
later than when the CPU clock is actually incremented; in the interval timer
case, this can result in notification signals being sent long after tasks
have stopped running. (This is the same problem as in b/116538398, which
motivated the special-casing of ITIMER_VIRTUAL and ITIMER_PROF described
above, but applied to POSIX interval timers.)
- The sentry does not impose a limit on the number of tasks that may be
concurrently marked running, so if more tasks are marked running than the
number of CPUs advertised to applications, application CPU utilization can
appear to exceed 100%.
This CL fixes these problems by introducing explicit per-Task and ThreadGroup
CPU clocks, directly advancing (up to Kernel.applicationCores of) them in the
CPU clock ticker, and directly expiring CPU timers when doing so. Itimer and
RLIMIT_CPU timers lose their special-casing and instead behave like other CPU
timers (see task_acct.go). Kernel.cpuClock is still required, but only for the
sentry watchdog.
Minor cleanup changes:
- Gather all stateify hooks in kernel_state.go.
- Replace kernel.randInt31n() with math/rand/v2, which fixes the same problem
(https://go.dev/blog/randv2#problem.rand).
Test workload:
```
#include <err.h>
#include <signal.h>
#include <time.h>
#include <chrono>
#include <thread>
constexpr int kNumTimers = 1000;
constexpr long kTimerPeriodNS = 10000000;
int main(int argc, char** argv) {
for (int i = 0; i < kNumTimers; i++) {
struct sigevent sev = {.sigev_notify = SIGEV_NONE};
timer_t timerid;
if (timer_create(CLOCK_THREAD_CPUTIME_ID, &sev, &timerid) < 0) {
err(1, "timer_create failed");
}
struct itimerspec it = {
.it_interval = {0, kTimerPeriodNS},
.it_value = {0, kTimerPeriodNS},
};
if (timer_settime(timerid, 0, &it, nullptr) < 0) {
err(1, "timer_settime failed");
}
}
std::this_thread::sleep_for(std::chrono::seconds(5));
return 0;
}
```
Before this CL:
```
# /usr/bin/time ./runsc --ignore-cgroups --platform kvm --network none do $(pwd)/workloads/threadcputimers
1.50user 0.17system 0:05.25elapsed 31%CPU (0avgtext+0avgdata 35792maxresident)k
0inputs+184outputs (10major+20889minor)pagefaults 0swaps
```
After this CL:
```
# /usr/bin/time ./runsc --ignore-cgroups --platform kvm --network none do $(pwd)/workloads/threadcputimers
0.10user 0.12system 0:05.22elapsed 4%CPU (0avgtext+0avgdata 34040maxresident)k
0inputs+192outputs (6major+20929minor)pagefaults 0swaps
```
PiperOrigin-RevId: 695198313
This allows "remote" locking of ThreadGroup.signalHandlers.mu without needing
to lock TaskSet.mu, analogously to Linux's lock_task_sighand().
This reveals a bug: kernel.Task.sendSignal[Timer]Locked() unintentionally
requires TaskSet.mu to be locked since it reads Task.exitState. To fix this,
use atomic memory operations on Task.exitState when required.
PiperOrigin-RevId: 681128063
When a pages file is provided to `runsc restore`, reads from that file are
asynchronous (via statefile.AsyncReader) in order to maximize throughput.
However, all such reads must complete before Kernel.LoadFrom() returns, so
applications cannot execute before MemoryFile loading is complete. The main
objective of this CL is to allow reads to continue after Kernel.LoadFrom()
returns, allowing applications to execute while MemoryFile loading is still in
progress. This behavior is user-visible: it affects whether deleting the pages
file frees disk space immediately on POSIX filesystems, may affect whether
deletion is possible on non-POSIX filesystems, and prevents unmounting
regardless. Thus it is flag-guarded as `runsc restore --background`.
MemoryFile ranges that have yet to be loaded, but that are being waited-for by
applications, should be prioritized over ranges for which no application is
waiting. This requires that application requests for data (calls to
MemoryFile.(memmap.File).DataFD/MapInternal()) are able to determine which
ranges have not yet been loaded, request reads for such ranges with elevated
priority, and wait for only those reads to be completed; none of these are
supported by the existing statefile.AsyncReader.
Thus:
- Add //pkg/sentry/pgalloc/aio, which provides an async I/O API that is
designed to be easily implementable using a goroutine pool, Linux native AIO,
or io_uring, though only includes a goroutine pool implementation. (io_uring
is widely disabled due to security vulnerabilities. In my testing, Linux
native AIO is slower than the goroutine pool, but this may change with lower
GOMAXPROCS which needs further testing.)
- Move I/O scheduling into pgalloc: introduce an async page loader goroutine
that is started by MemoryFile.LoadFrom() when async page loading is requested
(implicitly, via the existence of a pages file), which is responsible for
driving submission of read requests and handling their completions.
PiperOrigin-RevId: 679321884
With S/R enabled, the kernel is replaced during the container creation
before attempting to restore in a new sandbox. The old kernel which was
being replaced did not release the resources resulting in ref leaks. This CL
releases the resources before replacing the kernel in restore.
PiperOrigin-RevId: 603822183
Adds a gofer connection for /dev directory on the gofer when GPU functionality
is requested. This gofer connection is currently unused. The gofer client is
owned by the kernel, which injects the connection into the context. The gofer
connection is closed on container exit. S/R should be supported with this.
PiperOrigin-RevId: 581298536
Also add "num" at the end of it so make it clear that it's an *inclusive*
limit. Fix the uses of it which didn't treat it as inclusive.
No strong opinion on where it should go, but having it available in
other places of the codebase is helpful. It's already in two places,
and I'd like to have it available in two more places:
- In the seccomp program compiler, so that it knows how many syscalls
it should check for deterministic-ness and caching the results.
- In the seccomp fuzzer, to build an initial syscall corpus.
PiperOrigin-RevId: 576660986
The intention of this change is to cover a sufficient surface to accommodate
the use of running Docker within gVisor, rather than a full implementation.
This implements the following features:
- Keys as a first-class concept in the kernel.
- Tracking keys in user namespaces.
- Task session keyrings: possession, inheritance.
- Key permission enforcement.
- The following `keyctl(2)` operations:
- `KEYCTL_GET_KEYRING_ID`
- `KEYCTL_DESCRIBE`
- `KEYCTL_JOIN_SESSION_KEYRING`
- `KEYCTL_SETPERM`
Notably, this does not implement:
- The ability to actually add any keys other than the session keyring
(which does not hold any cryptographic key data).
- Other special keyrings (thread keyring, process keyring, user session
keyring, etc.).
- Lots of `keyctl(2)` operations.
- Key expiration.
- Key garbage collection. Keys live until their user namespace is destroyed.
However, each user namespace is limited to 200 keys, so memory growth is
bounded.
- `add_key(2)`
- `request_key(2)`
... However, this makes design choices that seem odd given the limited scope
of this change, but make sense when taking into account the desire to
eventually accommodate them in the future. For example, there are many
`switch` statements with only one option for session keyrings, which would get
more options when adding support for other special keyrings. Similarly, the
signature of `PossessedKeys` takes in all 3 special "possessed" keyrings, but
currently only ever gets the session keyring as non-nil.
PiperOrigin-RevId: 567047896
This change introduces the nsfs file system. Each new namespace allocates
a new nsfs inode.
Here are reasons why we need these inodes:
* each namespace has to have an unique id.
* proc/pid/ns/ contains one entry for each namespace. Bind mounting one of
the files in this directory to somewhere else in the filesystem keeps the
corresponding namespace alive even if all processes currently in
the namespace terminate.
* setns() allows the calling process to join an existing namespace specified
by a file descriptor.
PiperOrigin-RevId: 550694515
FDTable.descriptorTable is a slice of unsafe.Pointer-s and its maximum length
is MaxInt32. It requires up to 16GB of memory. A process can use just a few
descriptors but sets one or more of them to high numbers. In this case,
FDTable.descriptorTable is extended to the maximum size.
The problem here is that go-runtime zeros memory regions when they are reused.
In the case of fdtable, the memory region is 16GB, so it is a time consuming
operation. Second, it forces the kernel to allocate physical pages to
the entire region.
This change adds another level to descriptorTable, so the first level is
a slice of buckets where each bucket is a slice of descriptors. The bucket
size is fixed to 512 entries to fit one page.
Before:
BenchmarkFDLookupAndDecRef-12 50834290 23.70 ns/op
BenchmarkCreateWithMaxFD-12 2 7194873988 ns/op
BenchmarkFDLookupAndDecRefConcurrent-12 23775555 49.68 ns/op
BenchmarkTableLookup-12 412888780 2.835 ns/op
BenchmarkTableMapLookup-12 87944782 12.84 ns/op
After:
BenchmarkFDLookupAndDecRef-12 46229940 25.03 ns/op
BenchmarkCreateWithMaxFD-12 13 82573899 ns/op
BenchmarkFDLookupAndDecRefConcurrent-12 21889380 54.13 ns/op
BenchmarkTableLookup-12 415851230 2.821 ns/op
BenchmarkTableMapLookup-12 97236267 11.89 ns/op
Reported-by: syzbot+af17678e3bfb7ca7c65a@syzkaller.appspotmail.com
PiperOrigin-RevId: 539138632
This CL does the following:
- Add the ability for nested locks to have names.
- Give names to all current uses of nested locks in the codebase.
- Truncate `lockdep` debug stack traces to avoid the clutter from the
`lockdep` code itself
- Simplify `lockdep` to not longer require `classMap`.
PiperOrigin-RevId: 491486620
Jamie LiuandgVisor bot
Fabricio VoznikaandgVisor bot
Ayush RanjanandgVisor bot
Nayana BidariandgVisor bot
Etienne PerotandgVisor bot
Andrei VaginandgVisor bot
Jing ChenandgVisor bot
Adin ScannellandgVisor bot