When using overlayfs with tmpfs as the upper layer (common case), depending on
the application, a lot of whiteouts can be created. This leads to a lot of
memory allocation because new dentry and inode structs need to be allocated for
each whiteout. With this change, we at least avoid the inode allocations.
This is analogous with what Linux does. See fs/overlayfs/ovl_entry.h:ovl_fs's
field `whiteout` with comment "Shared whiteout cache".
PiperOrigin-RevId: 704897805
For vfs.FileDescriptions for which FileDescriptionOptions.UseDentryMetadata is
true, memmap.MappingIdentity.Device/InodeID() => FileDescription.Stat() =>
FilesystemImpl.StatAt() takes fsimpl locks for path traversal, which violates
the lock ordering and is unnecessary since no path is being traversed. Fix this
by carving out a special case where FilesystemImpl.Stat() (and
FileDescriptionImpl.Stat()) are required to meet the lock ordering requirements
of memmap.MappingIdentity.Device/InodeID(), and implement that special case by
skipping path traversal (and gofer revalidation) locks when not required.
PiperOrigin-RevId: 698608924
Right now, entries are never removed from dirInoCache and if someone creates
and deletes directories in a loop, they observe memory leaks.
PiperOrigin-RevId: 698195540
This lock was introduced in cl/696713993 and only protects parent/name for all
dentries, which is all that's required by InotifyWithParent().
PiperOrigin-RevId: 697016824
- Add type parameter Filesystem to vfs/genericfstree, which is required to
provide `ancestryMu sync.RWMutex`, and add such a RWMutex to all FSImpls that
use genericfstree.
- Modify genericfstree.PrependPath() and genericfstree.IsDescendant() to use
ancestryMu to ensure atomicity. For callers of genericfstree.PrependPath(),
this means that (broader) FSImpl locks no longer need to be held during the
call. For callers of genericfstree.IsDescendant(), this means that we can
remove documentation warnings about its non-atomicity.
- Minor cleanup: Remove useless variable `start`, which is always 0, from
MM.ReadMaps/SmapsDataInto().
PiperOrigin-RevId: 696713993
This allows for external information to be passed to restore code.
Similar to c087777e37 ("Plumb restore context to afterLoad()").
Updates #1956.
PiperOrigin-RevId: 614125262
All other file systems don't support whiteouts and trusted.overlay attributes.
This restriction is applied only from mounts created from inside the sandbox.
We need this change to support applications such as Docker that is trying to
construct overlay mounts and falls back to other options if it fails.
PiperOrigin-RevId: 579343274
It is an idea of running codespell as part of our presubmit checks.
Before enabling it for new changes, let's fix what it has found.
Signed-off-by: Andrei Vagin <avagin@gmail.com>
IsDescendant gives the VFS layer an easy way to check if a dentry is a
descendant of another, which is important for some mounting procedures.
This method does not take any locks when accessing the parent, so parent
fields need to switch to be atomic to avoid data races.
PiperOrigin-RevId: 570454446
This is an optimization that will prevent needless lookups on lower layers for
directories. This is analogous to the Linux commit 97c684cc9110 ("ovl: create
directories inside merged parent opaque").
If mkdir(2) does succeed, then none of the lower layers have a file at that
position, otherwise mkdir(2) would have failed with EEXIST.
Changes on the lower layer underneath a new upper layer directory are not
visible. See the following for example:
```
$ sudo mount -t overlay -o lowerdir=lower,upperdir=upper,workdir=workdir none overlay
$ mkdir -p overlay/dir/dir1
$ mkdir -p lower/dir/dir2
$ ls overlay/dir/
dir1
```
Benchmarking shows that this change cuts ABSL build time by 9.5%!
```
goos: linux
goarch: amd64
cpu: Intel(R) Xeon(R) CPU @ 2.20GHz
│ /tmp/benchout.runsc │ /tmp/benchout.runsc-opt │
│ sec/op │ sec/op vs base │
BuildABSL/page_cache.clean/filesystem.bindfs-8 71.16 ± 12% 64.40 ± 3% -9.50% (p=0.003 n=8)
BuildGRPC/page_cache.clean/filesystem.bindfs-8 418.2 ± 1% 419.9 ± 1% ~ (p=0.721 n=8)
RubySpecTest/page_cache.clean/filesystem.bindfs-8 76.67 ± 5% 75.17 ± 1% ~ (p=0.382 n=8)
geomean 131.6 126.7 -3.78%
│ /tmp/benchout.runsc │ /tmp/benchout.runsc-opt │
│ load.sec │ load.sec vs base │
RubySpecTest/page_cache.clean/filesystem.bindfs-8 12.95 ± 6% 12.80 ± 5% ~ (p=0.625 n=8)
```
PiperOrigin-RevId: 538823897
This catches up the interface to the `EmitUnimplementedEvent` method signature
on `kernel.Kernel`.
Also add build-time test to verify that `kernel.Kernel` implements this
interface, in order to catch such breakages at build time in the future.
PiperOrigin-RevId: 519000411
All filesystems except tmpfs had the same bug regarding handling symlinks
in the terminal path component. When they tried to open(2) such a symlink
file, these implementations called into stepLocked() with
Then these implementations also called vfs.HandleSymlink() which would
also advance rp (when rp.Done() was already true).
This was tripping the invariants checks in sentry/vfs/debug.go.
PiperOrigin-RevId: 504098263
Linux overlayfs uses the max of all layers' filename length limit. Do the same
in gVisor.
This is needed to get PHP runtime test ext/standard/tests/strings/007.phpt to
pass with overlayfs.
PiperOrigin-RevId: 493942515
This CL does the following:
- Add the ability for nested locks to have names.
- Give names to all current uses of nested locks in the codebase.
- Truncate `lockdep` debug stack traces to avoid the clutter from the
`lockdep` code itself
- Simplify `lockdep` to not longer require `classMap`.
PiperOrigin-RevId: 491486620
Earlier, we were only copying up regular files. We were missing
out device files. ensureOpenableLocked() should ensure that
files are copied up if mode is writable. Without this, we
violate the preconditions for openCopiedUp().
Also, without this, open(file, O_WRONLY) on a lower layer pipe
returns EROFS, which is confusing because the upper layer is
still writable. Now it will return EPERM which indicates that
the pipe can not be copied up.
PiperOrigin-RevId: 465838606
Ayush RanjanandgVisor bot
Jamie LiuandgVisor bot
Andrei VaginandgVisor bot
Fabricio VoznikaandgVisor bot
Jing ChenandgVisor bot
Andrei Vagin
Lucas ManningandgVisor bot
Etienne PerotandgVisor bot
Adin ScannellandgVisor bot
Kevin KrakauerandgVisor bot