- Add type parameter Filesystem to vfs/genericfstree, which is required to
provide `ancestryMu sync.RWMutex`, and add such a RWMutex to all FSImpls that
use genericfstree.
- Modify genericfstree.PrependPath() and genericfstree.IsDescendant() to use
ancestryMu to ensure atomicity. For callers of genericfstree.PrependPath(),
this means that (broader) FSImpl locks no longer need to be held during the
call. For callers of genericfstree.IsDescendant(), this means that we can
remove documentation warnings about its non-atomicity.
- Minor cleanup: Remove useless variable `start`, which is always 0, from
MM.ReadMaps/SmapsDataInto().
PiperOrigin-RevId: 696713993
memmap.Mappable.Translate() is passed a hostarch.AccessType indicating what
permissions are *immediately* required; it returns permissions in
memmap.Translation.Perms that are granted *to MM* until invalidation. MM
ensures that the permissions granted to the application are the intersection of
those granted by Translate, and those granted by VMA permissions; see
determination of pma.effectivePerms in
mm.MemoryManager.getPMAsInternalLocked(). This mechanism is used to avoid
marking pages dirty in the sentry's page cache for gofer-backed files until
PROT_WRITE pages are actually written to; see gofer.dentry.Translate(). In most
other cases, granting all supported permissions (to MM) up-front avoids a
redundant page fault for pages that are touched first for reading, and later
for writing.
Also:
- Prevent PROT_WRITE mappings of erofs files at mmap()/mprotect() time, rather
than raising SIGBUS when writing to such mappings.
- Map nvproxy.frontendFD with PlatformEffectPopulate. This is the original goal
of this CL; however, before this rest of this CL, MM.MMap() =>
MM.populateVMAAndUnlock() => MM.getPMAsLocked(at=hostarch.NoAccess) =>
MM.getPMAsInternalLocked(at=hostarch.NoAccess) =>
nvproxy.frontendFD.Translate(at=hostarch.NoAccess) returns Translations with
no permissions, causing MM.mapASLocked() to no-op.
PiperOrigin-RevId: 679360594
MapInternal() returns a coherent memory mapping of the host file descriptor
represented by a memmap.File, in the sentry's address space. This is
principally used when the sentry needs to access the contents of application
memory (for e.g. syscall arguments passed by pointer, or the source/destination
of a write()/read() syscall); it usually looks up the memmap.Files backing
application addresses and obtains mappings via MapInternal().
/dev/nvidia-uvm cannot generally be mapped into the sentry's address space, for
reasons described by
https://github.com/google/gvisor/blob/master/g3doc/proposals/nvidia_driver_proxy.md#unified-virtual-memory-uvm
(in short, nvidia-uvm requires that a given page at file offset X can only be
mapped at address X). To allow the sentry to access the contents of such
mappings, make it possible for memmap.File.MapInternal() to indicate that a
fallback to buffered I/O is required, add interface methods
memmap.File.Buffer{Read,Write}At() to perform this buffered I/O, and implement
this fallback in the MM I/O path.
This CL does not use the new buffered I/O fallback anywhere; a following CL
adds it to nvproxy's nvidia-uvm.
Updates #10331
PiperOrigin-RevId: 629830825
FD numbers can vary between depending on the options used with
runsc command. For example, there are extra FDs passed to
`runsc boot` if `debug-log` is enabled. So instead of requiring
all FDs to have the exact same numbering during restore, provide
a mechanism to remap the FD. Each host FD has a unique identifier
with a map to their corresponding FD. Then during restore, FD
numbers are remapped to the correct ones.
Updates #1956
PiperOrigin-RevId: 615215783
This allows for external information to be passed to restore code.
Similar to c087777e37 ("Plumb restore context to afterLoad()").
Updates #1956.
PiperOrigin-RevId: 614125262
Tmpfs with file-backed are widely used:
1. Via --overlay2 flag. The default is root:self so the root mount uses this.
2. EmptyDir mounts with default medium are created as tmpfs with file backend.
This change unblocks (1) use case from being used with checkpoint/restore.
For (2), checkpoint/restore is not yet supported in multicontainers.
Most notably, this allows checkpoint/restore to work with default runsc flags.
PiperOrigin-RevId: 586291915
This patch adds the block based dirent lookup support to pkg/erofs,
which will do the dirent lookup by doing binary search on disk data
directly. This is helpful for searching files in large directories
and also reduces the memory overhead.
Signed-off-by: Tiwei Bie <tiwei.btw@antgroup.com>
- Add more sanity checks on dirent name offset and length.
- Both of "super block" and "superblock" are used in comments and
strings now, let's convert all "super block" to "superblock".
- Add some comments that can add clarity.
- Refactor the code in tests to make it easier to add more tests.
Signed-off-by: Tiwei Bie <tiwei.btw@antgroup.com>
This patch adds the EROFS rootfs support. No gofer process will be
created for the container, when there is no need to pass through host
files into a container via the gofer process. Annotations for rootfs
are also introduced to provide extra information, including the mount
source, mount type and overlay config. Additionally, busybox-static
is added to the default image and will be used to build the EROFS
rootfs images during the test.
Updates #8956
Signed-off-by: Tiwei Bie <tiwei.btw@antgroup.com>
For each directory, _all_ the on-disk directory entries are _strictly_
recorded in alphabetical order. There is no special treatment of "." and
"..". Thanks to @hsiangkao for pointing out this bug.
Fixes: eca83ac68c ("Add initial support for EROFS")
Reported-by: Gao Xiang <xiang@kernel.org>
Signed-off-by: Tiwei Bie <tiwei.btw@antgroup.com>
This patch adds initial support for EROFS [1]. Below is a brief
summary of the supported features.
Both inode formats are supported:
- compact format (32 bytes);
- extended format (64 bytes);
Below data layouts are supported:
- flat file data without data inline (no extent);
- flat file data with tail packing data inline (no extent);
Below file types are supported:
- directory;
- regular file;
- symlink;
Special files (e.g. fifo) can be listed, but cannot be accessed.
With this patch, sentry will be able to mount the EROFS image
created with below command and access the files on it.
mkfs.erofs -E noinline_data <IMAGE FILE> <SOURCE DIRECTORY>
[1] https://docs.kernel.org/filesystems/erofs.html
Updates #8956
Signed-off-by: Tiwei Bie <tiwei.btw@antgroup.com>
Jamie LiuandgVisor bot
Fabricio VoznikaandgVisor bot
Ayush RanjanandgVisor bot
Tiwei Bie