diff --git a/Documentation/admin-guide/binfmt-misc.rst b/Documentation/admin-guide/binfmt-misc.rst index c0a34fbf8022..3e2b2a9415bc 100644 --- a/Documentation/admin-guide/binfmt-misc.rst +++ b/Documentation/admin-guide/binfmt-misc.rst @@ -26,7 +26,8 @@ Here is what the fields mean: name below ``/proc/sys/fs/binfmt_misc``; cannot contain slashes ``/`` for obvious reasons. - ``type`` - is the type of recognition. Give ``M`` for magic and ``E`` for extension. + is the type of recognition. Give ``M`` for magic, ``E`` for extension and + ``B`` for a bpf-backed handler (see below). - ``offset`` is the offset of the magic/mask in the file, counted in bytes. This defaults to 0 if you omit it (i.e. you write ``:name:type::magic...``). @@ -48,7 +49,8 @@ Here is what the fields mean: filename extension matching. - ``interpreter`` is the program that should be invoked with the binary as first - argument (specify the full path) + argument (specify the full path). For ``B`` entries this field + carries the name of the bpf handler instead (see below). - ``flags`` is an optional field that controls several aspects of the invocation of the interpreter. It is a string of capital letters, each controls a @@ -88,6 +90,32 @@ Here is what the fields mean: emulation is installed and uses the opened image to spawn the emulator, meaning it is always available once installed, regardless of how the environment changes. + ``T`` - transparent + Run the interpreter transparently. The binary is handed to + the interpreter through ``AT_EXECFD`` (``T`` implies ``O``), + the argument vector is left exactly as the caller built it + and the kernel labels ``/proc/pid/exe`` with the binary + instead of the interpreter. The interpreter has to load the + binary from ``AT_EXECFD`` and follow the + ``AT_FLAGS_TRANSPARENT_INTERP`` contract. Combining ``T`` + with ``P`` is rejected: transparency preserves the whole + argument vector, argv[0] included. + ``L`` - loader substitution + Do not run the interpreter on the binary at all: load the + binary itself as a fully native exec and substitute the + interpreter for the loader named in the binary's + ``PT_INTERP``. See the "Loader substitution" section + below. ``L`` rejects ``T``, ``P``, ``O`` and ``C``; + ``F`` composes. + ``D`` - registered disabled + The entry is created disabled instead of being matchable at + once, and has to be enabled by writing ``1`` to its file + before it dispatches anything. This splits a registration + into creating the entry and activating it, leaving room to + configure it in between - which is what a ``B`` entry that + binds interpreters needs; see the bpf section below. The flag + is spent on the registration and is not read back: what an + entry file reports afterwards is whether it is enabled. There are some restrictions: @@ -96,6 +124,11 @@ There are some restrictions: - the magic must reside in the first 128 bytes of the file, i.e. offset+size(magic) has to be less than 128 - the interpreter string may not exceed 127 characters + - an interpreter used with ``C`` or ``L`` but without ``F`` has to be + named by an absolute path. It is opened when the binary is executed, so + a relative one would be resolved against the working directory of + whoever runs the binary + To use binfmt_misc you have to mount it first. You can mount it with ``mount -t binfmt_misc none /proc/sys/fs/binfmt_misc`` command, or you can add @@ -133,7 +166,209 @@ or 1 (to enable) to ``/proc/sys/fs/binfmt_misc/status`` or Catting the file tells you the current status of ``binfmt_misc/the_entry``. You can remove one entry or all entries by echoing -1 to ``/proc/.../the_name`` -or ``/proc/sys/fs/binfmt_misc/status``. +or ``/proc/sys/fs/binfmt_misc/status``. A single entry can also be removed +by simply unlinking (``rm``) ``/proc/.../the_name``. + + +bpf-backed handlers +------------------- + +With ``CONFIG_BINFMT_MISC_BPF`` both the matching and the interpreter +selection can be delegated to bpf programs. A handler is an instance of the +``binfmt_misc_ops`` struct_ops with a ``match`` and a ``load`` program and a +``name``. Once the struct_ops map is registered the handler can be activated +with a ``B`` entry that references it by name in the ``interpreter`` field +and carries neither offset, magic, nor mask:: + + echo ':qemu:B::::my_handler:' > register + +Both programs receive the ``linux_binprm`` of the binary and both can +sleep. The ``match`` program decides whether the handler applies: it is +consulted during the entry walk exactly like magic and extension matching, +in the same registration order with the same first-match-wins semantics. +Unlike static matching it is not limited to the prefetched first bytes of +the file in ``bprm->buf``: it can read the file, e.g. to parse ELF program +headers whose data sits at arbitrary offsets. It only decides, though: the +selection kfuncs below are rejected in it. The ``load`` program of the +matched handler then selects the interpreter: it can equally read the file +and derive the interpreter from the binary's location. It selects the +interpreter by calling the ``bpf_binprm_set_interp()`` kfunc with an +absolute path and returning ``0``. A match is committed: a failing +``load`` fails the exec with its error instead of falling through to later +entries; ``-ENOEXEC`` lets the remaining binary formats have a go. A path +selected this way is opened with the credentials of the task doing the +exec, exactly as a statically registered interpreter without ``F`` would +be. + +An entry can instead bind the interpreters its handler may use, so that no +path is resolved at exec time at all. An entry registered with ``D`` is not +matchable yet, which is what leaves it open to being given them, one +``+name path`` write at a time:: + + echo ':qemu:B::::my_handler:D' > register + echo '+aarch64 /usr/bin/qemu-aarch64' > qemu + echo '+arm /usr/bin/qemu-arm' > qemu + echo 1 > qemu + +Each path is opened during its write, in the writing process's context and +with the credentials the entry file was opened with, exactly the way ``F`` +pre-opens a static entry's interpreter; the paths must be absolute. The +path is everything past the first space, so there is nothing it cannot +express, and no interpreter has to fit in a register string. An entry +binds at most 100 interpreters; a write past that is refused with +``-ENOSPC``. To bind a file that has no path of its own - already +unlinked, a ``memfd``, or reachable only in another mount namespace - +open it and write ``/proc/self/fd/N``. + +The ``load`` program then selects one per exec by name with the +``bpf_binprm_select_interp()`` kfunc, and every exec runs a clone of the +file that was opened. The path decides which file is bound and nothing +else: it is not resolved again, in any namespace, so what it holds later - +or what it holds in the namespace of whoever runs the binary - no longer +decides anything. + +Enabling the entry ends this. Its interpreters are read at exec time with +nothing but a reference held on the entry, so an entry that has ever been +matchable can never have its set changed again: the first ``1`` seals it, +from then on ``+`` is refused with ``-EBUSY``, and an entry registered +without ``D`` is sealed from the start. Binding a name twice is refused +with ``-EEXIST``. + +Selection is by name so that the configuration and the program need not +agree on an order, and so that a handler is not tied to where a distribution +puts its interpreters. A name is a single word of printable ASCII, at most +32 characters; a name the entry did not bind gives the program ``-ENOENT``, +which it can act on or return. The interpreter runs under the path it was +registered under, and the entry reports what it bound:: + + $ cat /proc/sys/fs/binfmt_misc/qemu + enabled + bpf my_handler + bpf-interpreter aarch64 /usr/bin/qemu-aarch64 + bpf-interpreter arm /usr/bin/qemu-arm + flags: + +The path reported is the one the interpreter was bound under, which named +the file at that moment; it is not re-resolved, so it is a record of what +was bound rather than a promise about what that path holds now. + +The ``load`` program can also pass a single argument to the interpreter with +the ``bpf_binprm_set_interp_arg()`` kfunc. It is inserted between the +interpreter and the binary, exactly like the optional argument of a ``#!`` +interpreter line, e.g. for a handler that resolves ``$ORIGIN`` in a script's +``#!`` path and needs to preserve the argument that followed it. + +The invocation flags a static entry fixes at registration - ``P``, ``C``, +``O``, ``T`` and ``L`` - are per-exec choices for a bpf handler, made by the +``load`` program with the ``bpf_binprm_set_flags()`` kfunc, so a single +handler can decide them differently for each binary it handles: + +- ``BPF_BINPRM_PRESERVE_ARGV0`` keeps the caller's ``argv[0]`` (the ``P`` + flag). +- ``BPF_BINPRM_CREDENTIALS`` computes credentials from the binary (the ``C`` + flag), bounded to user namespaces that map the binary's owner just like + any other setuid exec. +- ``BPF_BINPRM_EXECFD`` opens the binary on the interpreter's behalf and + passes it through the ``AT_EXECFD`` aux vector entry (the ``O`` flag), so + the interpreter can run binaries it could not open by path. +- ``BPF_BINPRM_TRANSPARENT`` runs the interpreter transparently (the ``T`` + flag): the binary is handed over through ``AT_EXECFD`` as + with ``BPF_BINPRM_EXECFD``, but the argument vector is also left as the + caller passed it. An interpreter that loads the binary from ``AT_EXECFD`` + then appears in ``argv[0]`` and ``/proc/pid/cmdline`` as a direct + execution of the binary. ``BPF_BINPRM_PRESERVE_ARGV0`` and a staged + interpreter argument are rejected in combination with it, just as ``P`` + is with ``T``. It also lets a handler + run a binary passed as an inaccessible ``O_CLOEXEC`` file descriptor to + ``execveat()``, which a path-splicing dispatch cannot: the interpreter + has no path by which to open it. +- ``BPF_BINPRM_LOADER`` substitutes the interpreter for the binary's + ``PT_INTERP`` and runs the binary as a fully native exec (the ``L`` + flag). It excludes the other flags and a staged interpreter argument. + +Because these are program choices, a ``B`` entry carries no invocation +flags in the register string; ``F`` has none to spell for it either, since +the interpreters it binds already pre-open what ``F`` would. The +registration directive ``D`` is the exception: it decides how the entry +starts out, not how the interpreter is invoked. + +Handlers are looked up in the user namespace the struct_ops map was +registered in, falling back to ancestor namespaces, mirroring how +binfmt_misc instances themselves are looked up. The entry keeps the handler +alive; deleting the struct_ops map only prevents new activations. + + +Transparent interpreters +------------------------ + +With the ``T`` flag or ``BPF_BINPRM_TRANSPARENT`` the dispatch is invisible +to the resulting process. The argument vector is left exactly as the caller +built it. The binary is passed through ``AT_EXECFD``. The kernel also labels +``/proc/pid/exe`` correctly. The binary's file is write-denied while the +process runs and the interpreter's is not, exactly as if the binary had been +executed directly. A transparent entry does not change how credentials are +derived. As +with any other entry, set*id bits of the binary are only honored with ``C`` (or +``BPF_BINPRM_CREDENTIALS``). + +The interpreter has to be built for this contract. The kernel announces it +with ``AT_FLAGS_TRANSPARENT_INTERP`` in the ``AT_FLAGS`` aux vector entry +next to ``AT_EXECFD``. The argument vector belongs entirely to the program, +nothing was spliced in, so the interpreter doesn't consume arguments and +simply loads the program from the descriptor. The bit is also the loader's +license to finish the identity. After mapping the program it may retarget the +``AT_PHDR``/``AT_ENTRY``/``AT_BASE`` entries of ``/proc/pid/auxv`` and the +code/data statistics markers via one ``PR_SET_MM_MAP`` which completes +what attaching debuggers observe. What remains visibly different from a +direct execution is the address space layout. The interpreter occupies +the main-image position and the program lives in the mmap region. + + +Loader substitution +------------------- + +The ``L`` flag turns the execution model around. Instead of running the +registered interpreter with the binary as its payload the kernel loads +the matched binary itself as the main image and substitutes the registered +interpreter for the loader named in the binary's ``PT_INTERP``. + +Because the exec is native, there is no dispatch identity to +reconstruct and no contract the substitute has to implement. A stock +dynamic loader works unchanged. The argument vector is untouched, +credentials and ``AT_SECURE`` derive from the binary, there is no +``AT_EXECFD`` and no marker in the aux vector, the binary sits in the +main-image slot with the native brk placement so ``/proc/pid/maps``, +core dumps and perf mmap records have the native shape, and the +identity is already complete when ``PTRACE_EVENT_EXEC`` stops the +tracee. So launching under a debugger works, not just attaching. ``L`` +entries are for ELF binaries of a native architecture. Foreign-arch +emulation and non-ELF payloads remain the domain of the classic and +transparent modes. + +The override applies when the format that finally claims the file is +ELF with a ``PT_INTERP``. A matched binary without one or an +interpreter-less ``ET_DYN`` drops the override and runs natively. A file +claimed by another format - a ``#!`` script, say - is handled by that +format as if the entry had not matched. ``L`` is therefore not an +enforcement mechanism: it decides how a binary that asks for a loader is +run, it does not guarantee that everything matching the entry runs under +the substitute. A format that cannot consume the override at all instead +refuses the exec with ``ENOEXEC`` before the point of no return. + +A wrong-architecture ELF fails the whole exec with ``ENOEXEC`` exactly +as if no entry had matched. A substitute that is not ELF of the right +architecture fails with ``ELIBBAD``. The usual ``PT_INTERP`` sanity +checks on the binary still apply. But the segment's content is otherwise +irrelevant. + +``L`` rejects the classic-dispatch flags ``T``, ``P``, ``O`` and ``C`` +at registration. ``F`` composes and is valuable: with it the substitute +is opened at registration time, so later mount namespace or path changes +cannot redirect it. Without it the substitute is opened when the binary +is executed, and the path is resolved in the mount namespace and root of +whoever runs the binary, which is why it has to be absolute. As with +``C``, register only trusted interpreters. The substituted loader runs +with credentials derived from the binary. Hints diff --git a/Documentation/filesystems/failfs.rst b/Documentation/filesystems/failfs.rst new file mode 100644 index 000000000000..21ff2db7941d --- /dev/null +++ b/Documentation/filesystems/failfs.rst @@ -0,0 +1,73 @@ +.. SPDX-License-Identifier: GPL-2.0 + +====== +failfs +====== + +failfs is a kernel-internal filesystem that fails every operation +reaching it with ``EOPNOTSUPP``. It is the counterpart to nullfs. Where +nullfs is permanently empty, failfs means "nothing is supported here". +It cannot be mounted from userspace, nothing can be mounted on top of +it. It cannot be cloned. + +The only way into it is the ``FD_FAILFS_ROOT`` file descriptor sentinel which +is understood by ``fchdir(2)`` and ``fchroot(2)``. + +Semantics +========= + +Every path walk of a component through failfs fails with +``EOPNOTSUPP`` before that component is parsed, including ``.``. + +No path lookup can open the root, not even with ``O_PATH``. + +A process with its working directory in failfs fails every +``AT_FDCWD``-relative lookup. As with any working directory that is +unreachable from the process root, the ``getcwd(2)`` system call returns +a path prefixed with ``(unreachable)``. + +A process with its root directory in failfs fails every absolute path +lookup including absolute symlinks and the interpreter of dynamically +linked binaries. In other words, this fails exec. + +Lookups anchored at explicit directory file descriptors keep working. It +is the ``fs_struct`` equivalent of ``RESOLVE_BENEATH``. The process must +anchor every lookup at a file descriptor it explicitly holds. + +Entering +======== + +``fchroot(FD_FAILFS_ROOT, 0)`` requires ``CAP_SYS_CHROOT`` in the +caller's user namespace, mirroring ``chroot(2)``. Unprivileged callers +may enter if all of the following hold: + +* ``no_new_privs`` is set: setuid binaries on regular mounts remain + reachable via inherited directory file descriptors and executing them + with an unusable root directory is the classic confused deputy. + +* The caller is not already chrooted: the root directory is what + confines ``..`` resolution and the failfs root can never be reached by + walking up a real mount tree, so moving the root of a chrooted task to + failfs would allow it to escape its chroot via ``openat(fd, "..")``. + +* The caller does not share its ``fs_struct``: ``no_new_privs`` is + checked on the calling thread, but the root lives in the ``fs_struct``. + A ``CLONE_FS`` sibling without ``no_new_privs`` could otherwise execute + a setuid binary with the failfs root, so entry requires ``fs->users == + 1``, the same restriction ``setns(2)`` applies for the mount and user + namespaces. + +Leaving +======= + +Backing out is currently hard, but this is a property of the current +implementation, not a guaranteed interface, and may be loosened later. +For now a process that entered failfs counts as chrooted, so it cannot +create user namespaces to regain ``CAP_SYS_CHROOT``, and ``chroot(2)`` +or ``fchroot(2)`` back out require ``CAP_SYS_CHROOT``. The remaining way +out today is ``setns(2)`` with a mount namespace file descriptor, which +requires ``CAP_SYS_ADMIN`` over the target mount namespace as well as +``CAP_SYS_CHROOT`` and ``CAP_SYS_ADMIN`` in the caller's user namespace +and resets both root and working directory. A process that holds no such +file descriptor and restricts ``*chdir()``/``*chroot()``/``setns()`` via +seccomp cannot currently get back out. diff --git a/Documentation/filesystems/index.rst b/Documentation/filesystems/index.rst index 1f71cf159547..734a45e51667 100644 --- a/Documentation/filesystems/index.rst +++ b/Documentation/filesystems/index.rst @@ -91,6 +91,7 @@ Documentation for filesystem implementations. ext3 ext4/index f2fs + failfs gfs2/index hfs hfsplus diff --git a/Documentation/filesystems/locking.rst b/Documentation/filesystems/locking.rst index 08d01bc62c31..c274c5eef733 100644 --- a/Documentation/filesystems/locking.rst +++ b/Documentation/filesystems/locking.rst @@ -61,7 +61,7 @@ inode_operations prototypes:: - int (*create) (struct mnt_idmap *, struct inode *,struct dentry *,umode_t, bool); + int (*create) (struct mnt_idmap *, struct inode *,struct dentry *,umode_t); struct dentry * (*lookup) (struct inode *,struct dentry *, unsigned int); int (*link) (struct dentry *,struct inode *,struct dentry *); int (*unlink) (struct inode *,struct dentry *); diff --git a/Documentation/filesystems/overlayfs.rst b/Documentation/filesystems/overlayfs.rst index eb846518e6ac..1a29a5afabd7 100644 --- a/Documentation/filesystems/overlayfs.rst +++ b/Documentation/filesystems/overlayfs.rst @@ -347,6 +347,22 @@ The resulting access permissions should be the same. The difference is in the time of copy (on-demand vs. up-front). +Idmapped mounts +--------------- + +The overlay mount itself can be turned into an idmapped mount by applying an +idmapping to it with mount_setattr(2) and MOUNT_ATTR_IDMAP, just like for +other filesystems that support idmapped mounts. + +The mount idmapping only changes how ownership and permissions of the overlay +inodes are presented to and interpreted for the caller. It does not change +how overlayfs accesses the underlying layers: those are still accessed with +the stashed mounter's credentials through their own mounts, which may +themselves be idmapped. The overlay mount idmapping and any layer idmapping +compose, an underlying id is first mapped according to the relevant layer +idmapping and then according to the overlay mount idmapping. + + Multiple lower layers --------------------- diff --git a/Documentation/filesystems/porting.rst b/Documentation/filesystems/porting.rst index d13f0a23c882..60880eb0c49d 100644 --- a/Documentation/filesystems/porting.rst +++ b/Documentation/filesystems/porting.rst @@ -1173,7 +1173,7 @@ these conditions don't require explicit checks: - if LOOKUP_CREATE is NOT given, then the dentry won't be negative, ERR_PTR(-ENOENT) is returned instead - if LOOKUP_EXCL IS given, then the dentry won't be positive, - ERR_PTR(-EEXIST) is rreturned instread + ERR_PTR(-EEXIST) is returned instead LOOKUP_EXCL now means "target must not exist". It can be combined with LOOK_CREATE or LOOKUP_RENAME_TARGET. @@ -1401,3 +1401,11 @@ as with d_dispose_if_unused() these are not trivial; with this variant of API it's more explicit, since grabbing ->d_lock is caller-side, but d_dispose_if_unused() had all the same issues. It's a low-level primitive; use only if you have no alternative. + +--- + +**mandatory** + +The .create inode_operation no longer receives the 'excl' arg. It must +always assume the file does not already exist. If the filesystem needs +to be involved in non-exclusive create, it should provide atomic_open. diff --git a/Documentation/filesystems/vfs.rst b/Documentation/filesystems/vfs.rst index 7c753148af88..651b83b00440 100644 --- a/Documentation/filesystems/vfs.rst +++ b/Documentation/filesystems/vfs.rst @@ -415,7 +415,7 @@ As of kernel 2.6.22, the following members are defined: .. code-block:: c struct inode_operations { - int (*create) (struct mnt_idmap *, struct inode *,struct dentry *, umode_t, bool); + int (*create) (struct mnt_idmap *, struct inode *,struct dentry *, umode_t); struct dentry * (*lookup) (struct inode *,struct dentry *, unsigned int); int (*link) (struct dentry *,struct inode *,struct dentry *); int (*unlink) (struct inode *,struct dentry *); diff --git a/MAINTAINERS b/MAINTAINERS index be2affcd65bb..49ce8b7a90fc 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -9494,11 +9494,6 @@ L: linux-fbdev@vger.kernel.org S: Maintained F: drivers/video/fbdev/efifb.c -EFS FILESYSTEM -S: Orphan -W: http://aeschi.ch.eu.org/efs/ -F: fs/efs/ - EHEA (IBM pSeries eHEA 10Gb ethernet adapter) DRIVER L: netdev@vger.kernel.org S: Orphan diff --git a/arch/alpha/kernel/syscalls/syscall.tbl b/arch/alpha/kernel/syscalls/syscall.tbl index f31b7afffc34..52e3538cc7df 100644 --- a/arch/alpha/kernel/syscalls/syscall.tbl +++ b/arch/alpha/kernel/syscalls/syscall.tbl @@ -511,3 +511,4 @@ 579 common file_setattr sys_file_setattr 580 common listns sys_listns 581 common rseq_slice_yield sys_rseq_slice_yield +582 common fchroot sys_fchroot diff --git a/arch/arm/tools/syscall.tbl b/arch/arm/tools/syscall.tbl index 94351e22bfcf..55717ed32c27 100644 --- a/arch/arm/tools/syscall.tbl +++ b/arch/arm/tools/syscall.tbl @@ -486,3 +486,4 @@ 469 common file_setattr sys_file_setattr 470 common listns sys_listns 471 common rseq_slice_yield sys_rseq_slice_yield +472 common fchroot sys_fchroot diff --git a/arch/arm64/tools/syscall_32.tbl b/arch/arm64/tools/syscall_32.tbl index 62d93d88e0fe..df2d1d82fb3c 100644 --- a/arch/arm64/tools/syscall_32.tbl +++ b/arch/arm64/tools/syscall_32.tbl @@ -483,3 +483,4 @@ 469 common file_setattr sys_file_setattr 470 common listns sys_listns 471 common rseq_slice_yield sys_rseq_slice_yield +472 common fchroot sys_fchroot diff --git a/arch/m68k/kernel/syscalls/syscall.tbl b/arch/m68k/kernel/syscalls/syscall.tbl index 248934257101..ba7a4d8903d0 100644 --- a/arch/m68k/kernel/syscalls/syscall.tbl +++ b/arch/m68k/kernel/syscalls/syscall.tbl @@ -471,3 +471,4 @@ 469 common file_setattr sys_file_setattr 470 common listns sys_listns 471 common rseq_slice_yield sys_rseq_slice_yield +472 common fchroot sys_fchroot diff --git a/arch/microblaze/kernel/syscalls/syscall.tbl b/arch/microblaze/kernel/syscalls/syscall.tbl index 223d26303627..c55a5c96f49b 100644 --- a/arch/microblaze/kernel/syscalls/syscall.tbl +++ b/arch/microblaze/kernel/syscalls/syscall.tbl @@ -477,3 +477,4 @@ 469 common file_setattr sys_file_setattr 470 common listns sys_listns 471 common rseq_slice_yield sys_rseq_slice_yield +472 common fchroot sys_fchroot diff --git a/arch/mips/kernel/syscalls/syscall_n32.tbl b/arch/mips/kernel/syscalls/syscall_n32.tbl index 7430714e2b8f..9ae88e4eac61 100644 --- a/arch/mips/kernel/syscalls/syscall_n32.tbl +++ b/arch/mips/kernel/syscalls/syscall_n32.tbl @@ -410,3 +410,4 @@ 469 n32 file_setattr sys_file_setattr 470 n32 listns sys_listns 471 n32 rseq_slice_yield sys_rseq_slice_yield +472 n32 fchroot sys_fchroot diff --git a/arch/mips/kernel/syscalls/syscall_n64.tbl b/arch/mips/kernel/syscalls/syscall_n64.tbl index 630aab9e5425..83dc93a0712f 100644 --- a/arch/mips/kernel/syscalls/syscall_n64.tbl +++ b/arch/mips/kernel/syscalls/syscall_n64.tbl @@ -386,3 +386,4 @@ 469 n64 file_setattr sys_file_setattr 470 n64 listns sys_listns 471 n64 rseq_slice_yield sys_rseq_slice_yield +472 n64 fchroot sys_fchroot diff --git a/arch/mips/kernel/syscalls/syscall_o32.tbl b/arch/mips/kernel/syscalls/syscall_o32.tbl index 128653112284..9c62429c9b7b 100644 --- a/arch/mips/kernel/syscalls/syscall_o32.tbl +++ b/arch/mips/kernel/syscalls/syscall_o32.tbl @@ -459,3 +459,4 @@ 469 o32 file_setattr sys_file_setattr 470 o32 listns sys_listns 471 o32 rseq_slice_yield sys_rseq_slice_yield +472 o32 fchroot sys_fchroot diff --git a/arch/parisc/kernel/syscalls/syscall.tbl b/arch/parisc/kernel/syscalls/syscall.tbl index c6331dad9461..88adc4016cce 100644 --- a/arch/parisc/kernel/syscalls/syscall.tbl +++ b/arch/parisc/kernel/syscalls/syscall.tbl @@ -470,3 +470,4 @@ 469 common file_setattr sys_file_setattr 470 common listns sys_listns 471 common rseq_slice_yield sys_rseq_slice_yield +472 common fchroot sys_fchroot diff --git a/arch/powerpc/kernel/syscalls/syscall.tbl b/arch/powerpc/kernel/syscalls/syscall.tbl index 4fcc7c58a105..cfbb70039ff0 100644 --- a/arch/powerpc/kernel/syscalls/syscall.tbl +++ b/arch/powerpc/kernel/syscalls/syscall.tbl @@ -562,3 +562,4 @@ 469 common file_setattr sys_file_setattr 470 common listns sys_listns 471 nospu rseq_slice_yield sys_rseq_slice_yield +472 common fchroot sys_fchroot diff --git a/arch/s390/kernel/syscalls/syscall.tbl b/arch/s390/kernel/syscalls/syscall.tbl index 09a7ef04d979..1b45e68a217b 100644 --- a/arch/s390/kernel/syscalls/syscall.tbl +++ b/arch/s390/kernel/syscalls/syscall.tbl @@ -398,3 +398,4 @@ 469 common file_setattr sys_file_setattr 470 common listns sys_listns 471 common rseq_slice_yield sys_rseq_slice_yield +472 common fchroot sys_fchroot diff --git a/arch/sh/kernel/syscalls/syscall.tbl b/arch/sh/kernel/syscalls/syscall.tbl index 70b315cbe710..ace068dff0de 100644 --- a/arch/sh/kernel/syscalls/syscall.tbl +++ b/arch/sh/kernel/syscalls/syscall.tbl @@ -475,3 +475,4 @@ 469 common file_setattr sys_file_setattr 470 common listns sys_listns 471 common rseq_slice_yield sys_rseq_slice_yield +472 common fchroot sys_fchroot diff --git a/arch/sparc/kernel/syscalls/syscall.tbl b/arch/sparc/kernel/syscalls/syscall.tbl index 7e71bf7fcd14..5b9fe0e8140f 100644 --- a/arch/sparc/kernel/syscalls/syscall.tbl +++ b/arch/sparc/kernel/syscalls/syscall.tbl @@ -517,3 +517,4 @@ 469 common file_setattr sys_file_setattr 470 common listns sys_listns 471 common rseq_slice_yield sys_rseq_slice_yield +472 common fchroot sys_fchroot diff --git a/arch/x86/entry/syscalls/syscall_32.tbl b/arch/x86/entry/syscalls/syscall_32.tbl index f832ebd2d79b..2c172ef48dfd 100644 --- a/arch/x86/entry/syscalls/syscall_32.tbl +++ b/arch/x86/entry/syscalls/syscall_32.tbl @@ -477,3 +477,4 @@ 469 i386 file_setattr sys_file_setattr 470 i386 listns sys_listns 471 i386 rseq_slice_yield sys_rseq_slice_yield +472 i386 fchroot sys_fchroot diff --git a/arch/x86/entry/syscalls/syscall_64.tbl b/arch/x86/entry/syscalls/syscall_64.tbl index 524155d655da..d5b6045b0090 100644 --- a/arch/x86/entry/syscalls/syscall_64.tbl +++ b/arch/x86/entry/syscalls/syscall_64.tbl @@ -396,6 +396,7 @@ 469 common file_setattr sys_file_setattr 470 common listns sys_listns 471 common rseq_slice_yield sys_rseq_slice_yield +472 common fchroot sys_fchroot # # Due to a historical design error, certain syscalls are numbered differently diff --git a/arch/xtensa/kernel/syscalls/syscall.tbl b/arch/xtensa/kernel/syscalls/syscall.tbl index a9bca4e484de..d354bb231796 100644 --- a/arch/xtensa/kernel/syscalls/syscall.tbl +++ b/arch/xtensa/kernel/syscalls/syscall.tbl @@ -442,3 +442,4 @@ 469 common file_setattr sys_file_setattr 470 common listns sys_listns 471 common rseq_slice_yield sys_rseq_slice_yield +472 common fchroot sys_fchroot diff --git a/block/bdev.c b/block/bdev.c index 85ce57bd2ae4..797d7f0ef609 100644 --- a/block/bdev.c +++ b/block/bdev.c @@ -304,7 +304,12 @@ int bdev_freeze(struct block_device *bdev) mutex_lock(&bdev->bd_fsfreeze_mutex); - if (atomic_inc_return(&bdev->bd_fsfreeze_count) > 1) { + /* A device being removed from its filesystem refuses freezes. */ + if (!atomic_inc_unless_negative(&bdev->bd_fsfreeze_count)) { + mutex_unlock(&bdev->bd_fsfreeze_mutex); + return -EBUSY; + } + if (atomic_read(&bdev->bd_fsfreeze_count) > 1) { mutex_unlock(&bdev->bd_fsfreeze_mutex); return 0; } @@ -340,18 +345,18 @@ int bdev_thaw(struct block_device *bdev) mutex_lock(&bdev->bd_fsfreeze_mutex); - /* - * If this returns < 0 it means that @bd_fsfreeze_count was - * already 0 and no decrement was performed. - */ - nr_freeze = atomic_dec_if_positive(&bdev->bd_fsfreeze_count); - if (nr_freeze < 0) + /* <= 0: not frozen (0) or a freeze deny is held (< 0); leave it. */ + nr_freeze = atomic_read(&bdev->bd_fsfreeze_count); + if (nr_freeze <= 0) goto out; error = 0; - if (nr_freeze > 0) + if (nr_freeze > 1) { + atomic_dec(&bdev->bd_fsfreeze_count); goto out; + } + /* Keep the count positive across the thaw so a deny is refused. */ mutex_lock(&bdev->bd_holder_lock); if (bdev->bd_holder_ops && bdev->bd_holder_ops->thaw) { error = bdev->bd_holder_ops->thaw(bdev); @@ -360,14 +365,52 @@ int bdev_thaw(struct block_device *bdev) mutex_unlock(&bdev->bd_holder_lock); } - if (error) - atomic_inc(&bdev->bd_fsfreeze_count); + if (!error) + atomic_dec(&bdev->bd_fsfreeze_count); out: mutex_unlock(&bdev->bd_fsfreeze_mutex); return error; } EXPORT_SYMBOL(bdev_thaw); +/** + * bdev_deny_freeze - make a block device unfreezable + * @bdev: block device + * + * Reserve @bdev against bdev_freeze() the way deny_write_access() reserves a + * file against writers. bd_fsfreeze_count is sign-encoded: > 0 counts active + * freezes, < 0 counts deniers, so a deny succeeds only while no freeze is in + * progress. While held, bdev_freeze() returns -EBUSY. Pair with + * bdev_allow_freeze(). + * + * A filesystem removing, adding or replacing a member device denies freezes on + * it for the duration, so a claim a freeze walk might act on is never torn down + * behind the freezer's back. The deny is device-scoped, not (device, + * superblock)-scoped: a device shared by several superblocks is refused for all + * of them. No in-tree filesystem removes a shared claim from a live superblock. + * + * Return: 0, or -EBUSY if the device is currently frozen. + */ +int bdev_deny_freeze(struct block_device *bdev) +{ + return atomic_dec_unless_positive(&bdev->bd_fsfreeze_count) ? 0 : -EBUSY; +} +EXPORT_SYMBOL_GPL(bdev_deny_freeze); + +/** + * bdev_allow_freeze - allow freezing a block device again + * @bdev: block device + * + * Undo one bdev_deny_freeze(). + */ +void bdev_allow_freeze(struct block_device *bdev) +{ + /* A deny must be held, i.e. the count must be negative. */ + WARN_ON_ONCE(atomic_read(&bdev->bd_fsfreeze_count) >= 0); + atomic_inc(&bdev->bd_fsfreeze_count); +} +EXPORT_SYMBOL_GPL(bdev_allow_freeze); + /* * pseudo-fs */ @@ -1152,6 +1195,39 @@ put_no_open: blkdev_put_no_open(bdev); } +/** + * bdev_yield_claim - give up the holder claim on an open block device + * @bdev_file: open block device + * + * Yield the holder and any write access for @bdev_file without closing it, so + * the caller can still act on the device - e.g. bdev_allow_freeze() it - before + * the final bdev_fput(). bdev_fput() yields too, so calling it afterwards is + * safe. + */ +void bdev_yield_claim(struct file *bdev_file) +{ + struct block_device *bdev; + struct gendisk *disk; + + if (!bdev_file->private_data) + return; + + bdev = file_bdev(bdev_file); + disk = bdev->bd_disk; + + mutex_lock(&disk->open_mutex); + bdev_yield_write_access(bdev_file); + bd_yield_claim(bdev_file); + /* + * Tell release we already gave up our hold on the + * device and if write restrictions are available that + * we already gave up write access to the device. + */ + bdev_file->private_data = BDEV_I(bdev_file->f_mapping->host); + mutex_unlock(&disk->open_mutex); +} +EXPORT_SYMBOL_GPL(bdev_yield_claim); + /** * bdev_fput - yield claim to the block device and put the file * @bdev_file: open block device @@ -1165,22 +1241,7 @@ void bdev_fput(struct file *bdev_file) if (WARN_ON_ONCE(bdev_file->f_op != &def_blk_fops)) return; - if (bdev_file->private_data) { - struct block_device *bdev = file_bdev(bdev_file); - struct gendisk *disk = bdev->bd_disk; - - mutex_lock(&disk->open_mutex); - bdev_yield_write_access(bdev_file); - bd_yield_claim(bdev_file); - /* - * Tell release we already gave up our hold on the - * device and if write restrictions are available that - * we already gave up write access to the device. - */ - bdev_file->private_data = BDEV_I(bdev_file->f_mapping->host); - mutex_unlock(&disk->open_mutex); - } - + bdev_yield_claim(bdev_file); fput(bdev_file); } EXPORT_SYMBOL(bdev_fput); @@ -1217,6 +1278,18 @@ int lookup_bdev(const char *pathname, dev_t *dev) if (!may_open_dev(&path)) goto out_path_put; + /* + * Reject a block device inode with i_rdev == 0. A dev_t of 0 is + * never valid for a block device: no real block device driver + * registers major 0. Fake block device inodes (e.g. fuse with + * rootmode=S_IFBLK) can expose i_rdev == 0, and letting that + * propagate would confuse superblock lookup and trigger warnings + * in the device-to-superblock table (super_dev_register). + */ + error = -ENODEV; + if (!inode->i_rdev) + goto out_path_put; + *dev = inode->i_rdev; error = 0; out_path_put: diff --git a/block/fops.c b/block/fops.c index 15783a6180de..4cff76e9eb71 100644 --- a/block/fops.c +++ b/block/fops.c @@ -453,8 +453,10 @@ static int blkdev_iomap_begin(struct inode *inode, loff_t offset, loff_t length, return 0; } +static DEFINE_IOMAP_ITER_NEXT(blkdev_iomap_next, blkdev_iomap_begin); + static const struct iomap_ops blkdev_iomap_ops = { - .iomap_begin = blkdev_iomap_begin, + .iomap_next = blkdev_iomap_next, }; #ifdef CONFIG_BUFFER_HEAD diff --git a/drivers/base/devtmpfs.c b/drivers/base/devtmpfs.c index b1c4ceb65026..aef0fcc6aba1 100644 --- a/drivers/base/devtmpfs.c +++ b/drivers/base/devtmpfs.c @@ -413,7 +413,7 @@ static noinline int __init devtmpfs_setup(void *p) { int err; - err = ksys_unshare(CLONE_NEWNS); + err = ksys_unshare(UNSHARE_EMPTY_MNTNS); if (err) goto out; err = init_mount("devtmpfs", "/", "devtmpfs", DEVTMPFS_MFLAGS, NULL); diff --git a/drivers/block/rnbd/rnbd-srv.c b/drivers/block/rnbd/rnbd-srv.c index 10e8c438bb43..79c9a5fb418f 100644 --- a/drivers/block/rnbd/rnbd-srv.c +++ b/drivers/block/rnbd/rnbd-srv.c @@ -11,6 +11,7 @@ #include #include +#include #include "rnbd-srv.h" #include "rnbd-srv-trace.h" @@ -734,7 +735,8 @@ static int process_msg_open(struct rnbd_srv_session *srv_sess, goto reject; } - bdev_file = bdev_file_open_by_path(full_path, open_flags, NULL, NULL); + scoped_with_init_fs() + bdev_file = bdev_file_open_by_path(full_path, open_flags, NULL, NULL); if (IS_ERR(bdev_file)) { ret = PTR_ERR(bdev_file); pr_err("Opening device '%s' on session %s failed, failed to open the block device, err: %pe\n", diff --git a/drivers/char/misc_minor_kunit.c b/drivers/char/misc_minor_kunit.c index e930c78e1ef9..e85210cbb640 100644 --- a/drivers/char/misc_minor_kunit.c +++ b/drivers/char/misc_minor_kunit.c @@ -5,6 +5,7 @@ #include #include #include +#include #include /* static minor (LCD_MINOR) */ @@ -160,18 +161,22 @@ static void __init miscdev_test_can_open(struct kunit *test, struct miscdevice * char *devname; devname = kasprintf(GFP_KERNEL, "/dev/%s", misc->name); - ret = init_mknod(devname, S_IFCHR | 0600, - new_encode_dev(MKDEV(MISC_MAJOR, misc->minor))); - if (ret != 0) - KUNIT_FAIL(test, "failed to create node\n"); - filp = filp_open(devname, O_RDONLY, 0); - if (IS_ERR(filp)) - KUNIT_FAIL(test, "failed to open misc device: %ld\n", PTR_ERR(filp)); - else - fput(filp); + /* Tests run in a nullfs kthread; borrow the init fs to resolve /dev. */ + scoped_with_init_fs() { + ret = init_mknod(devname, S_IFCHR | 0600, + new_encode_dev(MKDEV(MISC_MAJOR, misc->minor))); + if (ret != 0) + KUNIT_FAIL(test, "failed to create node\n"); - init_unlink(devname); + filp = filp_open(devname, O_RDONLY, 0); + if (IS_ERR(filp)) + KUNIT_FAIL(test, "failed to open misc device: %ld\n", PTR_ERR(filp)); + else + fput(filp); + + init_unlink(devname); + } kfree(devname); } diff --git a/drivers/crypto/ccp/sev-dev.c b/drivers/crypto/ccp/sev-dev.c index ca473ca198b8..0f72352030ba 100644 --- a/drivers/crypto/ccp/sev-dev.c +++ b/drivers/crypto/ccp/sev-dev.c @@ -260,20 +260,16 @@ static int sev_cmd_buffer_len(int cmd) static struct file *open_file_as_root(const char *filename, int flags, umode_t mode) { - struct path root __free(path_put) = {}; - - task_lock(&init_task); - get_fs_root(init_task.fs, &root); - task_unlock(&init_task); - CLASS(prepare_creds, cred)(); if (!cred) return ERR_PTR(-ENOMEM); cred->fsuid = GLOBAL_ROOT_UID; - scoped_with_creds(cred) - return file_open_root(&root, filename, flags, mode); + scoped_with_init_fs() { + scoped_with_creds(cred) + return filp_open(filename, flags, mode); + } } static int sev_read_init_ex_file(void) diff --git a/drivers/target/target_core_alua.c b/drivers/target/target_core_alua.c index 10250aca5a81..140154d93c43 100644 --- a/drivers/target/target_core_alua.c +++ b/drivers/target/target_core_alua.c @@ -18,6 +18,8 @@ #include #include #include +#include +#include #include #include @@ -856,10 +858,17 @@ static int core_alua_write_tpg_metadata( unsigned char *md_buf, u32 md_buf_len) { - struct file *file = filp_open(path, O_RDWR | O_CREAT | O_TRUNC, 0600); + struct file *file; loff_t pos = 0; int ret; + if (tsk_is_kthread(current)) { + scoped_with_init_fs() + file = filp_open(path, O_RDWR | O_CREAT | O_TRUNC, 0600); + } else { + file = filp_open(path, O_RDWR | O_CREAT | O_TRUNC, 0600); + } + if (IS_ERR(file)) { pr_err("filp_open(%s) for ALUA metadata failed\n", path); return -ENODEV; diff --git a/drivers/target/target_core_pr.c b/drivers/target/target_core_pr.c index 1a77b4bb62b0..25b1bcacc0c8 100644 --- a/drivers/target/target_core_pr.c +++ b/drivers/target/target_core_pr.c @@ -18,6 +18,7 @@ #include #include #include +#include #include #include @@ -1969,7 +1970,8 @@ static int __core_scsi3_write_aptpl_to_file( if (!path) return -ENOMEM; - file = filp_open(path, flags, 0600); + scoped_with_init_fs() + file = filp_open(path, flags, 0600); if (IS_ERR(file)) { pr_err("filp_open(%s) for APTPL metadata" " failed\n", path); diff --git a/fs/9p/vfs_inode.c b/fs/9p/vfs_inode.c index 5783d0336f96..3829554ca369 100644 --- a/fs/9p/vfs_inode.c +++ b/fs/9p/vfs_inode.c @@ -645,7 +645,6 @@ error: * @dir: The parent directory * @dentry: The name of file to be created * @mode: The UNIX file mode to set - * @excl: True if the file must not yet exist * * open(.., O_CREAT) is handled in v9fs_vfs_atomic_open(). This is only called * for mknod(2). @@ -654,7 +653,7 @@ error: static int v9fs_vfs_create(struct mnt_idmap *idmap, struct inode *dir, - struct dentry *dentry, umode_t mode, bool excl) + struct dentry *dentry, umode_t mode) { struct v9fs_session_info *v9ses = v9fs_inode2v9ses(dir); u32 perm = unixmode2p9mode(v9ses, mode); @@ -689,7 +688,7 @@ static struct dentry *v9fs_vfs_mkdir(struct mnt_idmap *idmap, struct inode *dir, p9_debug(P9_DEBUG_VFS, "name %pd\n", dentry); v9ses = v9fs_inode2v9ses(dir); - perm = unixmode2p9mode(v9ses, mode | S_IFDIR); + perm = unixmode2p9mode(v9ses, mode); fid = v9fs_create(v9ses, dir, dentry, NULL, perm, P9_OREAD); if (IS_ERR(fid)) return ERR_CAST(fid); diff --git a/fs/9p/vfs_inode_dotl.c b/fs/9p/vfs_inode_dotl.c index f7396d20cb6c..116b29e95f21 100644 --- a/fs/9p/vfs_inode_dotl.c +++ b/fs/9p/vfs_inode_dotl.c @@ -213,12 +213,11 @@ int v9fs_open_to_dotl_flags(int flags) * @dir: directory inode that is being created * @dentry: dentry that is being deleted * @omode: create permissions - * @excl: True if the file must not yet exist * */ static int v9fs_vfs_create_dotl(struct mnt_idmap *idmap, struct inode *dir, - struct dentry *dentry, umode_t omode, bool excl) + struct dentry *dentry, umode_t omode) { return v9fs_vfs_mknod_dotl(idmap, dir, dentry, omode, 0); } @@ -362,7 +361,6 @@ static struct dentry *v9fs_vfs_mkdir_dotl(struct mnt_idmap *idmap, p9_debug(P9_DEBUG_VFS, "name %pd\n", dentry); v9ses = v9fs_inode2v9ses(dir); - omode |= S_IFDIR; if (dir->i_mode & S_ISGID) omode |= S_ISGID; diff --git a/fs/Kconfig b/fs/Kconfig index cf6ae64776e6..64ff193fb42c 100644 --- a/fs/Kconfig +++ b/fs/Kconfig @@ -315,7 +315,6 @@ source "fs/hfs/Kconfig" source "fs/hfsplus/Kconfig" source "fs/befs/Kconfig" source "fs/bfs/Kconfig" -source "fs/efs/Kconfig" source "fs/jffs2/Kconfig" # UBIFS File system configuration source "fs/ubifs/Kconfig" diff --git a/fs/Kconfig.binfmt b/fs/Kconfig.binfmt index 1949e25c7741..daeac4889d03 100644 --- a/fs/Kconfig.binfmt +++ b/fs/Kconfig.binfmt @@ -168,6 +168,20 @@ config BINFMT_MISC you have use for it; the module is called binfmt_misc. If you don't know what to answer at this point, say Y. +config BINFMT_MISC_BPF + bool "BPF-selected interpreters for misc binaries" + depends on BINFMT_MISC=y + depends on BPF_SYSCALL && BPF_JIT && DEBUG_INFO_BTF + help + Allow binfmt_misc binary type handlers to be implemented as bpf + struct_ops programs. Instead of matching a fixed magic and + redirecting to a fixed interpreter recorded at registration time + such handlers match binaries programmatically and compute the + interpreter to use per binary, e.g. relative to the location of + the binary itself. + + If you don't know what to answer at this point, say N. + config COREDUMP bool "Enable core dump support" if EXPERT default y diff --git a/fs/Makefile b/fs/Makefile index 89a8a9d207d1..5d3cd2bdcb9c 100644 --- a/fs/Makefile +++ b/fs/Makefile @@ -16,7 +16,7 @@ obj-y := open.o read_write.o file_table.o super.o \ stack.o fs_struct.o statfs.o fs_pin.o nsfs.o \ fs_dirent.o fs_context.o fs_parser.o fsopen.o init.o \ kernel_read_file.o mnt_idmapping.o remap_range.o pidfs.o \ - file_attr.o fserror.o nullfs.o + file_attr.o fserror.o nullfs.o failfs.o obj-$(CONFIG_BUFFER_HEAD) += buffer.o mpage.o obj-$(CONFIG_PROC_FS) += proc_namespace.o @@ -33,6 +33,7 @@ obj-$(CONFIG_FS_ENCRYPTION) += crypto/ obj-$(CONFIG_FS_VERITY) += verity/ obj-$(CONFIG_FILE_LOCKING) += locks.o obj-$(CONFIG_BINFMT_MISC) += binfmt_misc.o +obj-$(CONFIG_BINFMT_MISC_BPF) += binfmt_misc_bpf.o obj-$(CONFIG_BINFMT_SCRIPT) += binfmt_script.o obj-$(CONFIG_BINFMT_ELF) += binfmt_elf.o obj-$(CONFIG_COMPAT_BINFMT_ELF) += compat_binfmt_elf.o @@ -92,7 +93,6 @@ obj-$(CONFIG_HPFS_FS) += hpfs/ obj-$(CONFIG_NTFS_FS) += ntfs/ obj-$(CONFIG_NTFS3_FS) += ntfs3/ obj-$(CONFIG_UFS_FS) += ufs/ -obj-$(CONFIG_EFS_FS) += efs/ obj-$(CONFIG_JFFS2_FS) += jffs2/ obj-$(CONFIG_UBIFS_FS) += ubifs/ obj-$(CONFIG_AFFS_FS) += affs/ diff --git a/fs/affs/affs.h b/fs/affs/affs.h index 44a3f69d275f..d1c506c1f310 100644 --- a/fs/affs/affs.h +++ b/fs/affs/affs.h @@ -44,7 +44,6 @@ struct affs_inode_info { struct mutex i_link_lock; /* Protects internal inode access. */ struct mutex i_ext_lock; /* Protects internal inode access. */ #define i_hash_lock i_ext_lock - struct mapping_metadata_bhs i_metadata_bhs; u32 i_blkcnt; /* block count */ u32 i_extcnt; /* extended block count */ u32 *i_lc; /* linear cache of extended blocks */ @@ -152,7 +151,6 @@ extern bool affs_nofilenametruncate(const struct dentry *dentry); extern int affs_check_name(const unsigned char *name, int len, bool notruncate); extern int affs_copy_name(unsigned char *bstr, struct dentry *dentry); -struct mapping_metadata_bhs *affs_get_metadata_bhs(struct inode *inode); /* bitmap. c */ @@ -169,7 +167,7 @@ extern int affs_hash_name(struct super_block *sb, const u8 *name, unsigned int l extern struct dentry *affs_lookup(struct inode *dir, struct dentry *dentry, unsigned int); extern int affs_unlink(struct inode *dir, struct dentry *dentry); extern int affs_create(struct mnt_idmap *idmap, struct inode *dir, - struct dentry *dentry, umode_t mode, bool); + struct dentry *dentry, umode_t mode); extern struct dentry *affs_mkdir(struct mnt_idmap *idmap, struct inode *dir, struct dentry *dentry, umode_t mode); extern int affs_rmdir(struct inode *dir, struct dentry *dentry); diff --git a/fs/affs/amigaffs.c b/fs/affs/amigaffs.c index bed4fc805e8e..6cc0fc9a4cbf 100644 --- a/fs/affs/amigaffs.c +++ b/fs/affs/amigaffs.c @@ -57,7 +57,7 @@ affs_insert_hash(struct inode *dir, struct buffer_head *bh) AFFS_TAIL(sb, dir_bh)->hash_chain = cpu_to_be32(ino); affs_adjust_checksum(dir_bh, ino); - mmb_mark_buffer_dirty(dir_bh, &AFFS_I(dir)->i_metadata_bhs); + mark_buffer_dirty(dir_bh); affs_brelse(dir_bh); inode_set_mtime_to_ts(dir, inode_set_ctime_current(dir)); @@ -100,7 +100,7 @@ affs_remove_hash(struct inode *dir, struct buffer_head *rem_bh) else AFFS_TAIL(sb, bh)->hash_chain = ino; affs_adjust_checksum(bh, be32_to_cpu(ino) - hash_ino); - mmb_mark_buffer_dirty(bh, &AFFS_I(dir)->i_metadata_bhs); + mark_buffer_dirty(bh); AFFS_TAIL(sb, rem_bh)->parent = 0; retval = 0; break; @@ -180,7 +180,7 @@ affs_remove_link(struct dentry *dentry) affs_unlock_dir(dir); goto done; } - mmb_mark_buffer_dirty(link_bh, &AFFS_I(inode)->i_metadata_bhs); + mark_buffer_dirty(link_bh); memcpy(AFFS_TAIL(sb, bh)->name, AFFS_TAIL(sb, link_bh)->name, 32); retval = affs_insert_hash(dir, bh); @@ -188,7 +188,7 @@ affs_remove_link(struct dentry *dentry) affs_unlock_dir(dir); goto done; } - mmb_mark_buffer_dirty(bh, &AFFS_I(inode)->i_metadata_bhs); + mark_buffer_dirty(bh); affs_unlock_dir(dir); iput(dir); @@ -203,7 +203,7 @@ affs_remove_link(struct dentry *dentry) __be32 ino2 = AFFS_TAIL(sb, link_bh)->link_chain; AFFS_TAIL(sb, bh)->link_chain = ino2; affs_adjust_checksum(bh, be32_to_cpu(ino2) - link_ino); - mmb_mark_buffer_dirty(bh, &AFFS_I(inode)->i_metadata_bhs); + mark_buffer_dirty(bh); retval = 0; /* Fix the link count, if bh is a normal header block without links */ switch (be32_to_cpu(AFFS_TAIL(sb, bh)->stype)) { @@ -306,7 +306,7 @@ affs_remove_header(struct dentry *dentry) retval = affs_remove_hash(dir, bh); if (retval) goto done_unlock; - mmb_mark_buffer_dirty(bh, &AFFS_I(inode)->i_metadata_bhs); + mark_buffer_dirty(bh); affs_unlock_dir(dir); diff --git a/fs/affs/file.c b/fs/affs/file.c index 144b17482d12..23e088a7ed4f 100644 --- a/fs/affs/file.c +++ b/fs/affs/file.c @@ -140,14 +140,14 @@ affs_alloc_extblock(struct inode *inode, struct buffer_head *bh, u32 ext) AFFS_TAIL(sb, new_bh)->parent = cpu_to_be32(inode->i_ino); affs_fix_checksum(sb, new_bh); - mmb_mark_buffer_dirty(new_bh, &AFFS_I(inode)->i_metadata_bhs); + mark_buffer_dirty(new_bh); tmp = be32_to_cpu(AFFS_TAIL(sb, bh)->extension); if (tmp) affs_warning(sb, "alloc_ext", "previous extension set (%x)", tmp); AFFS_TAIL(sb, bh)->extension = cpu_to_be32(blocknr); affs_adjust_checksum(bh, blocknr - tmp); - mmb_mark_buffer_dirty(bh, &AFFS_I(inode)->i_metadata_bhs); + mark_buffer_dirty(bh); AFFS_I(inode)->i_extcnt++; mark_inode_dirty(inode); @@ -581,7 +581,7 @@ affs_extent_file_ofs(struct inode *inode, u32 newsize) memset(AFFS_DATA(bh) + boff, 0, tmp); be32_add_cpu(&AFFS_DATA_HEAD(bh)->size, tmp); affs_fix_checksum(sb, bh); - mmb_mark_buffer_dirty(bh, &AFFS_I(inode)->i_metadata_bhs); + mark_buffer_dirty(bh); size += tmp; bidx++; } else if (bidx) { @@ -603,7 +603,7 @@ affs_extent_file_ofs(struct inode *inode, u32 newsize) AFFS_DATA_HEAD(bh)->size = cpu_to_be32(tmp); affs_fix_checksum(sb, bh); bh->b_state &= ~(1UL << BH_New); - mmb_mark_buffer_dirty(bh, &AFFS_I(inode)->i_metadata_bhs); + mark_buffer_dirty(bh); if (prev_bh) { u32 tmp_next = be32_to_cpu(AFFS_DATA_HEAD(prev_bh)->next); @@ -613,8 +613,7 @@ affs_extent_file_ofs(struct inode *inode, u32 newsize) bidx, tmp_next); AFFS_DATA_HEAD(prev_bh)->next = cpu_to_be32(bh->b_blocknr); affs_adjust_checksum(prev_bh, bh->b_blocknr - tmp_next); - mmb_mark_buffer_dirty(prev_bh, - &AFFS_I(inode)->i_metadata_bhs); + mark_buffer_dirty(prev_bh); affs_brelse(prev_bh); } size += bsize; @@ -733,7 +732,7 @@ static int affs_write_end_ofs(const struct kiocb *iocb, AFFS_DATA_HEAD(bh)->size = cpu_to_be32( max(boff + tmp, be32_to_cpu(AFFS_DATA_HEAD(bh)->size))); affs_fix_checksum(sb, bh); - mmb_mark_buffer_dirty(bh, &AFFS_I(inode)->i_metadata_bhs); + mark_buffer_dirty(bh); written += tmp; from += tmp; bidx++; @@ -766,13 +765,12 @@ static int affs_write_end_ofs(const struct kiocb *iocb, bidx, tmp_next); AFFS_DATA_HEAD(prev_bh)->next = cpu_to_be32(bh->b_blocknr); affs_adjust_checksum(prev_bh, bh->b_blocknr - tmp_next); - mmb_mark_buffer_dirty(prev_bh, - &AFFS_I(inode)->i_metadata_bhs); + mark_buffer_dirty(prev_bh); } } affs_brelse(prev_bh); affs_fix_checksum(sb, bh); - mmb_mark_buffer_dirty(bh, &AFFS_I(inode)->i_metadata_bhs); + mark_buffer_dirty(bh); written += bsize; from += bsize; bidx++; @@ -801,14 +799,13 @@ static int affs_write_end_ofs(const struct kiocb *iocb, bidx, tmp_next); AFFS_DATA_HEAD(prev_bh)->next = cpu_to_be32(bh->b_blocknr); affs_adjust_checksum(prev_bh, bh->b_blocknr - tmp_next); - mmb_mark_buffer_dirty(prev_bh, - &AFFS_I(inode)->i_metadata_bhs); + mark_buffer_dirty(prev_bh); } } else if (be32_to_cpu(AFFS_DATA_HEAD(bh)->size) < tmp) AFFS_DATA_HEAD(bh)->size = cpu_to_be32(tmp); affs_brelse(prev_bh); affs_fix_checksum(sb, bh); - mmb_mark_buffer_dirty(bh, &AFFS_I(inode)->i_metadata_bhs); + mark_buffer_dirty(bh); written += tmp; from += tmp; bidx++; @@ -945,7 +942,7 @@ affs_truncate(struct inode *inode) } AFFS_TAIL(sb, ext_bh)->extension = 0; affs_fix_checksum(sb, ext_bh); - mmb_mark_buffer_dirty(ext_bh, &AFFS_I(inode)->i_metadata_bhs); + mark_buffer_dirty(ext_bh); affs_brelse(ext_bh); if (inode->i_size) { diff --git a/fs/affs/inode.c b/fs/affs/inode.c index 5dd1b016bcb0..d4a3f381c4bc 100644 --- a/fs/affs/inode.c +++ b/fs/affs/inode.c @@ -206,7 +206,7 @@ affs_write_inode(struct inode *inode, struct writeback_control *wbc) } } affs_fix_checksum(sb, bh); - mmb_mark_buffer_dirty(bh, &AFFS_I(inode)->i_metadata_bhs); + mark_buffer_dirty(bh); affs_brelse(bh); affs_free_prealloc(inode); return 0; @@ -266,11 +266,8 @@ affs_evict_inode(struct inode *inode) if (!inode->i_nlink) { inode->i_size = 0; affs_truncate(inode); - } else { - mmb_sync(&AFFS_I(inode)->i_metadata_bhs); } - mmb_invalidate(&AFFS_I(inode)->i_metadata_bhs); clear_inode(inode); affs_free_prealloc(inode); cache_page = (unsigned long)AFFS_I(inode)->i_lc; @@ -305,7 +302,7 @@ affs_new_inode(struct inode *dir) bh = affs_getzeroblk(sb, block); if (!bh) goto err_bh; - mmb_mark_buffer_dirty(bh, &AFFS_I(inode)->i_metadata_bhs); + mark_buffer_dirty(bh); affs_brelse(bh); inode->i_uid = current_fsuid(); @@ -393,17 +390,17 @@ affs_add_entry(struct inode *dir, struct inode *inode, struct dentry *dentry, s3 AFFS_TAIL(sb, bh)->link_chain = chain; AFFS_TAIL(sb, inode_bh)->link_chain = cpu_to_be32(block); affs_adjust_checksum(inode_bh, block - be32_to_cpu(chain)); - mmb_mark_buffer_dirty(inode_bh, &AFFS_I(inode)->i_metadata_bhs); + mark_buffer_dirty(inode_bh); set_nlink(inode, 2); ihold(inode); } affs_fix_checksum(sb, bh); - mmb_mark_buffer_dirty(bh, &AFFS_I(inode)->i_metadata_bhs); + mark_buffer_dirty(bh); dentry->d_fsdata = (void *)(long)bh->b_blocknr; affs_lock_dir(dir); retval = affs_insert_hash(dir, bh); - mmb_mark_buffer_dirty(bh, &AFFS_I(inode)->i_metadata_bhs); + mark_buffer_dirty(bh); affs_unlock_dir(dir); affs_unlock_link(inode); diff --git a/fs/affs/namei.c b/fs/affs/namei.c index c3c6532da4b0..6cb52efafe5f 100644 --- a/fs/affs/namei.c +++ b/fs/affs/namei.c @@ -243,7 +243,7 @@ affs_unlink(struct inode *dir, struct dentry *dentry) int affs_create(struct mnt_idmap *idmap, struct inode *dir, - struct dentry *dentry, umode_t mode, bool excl) + struct dentry *dentry, umode_t mode) { struct super_block *sb = dir->i_sb; struct inode *inode; @@ -287,7 +287,7 @@ affs_mkdir(struct mnt_idmap *idmap, struct inode *dir, if (!inode) return ERR_PTR(-ENOSPC); - inode->i_mode = S_IFDIR | mode; + inode->i_mode = mode; affs_mode_to_prot(inode); inode->i_op = &affs_dir_inode_operations; @@ -373,7 +373,7 @@ affs_symlink(struct mnt_idmap *idmap, struct inode *dir, } *p = 0; inode->i_size = i + 1; - mmb_mark_buffer_dirty(bh, &AFFS_I(inode)->i_metadata_bhs); + mark_buffer_dirty(bh); affs_brelse(bh); mark_inode_dirty(inode); @@ -443,8 +443,7 @@ affs_rename(struct inode *old_dir, struct dentry *old_dentry, /* TODO: move it back to old_dir, if error? */ done: - mmb_mark_buffer_dirty(bh, - &AFFS_I(retval ? old_dir : new_dir)->i_metadata_bhs); + mark_buffer_dirty(bh); affs_brelse(bh); return retval; } @@ -497,8 +496,8 @@ affs_xrename(struct inode *old_dir, struct dentry *old_dentry, retval = affs_insert_hash(old_dir, bh_new); affs_unlock_dir(old_dir); done: - mmb_mark_buffer_dirty(bh_old, &AFFS_I(new_dir)->i_metadata_bhs); - mmb_mark_buffer_dirty(bh_new, &AFFS_I(old_dir)->i_metadata_bhs); + mark_buffer_dirty(bh_old); + mark_buffer_dirty(bh_new); affs_brelse(bh_old); affs_brelse(bh_new); return retval; diff --git a/fs/affs/super.c b/fs/affs/super.c index b232251aa7bb..ed8225b3c93d 100644 --- a/fs/affs/super.c +++ b/fs/affs/super.c @@ -88,7 +88,7 @@ void affs_mark_sb_dirty(struct super_block *sb) spin_lock(&sbi->work_lock); if (!sbi->work_queued) { delay = msecs_to_jiffies(dirty_writeback_interval * 10); - queue_delayed_work(system_long_wq, &sbi->sb_work, delay); + queue_delayed_work(system_dfl_long_wq, &sbi->sb_work, delay); sbi->work_queued = 1; } spin_unlock(&sbi->work_lock); @@ -108,7 +108,6 @@ static struct inode *affs_alloc_inode(struct super_block *sb) i->i_lc = NULL; i->i_ext_bh = NULL; i->i_pa_cnt = 0; - mmb_init(&i->i_metadata_bhs, &i->vfs_inode.i_data); return &i->vfs_inode; } diff --git a/fs/afs/dir.c b/fs/afs/dir.c index 6df56fe9163f..81565366d937 100644 --- a/fs/afs/dir.c +++ b/fs/afs/dir.c @@ -34,7 +34,7 @@ static bool afs_lookup_filldir(struct dir_context *ctx, const char *name, int nl u64 ino, u32 uniquifier); #define AFS_LOOKUP ((filldir_t)0x137UL) static int afs_create(struct mnt_idmap *idmap, struct inode *dir, - struct dentry *dentry, umode_t mode, bool excl); + struct dentry *dentry, umode_t mode); static struct dentry *afs_mkdir(struct mnt_idmap *idmap, struct inode *dir, struct dentry *dentry, umode_t mode); static int afs_rmdir(struct inode *dir, struct dentry *dentry); @@ -1333,7 +1333,7 @@ static struct dentry *afs_mkdir(struct mnt_idmap *idmap, struct inode *dir, op->file[0].modification = true; op->file[0].update_ctime = true; op->dentry = dentry; - op->create.mode = S_IFDIR | mode; + op->create.mode = mode; op->create.reason = afs_edit_dir_for_mkdir; op->mtime = current_time(dir); op->ops = &afs_mkdir_operation; @@ -1633,7 +1633,7 @@ static const struct afs_operation_ops afs_create_operation = { * create a regular file on an AFS filesystem */ static int afs_create(struct mnt_idmap *idmap, struct inode *dir, - struct dentry *dentry, umode_t mode, bool excl) + struct dentry *dentry, umode_t mode) { struct afs_operation *op; struct afs_vnode *dvnode = AFS_FS_I(dir); diff --git a/fs/autofs/root.c b/fs/autofs/root.c index 186e960f1e23..b36439f4521e 100644 --- a/fs/autofs/root.c +++ b/fs/autofs/root.c @@ -741,7 +741,7 @@ static struct dentry *autofs_dir_mkdir(struct mnt_idmap *idmap, autofs_del_active(dentry); - inode = autofs_get_inode(dir->i_sb, S_IFDIR | mode); + inode = autofs_get_inode(dir->i_sb, mode); if (!inode) return ERR_PTR(-ENOMEM); diff --git a/fs/bad_inode.c b/fs/bad_inode.c index acf8613f5e36..486c40f73e51 100644 --- a/fs/bad_inode.c +++ b/fs/bad_inode.c @@ -29,7 +29,7 @@ static const struct file_operations bad_file_ops = static int bad_inode_create(struct mnt_idmap *idmap, struct inode *dir, struct dentry *dentry, - umode_t mode, bool excl) + umode_t mode) { return -EIO; } diff --git a/fs/bfs/dir.c b/fs/bfs/dir.c index 5b40ab09a796..91a4871fa051 100644 --- a/fs/bfs/dir.c +++ b/fs/bfs/dir.c @@ -68,22 +68,15 @@ static int bfs_readdir(struct file *f, struct dir_context *ctx) return 0; } -static int bfs_fsync(struct file *file, loff_t start, loff_t end, int datasync) -{ - return mmb_fsync(file, - &BFS_I(file->f_mapping->host)->i_metadata_bhs, - start, end, datasync); -} - const struct file_operations bfs_dir_operations = { .read = generic_read_dir, .iterate_shared = bfs_readdir, - .fsync = bfs_fsync, + .fsync = simple_fsync, .llseek = generic_file_llseek, }; static int bfs_create(struct mnt_idmap *idmap, struct inode *dir, - struct dentry *dentry, umode_t mode, bool excl) + struct dentry *dentry, umode_t mode) { int err; struct inode *inode; diff --git a/fs/bfs/inode.c b/fs/bfs/inode.c index e41efdd35db9..06e3a848b4ef 100644 --- a/fs/bfs/inode.c +++ b/fs/bfs/inode.c @@ -136,7 +136,6 @@ static int bfs_write_inode(struct inode *inode, struct writeback_control *wbc) unsigned long i_sblock; struct bfs_inode *di; struct buffer_head *bh; - int err = 0; dprintf("ino=%08x\n", ino); @@ -165,13 +164,31 @@ static int bfs_write_inode(struct inode *inode, struct writeback_control *wbc) di->i_eoffset = cpu_to_le32(i_sblock * BFS_BSIZE + inode->i_size - 1); mark_buffer_dirty(bh); - if (wbc->sync_mode == WB_SYNC_ALL) { - sync_dirty_buffer(bh); - if (buffer_req(bh) && !buffer_uptodate(bh)) - err = -EIO; - } brelse(bh); mutex_unlock(&info->bfs_lock); + set_inode_metadata_writeback(inode); + return 0; +} + +static int bfs_sync_inode_metadata(struct inode *inode, + struct writeback_control *wbc) +{ + int err = 0; + struct bfs_inode *di; + struct buffer_head *bh; + + di = find_inode(inode->i_sb, (u16)inode->i_ino, &bh); + if (IS_ERR(di)) + return PTR_ERR(di); + + sync_dirty_buffer(bh); + if (buffer_write_io_error(bh)) { + err = -EIO; + goto out; + } + err = mmb_sync(&BFS_I(inode)->i_metadata_bhs); +out: + brelse(bh); return err; } @@ -302,6 +319,7 @@ static const struct super_operations bfs_sops = { .alloc_inode = bfs_alloc_inode, .free_inode = bfs_free_inode, .write_inode = bfs_write_inode, + .sync_inode_metadata = bfs_sync_inode_metadata, .evict_inode = bfs_evict_inode, .put_super = bfs_put_super, .statfs = bfs_statfs, diff --git a/fs/binfmt_elf.c b/fs/binfmt_elf.c index 16a56b6b3f6c..00ff35cad441 100644 --- a/fs/binfmt_elf.c +++ b/fs/binfmt_elf.c @@ -179,7 +179,6 @@ create_elf_tables(struct linux_binprm *bprm, const struct elfhdr *exec, unsigned char k_rand_bytes[16]; int items; elf_addr_t *elf_info; - elf_addr_t flags = 0; int ei_index; const struct cred *cred = current_cred(); struct vm_area_struct *vma; @@ -254,9 +253,7 @@ create_elf_tables(struct linux_binprm *bprm, const struct elfhdr *exec, NEW_AUX_ENT(AT_PHENT, sizeof(struct elf_phdr)); NEW_AUX_ENT(AT_PHNUM, exec->e_phnum); NEW_AUX_ENT(AT_BASE, interp_load_addr); - if (bprm->interp_flags & BINPRM_FLAGS_PRESERVE_ARGV0) - flags |= AT_FLAGS_PRESERVE_ARGV0; - NEW_AUX_ENT(AT_FLAGS, flags); + NEW_AUX_ENT(AT_FLAGS, bprm_at_flags(bprm)); NEW_AUX_ENT(AT_ENTRY, e_entry); NEW_AUX_ENT(AT_UID, from_kuid_munged(cred->user_ns, cred->uid)); NEW_AUX_ENT(AT_EUID, from_kuid_munged(cred->user_ns, cred->euid)); @@ -904,7 +901,7 @@ static int load_elf_binary(struct linux_binprm *bprm) if (elf_interpreter[elf_ppnt->p_filesz - 1] != '\0') goto out_free_interp; - interpreter = open_exec(elf_interpreter); + interpreter = bprm_open_interpreter(bprm, elf_interpreter); kfree(elf_interpreter); retval = PTR_ERR(interpreter); if (IS_ERR(interpreter)) @@ -935,6 +932,9 @@ out_free_interp: goto out_free_ph; } + /* No PT_INTERP to substitute for: the override does not apply. */ + bprm_drop_loader(bprm); + elf_ppnt = elf_phdata; for (i = 0; i < elf_ex->e_phnum; i++, elf_ppnt++) switch (elf_ppnt->p_type) { diff --git a/fs/binfmt_elf_fdpic.c b/fs/binfmt_elf_fdpic.c index fe0b5c5ed2bc..068c46875c74 100644 --- a/fs/binfmt_elf_fdpic.c +++ b/fs/binfmt_elf_fdpic.c @@ -263,7 +263,8 @@ static int load_elf_fdpic_binary(struct linux_binprm *bprm) kdebug("Using ELF interpreter %s", interpreter_name); /* replace the program with the interpreter */ - interpreter = open_exec(interpreter_name); + interpreter = bprm_open_interpreter(bprm, + interpreter_name); retval = PTR_ERR(interpreter); if (IS_ERR(interpreter)) { interpreter = NULL; @@ -299,6 +300,9 @@ static int load_elf_fdpic_binary(struct linux_binprm *bprm) } + /* No PT_INTERP to substitute for: the override does not apply. */ + bprm_drop_loader(bprm); + if (is_constdisp(&exec_params.hdr)) exec_params.flags |= ELF_FDPIC_FLAG_CONSTDISP; @@ -509,7 +513,6 @@ static int create_elf_fdpic_tables(struct linux_binprm *bprm, char *k_platform, *k_base_platform; char __user *u_platform, *u_base_platform, *p; int loop; - unsigned long flags = 0; int ei_index; elf_addr_t *elf_info; @@ -649,9 +652,7 @@ static int create_elf_fdpic_tables(struct linux_binprm *bprm, NEW_AUX_ENT(AT_PHENT, sizeof(struct elf_phdr)); NEW_AUX_ENT(AT_PHNUM, exec_params->hdr.e_phnum); NEW_AUX_ENT(AT_BASE, interp_params->elfhdr_addr); - if (bprm->interp_flags & BINPRM_FLAGS_PRESERVE_ARGV0) - flags |= AT_FLAGS_PRESERVE_ARGV0; - NEW_AUX_ENT(AT_FLAGS, flags); + NEW_AUX_ENT(AT_FLAGS, bprm_at_flags(bprm)); NEW_AUX_ENT(AT_ENTRY, exec_params->entry_addr); NEW_AUX_ENT(AT_UID, (elf_addr_t) from_kuid_munged(cred->user_ns, cred->uid)); NEW_AUX_ENT(AT_EUID, (elf_addr_t) from_kuid_munged(cred->user_ns, cred->euid)); diff --git a/fs/binfmt_misc.c b/fs/binfmt_misc.c index c97f10b48b5b..ad8c4f64bf10 100644 --- a/fs/binfmt_misc.c +++ b/fs/binfmt_misc.c @@ -10,45 +10,98 @@ #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt -#include -#include -#include -#include -#include -#include +#include +#include #include -#include +#include +#include +#include +#include +#include #include -#include #include -#include -#include -#include -#include -#include #include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include #include +#include #include "internal.h" -#ifdef DEBUG -# define USE_DEBUG 1 -#else -# define USE_DEBUG 0 -#endif - -enum { - VERBOSE_STATUS = 1 /* make it zero to save 400 bytes kernel memory */ +/* Entry status and match type bit numbers. */ +enum binfmt_misc_entry_bits { + MISC_FMT_ENABLED_BIT = 0, + MISC_FMT_MAGIC_BIT = 1, + MISC_FMT_BPF_BIT = 2, }; -enum {Enabled, Magic}; -#define MISC_FMT_PRESERVE_ARGV0 (1UL << 31) -#define MISC_FMT_OPEN_BINARY (1UL << 30) -#define MISC_FMT_CREDENTIALS (1UL << 29) -#define MISC_FMT_OPEN_FILE (1UL << 28) +/* Entry behavior flags, fixed at registration time. */ +enum binfmt_misc_entry_flags { + MISC_FMT_PRESERVE_ARGV0 = (1U << 31), + MISC_FMT_OPEN_BINARY = (1U << 30), + MISC_FMT_CREDENTIALS = (1U << 29), + MISC_FMT_OPEN_FILE = (1U << 28), + MISC_FMT_TRANSPARENT = (1U << 27), + MISC_FMT_LOADER = (1U << 26), + MISC_FMT_DISABLED = (1U << 25), +}; -typedef struct { - struct list_head list; +/* The flags that shape the invocation; a 'B' handler picks those per exec. */ +#define MISC_FMT_INVOCATION_FLAGS (MISC_FMT_PRESERVE_ARGV0 | \ + MISC_FMT_OPEN_BINARY | \ + MISC_FMT_CREDENTIALS | \ + MISC_FMT_OPEN_FILE | \ + MISC_FMT_TRANSPARENT | \ + MISC_FMT_LOADER) + +/** + * struct binfmt_misc_flag - a flag character of the register string + * @c: the character userspace writes and reads back + * @flag: the entry flag it sets + * @implies: entry flags it turns on in addition + * @desc: what it does, for the registration debug output + */ +struct binfmt_misc_flag { + char c; + unsigned long flag; + unsigned long implies; + const char *desc; +}; + +static const struct binfmt_misc_flag misc_flags[] = { + { 'P', MISC_FMT_PRESERVE_ARGV0, 0, "preserve argv0" }, + { 'O', MISC_FMT_OPEN_BINARY, 0, "open binary" }, + { 'C', MISC_FMT_CREDENTIALS, MISC_FMT_OPEN_BINARY, "credentials from the binary" }, + { 'F', MISC_FMT_OPEN_FILE, 0, "open interpreter file now" }, + { 'T', MISC_FMT_TRANSPARENT, MISC_FMT_OPEN_BINARY, "transparent" }, + { 'L', MISC_FMT_LOADER, 0, "loader substitution" }, + { 'D', MISC_FMT_DISABLED, 0, "register disabled" }, +}; + +/* Look up a flag character, NULL if @c is not one. */ +static const struct binfmt_misc_flag *misc_flag_by_char(const char c) +{ + for (int i = 0; i < ARRAY_SIZE(misc_flags); i++) + if (misc_flags[i].c == c) + return &misc_flags[i]; + return NULL; +} + +struct binfmt_misc_entry { + struct hlist_node node; unsigned long flags; /* type, status, etc. */ int offset; /* offset of magic */ int size; /* size of magic/mask */ @@ -57,11 +110,13 @@ typedef struct { const char *interpreter; /* filename of interpreter */ char *name; struct dentry *dentry; - struct file *interp_file; + const struct binfmt_misc_ops *bpf_ops; /* bpf-backed handler ('B') */ + const char *bpf_ops_name; + struct list_head interps; /* the interpreters it bound */ refcount_t users; /* sync removal with load_misc_binary() */ -} Node; - -static struct file_system_type bm_fs_type; + struct rcu_head rcu; + char buf[]; /* register string, fields point in here */ +}; /* * Max length of the register string. Determined by: @@ -74,54 +129,88 @@ static struct file_system_type bm_fs_type; * - interp: ~50 bytes * - flags: 5 bytes * Round that up a bit, and then back off to hold the internal data - * (like struct Node). + * (like struct binfmt_misc_entry). */ #define MAX_REGISTER_LENGTH 1920 +/* Trailing delimiter pad so field parsing always terminates at a delimiter. */ +#define MISC_DELIM_PAD 8 + +/* Protects the entry walk in load_misc_binary(), which may sleep in it. */ +DEFINE_STATIC_SRCU_FAST(bm_entries_srcu); + +/* Check if @e's magic matches @bprm's buffer, applying the mask if set. */ +static bool entry_matches_magic(const struct binfmt_misc_entry *e, + const struct linux_binprm *bprm) +{ + const char *s = bprm->buf + e->offset; + int i; + + if (!e->mask) + return !memcmp(s, e->magic, e->size); + + for (i = 0; i < e->size; i++) + if ((s[i] ^ e->magic[i]) & e->mask[i]) + return false; + return true; +} + +/* Check if @e's registered extension matches @ext, NULL if there is none. */ +static bool entry_matches_extension(const struct binfmt_misc_entry *e, + const char *ext) +{ + return ext && !strcmp(e->magic, ext); +} + /** * search_binfmt_handler - search for a binary handler for @bprm * @misc: handle to binfmt_misc instance * @bprm: binary for which we are looking for a handler * * Search for a binary type handler for @bprm in the list of registered binary - * type handlers. + * type handlers. A 'B' entry's match program decides whether the handler + * applies; it may sleep to read the binary. The matched entry is returned + * with a reference taken while the walk still held it; a dying entry - + * unlinked with its last reference gone - cannot be matched and the walk + * moves on. * - * Return: binary type list entry on success, NULL on failure + * The caller must hold the bm_entries_srcu read lock, which allows an + * entry's evaluation to sleep. + * + * Return: referenced binary type list entry on success, NULL on failure */ -static Node *search_binfmt_handler(struct binfmt_misc *misc, - struct linux_binprm *bprm) +static struct binfmt_misc_entry * +search_binfmt_handler(struct binfmt_misc *misc, struct linux_binprm *bprm) { - char *p = strrchr(bprm->interp, '.'); - Node *e; + char *dot = strrchr(bprm->interp, '.'); + const char *ext = dot ? dot + 1 : NULL; + struct binfmt_misc_entry *e; /* Walk all the registered handlers. */ - list_for_each_entry(e, &misc->entries, list) { - char *s; - int j; - - /* Make sure this one is currently enabled. */ - if (!test_bit(Enabled, &e->flags)) + hlist_for_each_entry_rcu(e, &misc->entries, node, + srcu_read_lock_held(&bm_entries_srcu)) { + /* + * Make sure this one is currently enabled. An entry enters + * the list at most once and only whole: its configuration is + * ordered before the rcu insertion that makes it visible + * here. + */ + if (!test_bit(MISC_FMT_ENABLED_BIT, &e->flags)) continue; - /* Do matching based on extension if applicable. */ - if (!test_bit(Magic, &e->flags)) { - if (p && !strcmp(e->magic, p + 1)) - return e; - continue; - } - - /* Do matching based on magic & mask. */ - s = bprm->buf + e->offset; - if (e->mask) { - for (j = 0; j < e->size; j++) - if ((*s++ ^ e->magic[j]) & e->mask[j]) - break; + if (test_bit(MISC_FMT_BPF_BIT, &e->flags)) { + if (!e->bpf_ops->match(bprm)) + continue; + } else if (test_bit(MISC_FMT_MAGIC_BIT, &e->flags)) { + if (!entry_matches_magic(e, bprm)) + continue; } else { - for (j = 0; j < e->size; j++) - if ((*s++ ^ e->magic[j])) - break; + if (!entry_matches_extension(e, ext)) + continue; } - if (j == e->size) + + /* A dying entry cannot be matched, walk on. */ + if (refcount_inc_not_zero(&e->users)) return e; } @@ -133,165 +222,471 @@ static Node *search_binfmt_handler(struct binfmt_misc *misc, * @misc: handle to binfmt_misc instance * @bprm: binary for which we are looking for a handler * - * Try to find a binfmt handler for the binary type. If one is found take a - * reference to protect against removal via bm_{entry,status}_write(). + * Try to find a binfmt handler for the binary type. If one is found it is + * returned with a reference protecting it against removal via + * bm_{entry,status}_write(). * * Return: binary type list entry on success, NULL on failure */ -static Node *get_binfmt_handler(struct binfmt_misc *misc, - struct linux_binprm *bprm) +static struct binfmt_misc_entry *get_binfmt_handler(struct binfmt_misc *misc, + struct linux_binprm *bprm) { - Node *e; - - read_lock(&misc->entries_lock); - e = search_binfmt_handler(misc, bprm); - if (e) - refcount_inc(&e->users); - read_unlock(&misc->entries_lock); - return e; + guard(srcu_fast)(&bm_entries_srcu); + return search_binfmt_handler(misc, bprm); } /** - * put_binfmt_handler - put binary handler node - * @e: node to put + * binfmt_misc_find_interp - find a bound interpreter by name + * @interps: the interpreters the matched entry was registered with + * @name: the name to look for * - * Free node syncing with load_misc_binary() and defer final free to - * load_misc_binary() in case it is using the binary type handler we were - * requested to remove. + * Return: the interpreter on success, NULL if @interps has none by that name */ -static void put_binfmt_handler(Node *e) +const struct binfmt_misc_interp * +binfmt_misc_find_interp(const struct list_head *interps, const char *name) { - if (refcount_dec_and_test(&e->users)) { - if (e->flags & MISC_FMT_OPEN_FILE) { - exe_file_allow_write_access(e->interp_file); - filp_close(e->interp_file, NULL); - } - kfree(e); + struct binfmt_misc_interp *interp; + + list_for_each_entry(interp, interps, list) + if (!strcmp(interp->name, name)) + return interp; + return NULL; +} + +/* Undo the open_exec() a pre-opened interpreter file came from. */ +static void close_interp_file(struct file *f) +{ + if (IS_ERR_OR_NULL(f)) + return; + exe_file_allow_write_access(f); + filp_close(f, NULL); +} + +DEFINE_FREE(close_interp_file, struct file *, close_interp_file(_T)) + +/* + * Open an interpreter @path for execution: now, in the writer's context, + * and - since binfmt_misc mounts can be unprivileged - with @cred, the + * credentials the control file being written was opened with, not the + * writer's own. + */ +static struct file *open_interp_file(const struct cred *cred, const char *path) +{ + struct file *f; + + scoped_with_creds(cred) + f = open_exec(path); + if (IS_ERR(f)) + pr_notice("register: failed to install interpreter %s\n", path); + return f; +} + +/* Release the interpreters an entry was registered with. */ +static void entry_put_interpreters(struct binfmt_misc_entry *e) +{ + struct binfmt_misc_interp *interp, *tmp; + + list_for_each_entry_safe(interp, tmp, &e->interps, list) { + list_del(&interp->list); + close_interp_file(interp->file); + kfree(interp); } } /** - * load_binfmt_misc - load the binfmt_misc of the caller's user namespace + * entry_attach_interpreter - bind an opened interpreter to @e + * @e: entry being configured + * @name: name the load program will select it by; empty for the fixed + * interpreter of a static entry + * @path: the path @f was opened from + * @f: the interpreter, opened for execution * - * To be called in load_misc_binary() to load the relevant struct binfmt_misc. - * If a user namespace doesn't have its own binfmt_misc mount it can make use - * of its ancestor's binfmt_misc handlers. This mimicks the behavior of - * pre-namespaced binfmt_misc where all registered binfmt_misc handlers where - * available to all user and user namespaces on the system. + * Every exec runs a clone of @f, so the path decided which file is bound + * and nothing else: it is not resolved again, in any namespace. + * + * The caller has to have validated @name and @path, established that @e + * cannot be matched yet, and owns @f until this succeeds. + * + * Return: 0 on success, a negative errno on failure + */ +static int entry_attach_interpreter(struct binfmt_misc_entry *e, + const char *name, const char *path, + struct file *f) +{ + size_t nlen = strlen(name), plen = strlen(path); + struct binfmt_misc_interp *interp; + + if (binfmt_misc_find_interp(&e->interps, name)) + return -EEXIST; + if (list_count_nodes(&e->interps) >= BINFMT_MISC_INTERP_MAX) + return -ENOSPC; + + /* One allocation, both strings in it, like the entry's own buffer. */ + interp = kmalloc(struct_size(interp, name, nlen + plen + 2), + GFP_KERNEL_ACCOUNT); + if (!interp) + return -ENOMEM; + + interp->path = interp->name + nlen + 1; + strscpy(interp->name, name, nlen + 1); + strscpy(interp->name + nlen + 1, path, plen + 1); + interp->file = f; + /* Publish the node: a lockless cat may be walking the list. */ + list_add_tail_rcu(&interp->list, &e->interps); + pr_debug("register: interpreter: %s {%s}\n", name, path); + return 0; +} + +static void bm_entry_free_rcu(struct rcu_head *rcu) +{ + struct binfmt_misc_entry *e = container_of(rcu, struct binfmt_misc_entry, rcu); + + /* No walker that could sleep in the handler's programs is left. */ + if (e->bpf_ops) + binfmt_misc_put_ops(e->bpf_ops); + kfree(e); +} + +/** + * put_binfmt_handler - put binary handler entry + * @e: entry to put + * + * Free entry syncing with load_misc_binary() and defer final free to + * load_misc_binary() in case it is using the binary type handler we were + * requested to remove. Also the teardown for a registration that fails + * before add_entry() publishes the entry. + */ +static void put_binfmt_handler(struct binfmt_misc_entry *e) +{ + if (IS_ERR_OR_NULL(e)) + return; + + if (refcount_dec_and_test(&e->users)) { + entry_put_interpreters(e); + /* Walkers may still dereference this entry, even sleeping. */ + call_srcu(&bm_entries_srcu, &e->rcu, bm_entry_free_rcu); + } +} + +DEFINE_FREE(put_binfmt_handler, struct binfmt_misc_entry *, put_binfmt_handler(_T)) + +/* Drop everything a load program staged for this exec. */ +static void drop_staged_selection(struct linux_binprm *bprm) +{ + kfree(bprm->bpf_interp); + bprm->bpf_interp = NULL; + kfree(bprm->bpf_interp_arg); + bprm->bpf_interp_arg = NULL; + if (bprm->bpf_interp_file) { + fput(bprm->bpf_interp_file); + bprm->bpf_interp_file = NULL; + } + bprm->bpf_flags = 0; +} + +/** + * current_binfmt_misc - get the binfmt_misc instance of the caller's user namespace + * + * If a user namespace doesn't have its own binfmt_misc mount it uses the + * handlers of its closest ancestor with one. This mimics the behavior of + * pre-namespaced binfmt_misc where all registered handlers were available + * to all users and user namespaces on the system. The init user namespace + * instance is statically set up so the fallback is never reached in + * practice. * * Return: the binfmt_misc instance of the caller's user namespace */ -static struct binfmt_misc *load_binfmt_misc(void) +static struct binfmt_misc *current_binfmt_misc(void) { const struct user_namespace *user_ns; struct binfmt_misc *misc; - user_ns = current_user_ns(); - while (user_ns) { + for (user_ns = current_user_ns(); user_ns; user_ns = user_ns->parent) { /* Pairs with smp_store_release() in bm_fill_super(). */ misc = smp_load_acquire(&user_ns->binfmt_misc); if (misc) return misc; - - user_ns = user_ns->parent; } return &init_binfmt_misc; } +/** + * entry_select_interpreter - get the interpreter for the matched @e + * @e: matched binary type handler + * @bprm: binary that is being executed + * + * A static entry carries its interpreter path, for a 'B' entry the + * handler's load program selects it, either by path or by the name of one + * of the interpreters the entry bound. The match is committed, so a failing + * program fails the exec. + * + * Return: the interpreter on success, an ERR_PTR on failure + */ +static const char *entry_select_interpreter(const struct binfmt_misc_entry *e, + struct linux_binprm *bprm) +{ + int retval; + + /* + * Drop what a previous chain level staged before anything can pick it + * up. A static entry stages nothing but consumes a staged file just + * like a 'B' entry does. + */ + drop_staged_selection(bprm); + + if (!test_bit(MISC_FMT_BPF_BIT, &e->flags)) + return e->interpreter; + + /* The interpreters this entry lets the program choose from. */ + bprm->bpf_interps = &e->interps; + retval = e->bpf_ops->load(bprm); + bprm->bpf_interps = NULL; + if (retval) { + /* Keep a program-supplied error within errno range. */ + if (retval > 0 || retval < -MAX_ERRNO) + retval = -ENOEXEC; + goto drop_staged; + } + + /* Selecting an interpreter is part of the contract. */ + if (!bprm->bpf_interp) { + retval = -ENOEXEC; + goto drop_staged; + } + + return bprm->bpf_interp; + +drop_staged: + /* A failing load leaves nothing behind for later entries. */ + drop_staged_selection(bprm); + return ERR_PTR(retval); +} + +/** + * entry_invocation_flags - the invocation flags in effect for this exec + * @e: matched binary type handler + * @bprm: binary that is being executed + * + * A static entry fixes its flags at registration, a 'B' entry's load program + * picks them per exec with bpf_binprm_set_flags(). Translate the latter into + * the former, implications included, so the dispatch has one set to act on. + * + * Return: the invocation flags for this exec + */ +static unsigned long entry_invocation_flags(const struct binfmt_misc_entry *e, + struct linux_binprm *bprm) +{ + unsigned long flags = 0; + u64 bpf_flags; + + if (!test_bit(MISC_FMT_BPF_BIT, &e->flags)) + return e->flags; + + bpf_flags = bprm->bpf_flags; + /* Clear so they can't accumulate into a nested interpreter level. */ + bprm->bpf_flags = 0; + + if (bpf_flags & BPF_BINPRM_PRESERVE_ARGV0) + flags |= MISC_FMT_PRESERVE_ARGV0; + if (bpf_flags & BPF_BINPRM_EXECFD) + flags |= MISC_FMT_OPEN_BINARY; + if (bpf_flags & BPF_BINPRM_CREDENTIALS) + flags |= MISC_FMT_CREDENTIALS | MISC_FMT_OPEN_BINARY; + if (bpf_flags & BPF_BINPRM_TRANSPARENT) + flags |= MISC_FMT_TRANSPARENT | MISC_FMT_OPEN_BINARY; + if (bpf_flags & BPF_BINPRM_LOADER) + flags |= MISC_FMT_LOADER; + + return flags; +} + +/** + * entry_open_interpreter - open the entry's interpreter for execution + * @e: matched binary type handler + * @bprm: binary that is being executed + * @interpreter: the interpreter selected for this exec + * + * An 'F' entry hands out a clone of the file it pre-opened at registration, + * and so does a 'B' entry whose load program selected one of the + * interpreters it bound. Any other entry opens the selected path. + * + * Return: the opened interpreter on success, an ERR_PTR on failure + */ +static struct file *entry_open_interpreter(const struct binfmt_misc_entry *e, + struct linux_binprm *bprm, + const char *interpreter) +{ + struct file *interp_file __free(fput) = NULL; + struct binfmt_misc_interp *interp; + struct file *bound; + int retval; + + if (bprm->bpf_interp_file) { + bound = bprm->bpf_interp_file; + } else if (e->flags & MISC_FMT_OPEN_FILE) { + /* An 'F' entry pre-opened exactly one interpreter. */ + interp = list_first_entry(&e->interps, + struct binfmt_misc_interp, list); + bound = interp->file; + } else { + return open_exec(interpreter); + } + + interp_file = file_clone_open(bound); + if (IS_ERR(interp_file)) + return interp_file; + + retval = exe_file_deny_write_access(interp_file); + if (retval) + return ERR_PTR(retval); + + return no_free_ptr(interp_file); +} + +/** + * build_interp_argv - splice the interpreter invocation into the argv + * @bprm: binary that is being executed + * @interpreter: the interpreter selected for this exec + * @flags: invocation flags in effect for this exec + * + * The interpreter becomes argv[0] and the binary its last argument, with an + * optional staged argument in between. The caller's argv[0] is dropped + * unless 'P' keeps it. + * + * Return: 0 on success, a negative error code on failure + */ +static int build_interp_argv(struct linux_binprm *bprm, const char *interpreter, + unsigned long flags) +{ + int retval; + + /* The interpreter has to be able to load the binary by path. */ + if (bprm->interp_flags & BINPRM_FLAGS_PATH_INACCESSIBLE) + return -ENOENT; + + /* The entry's own choice - not one accumulated from an earlier level. */ + if (flags & MISC_FMT_PRESERVE_ARGV0) { + bprm->interp_flags |= BINPRM_FLAGS_PRESERVE_ARGV0; + } else { + retval = remove_arg_zero(bprm); + if (retval) + return retval; + } + + /* make the binary the last argument to the interpreter */ + retval = copy_string_kernel(bprm->interp, bprm); + if (retval < 0) + return retval; + bprm->argc++; + + /* + * A single optional argument to the interpreter, inserted between it + * and the binary just like the argument of a #! interpreter line. + */ + if (bprm->bpf_interp_arg) { + retval = copy_string_kernel(bprm->bpf_interp_arg, bprm); + if (retval < 0) + return retval; + bprm->argc++; + /* Consumed - don't let it leak into a nested interpreter's argv. */ + kfree(bprm->bpf_interp_arg); + bprm->bpf_interp_arg = NULL; + } + + /* add the interp as argv[0] */ + retval = copy_string_kernel(interpreter, bprm); + if (retval < 0) + return retval; + bprm->argc++; + + return 0; +} + /* * the loader itself */ static int load_misc_binary(struct linux_binprm *bprm) { - Node *fmt; - struct file *interp_file = NULL; - int retval = -ENOEXEC; + struct binfmt_misc_entry *fmt __free(put_binfmt_handler) = NULL; + const char *interpreter; + struct file *interp_file; struct binfmt_misc *misc; + unsigned long flags; + int retval; - misc = load_binfmt_misc(); - if (!misc->enabled) - return retval; + /* Only binfmt_misc stages one and exec_binprm() clears it per round. */ + WARN_ON_ONCE(bprm->loader); + + misc = current_binfmt_misc(); + if (!READ_ONCE(misc->enabled)) + return -ENOEXEC; fmt = get_binfmt_handler(misc, bprm); if (!fmt) - return retval; + return -ENOEXEC; - /* Need to be able to load the file after exec */ - retval = -ENOENT; - if (bprm->interp_flags & BINPRM_FLAGS_PATH_INACCESSIBLE) - goto ret; + interpreter = entry_select_interpreter(fmt, bprm); + if (IS_ERR(interpreter)) + return PTR_ERR(interpreter); - if (fmt->flags & MISC_FMT_PRESERVE_ARGV0) { - bprm->interp_flags |= BINPRM_FLAGS_PRESERVE_ARGV0; - } else { - retval = remove_arg_zero(bprm); - if (retval) - goto ret; - } + flags = entry_invocation_flags(fmt, bprm); - /* make argv[1] be the path to the binary */ - retval = copy_string_kernel(bprm->interp, bprm); - if (retval < 0) - goto ret; - bprm->argc++; - - /* add the interp as argv[0] */ - retval = copy_string_kernel(fmt->interpreter, bprm); - if (retval < 0) - goto ret; - bprm->argc++; - - /* Update interp in case binfmt_script needs it. */ - retval = bprm_change_interp(fmt->interpreter, bprm); - if (retval < 0) - goto ret; - - if (fmt->flags & MISC_FMT_OPEN_FILE) { - interp_file = file_clone_open(fmt->interp_file); - if (!IS_ERR(interp_file)) { - int err = exe_file_deny_write_access(interp_file); - - if (err) { - fput(interp_file); - interp_file = ERR_PTR(err); - } - } - } else { - interp_file = open_exec(fmt->interpreter); - } - retval = PTR_ERR(interp_file); - if (IS_ERR(interp_file)) - goto ret; - - bprm->interpreter = interp_file; - if (fmt->flags & MISC_FMT_OPEN_BINARY) - bprm->have_execfd = 1; - if (fmt->flags & MISC_FMT_CREDENTIALS) - bprm->execfd_creds = 1; - - retval = 0; -ret: + /* No argv is built for a staged argument to land in. */ + if ((flags & (MISC_FMT_LOADER | MISC_FMT_TRANSPARENT)) && + bprm->bpf_interp_arg) + return -EINVAL; /* - * If we actually put the node here all concurrent calls to - * load_misc_binary() will have finished. We also know - * that for the refcount to be zero someone must have concurently - * removed the binary type handler from the list and it's our job to - * free it. + * Stash the interpreter for binfmt_elf to consume in place of the + * binary's PT_INTERP and decline the match, so the search continues + * to the real format in the same round. */ - put_binfmt_handler(fmt); + if (flags & MISC_FMT_LOADER) { + interp_file = entry_open_interpreter(fmt, bprm, interpreter); + if (IS_ERR(interp_file)) { + retval = PTR_ERR(interp_file); + /* Declining here would run the binary's own PT_INTERP. */ + return retval == -ENOEXEC ? -EACCES : retval; + } - return retval; + bprm->loader = interp_file; + return -ENOEXEC; + } + + if (!(flags & MISC_FMT_TRANSPARENT)) { + retval = build_interp_argv(bprm, interpreter, flags); + if (retval) + return retval; + } + + /* Update interp for the next round; sched_prepare_exec reports it. */ + retval = bprm_change_interp(interpreter, bprm); + if (retval < 0) + return retval; + + interp_file = entry_open_interpreter(fmt, bprm, interpreter); + if (IS_ERR(interp_file)) + return PTR_ERR(interp_file); + + /* Raise only past the last failure, or an -ENOEXEC decline leaks it. */ + if (flags & MISC_FMT_TRANSPARENT) + bprm->interp_flags |= BINPRM_FLAGS_TRANSPARENT_INTERP; + + bprm->interpreter = interp_file; + if (flags & MISC_FMT_OPEN_BINARY) + bprm->have_execfd = 1; + if (flags & MISC_FMT_CREDENTIALS) + bprm->execfd_creds = 1; + return 0; } /* Command parsers */ /* - * parses and copies one argument enclosed in del from *sp to *dp, - * recognising the \x special. - * returns pointer to the copied argument or NULL in case of an - * error (and sets err) or null argument length. + * Scan the argument starting at @s up to the delimiter @del, recognising + * the \x escape. Terminates the argument with a NUL and returns a pointer + * past it or NULL on a malformed escape. */ static char *scanarg(char *s, char del) { @@ -306,45 +701,129 @@ static char *scanarg(char *s, char del) return NULL; } } - s[-1] ='\0'; + s[-1] = '\0'; return s; } -static char *check_special_flags(char *sfs, Node *e) +/* Parse the 'flags' field, stopping at the first character that is not one. */ +static char *check_special_flags(char *p, struct binfmt_misc_entry *e) { - char *p = sfs; - int cont = 1; + for (;; p++) { + const struct binfmt_misc_flag *f = misc_flag_by_char(*p); - /* special flags */ - while (cont) { - switch (*p) { - case 'P': - pr_debug("register: flag: P (preserve argv0)\n"); - p++; - e->flags |= MISC_FMT_PRESERVE_ARGV0; - break; - case 'O': - pr_debug("register: flag: O (open binary)\n"); - p++; - e->flags |= MISC_FMT_OPEN_BINARY; - break; - case 'C': - pr_debug("register: flag: C (preserve creds)\n"); - p++; - /* this flags also implies the - open-binary flag */ - e->flags |= (MISC_FMT_CREDENTIALS | - MISC_FMT_OPEN_BINARY); - break; - case 'F': - pr_debug("register: flag: F: open interpreter file now\n"); - p++; - e->flags |= MISC_FMT_OPEN_FILE; - break; - default: - cont = 0; - } + if (!f) + return p; + pr_debug("register: flag: %c (%s)\n", f->c, f->desc); + e->flags |= f->flag | f->implies; } +} + +/* Parse the 'offset', 'magic' and 'mask' fields of an 'M' entry. */ +static char *parse_magic_fields(struct binfmt_misc_entry *e, char *p, char del) +{ + char *s; + + /* Parse the 'offset' field. */ + s = strchr(p, del); + if (!s) + return NULL; + *s = '\0'; + if (p != s) { + if (kstrtoint(p, 10, &e->offset) || e->offset < 0) + return NULL; + } + p = s + 1; + pr_debug("register: offset: %#x\n", e->offset); + + /* Parse the 'magic' field. */ + e->magic = p; + p = scanarg(p, del); + if (!p || !e->magic[0]) + return NULL; + print_hex_dump_debug( + KBUILD_MODNAME ": register: magic[raw]: ", + DUMP_PREFIX_NONE, 16, 1, e->magic, p - e->magic, true); + + /* Parse the 'mask' field. */ + e->mask = p; + p = scanarg(p, del); + if (!p) + return NULL; + if (!e->mask[0]) { + e->mask = NULL; + pr_debug("register: mask[raw]: none\n"); + } else { + print_hex_dump_debug( + KBUILD_MODNAME ": register: mask[raw]: ", + DUMP_PREFIX_NONE, 16, 1, e->mask, p - e->mask, true); + } + + /* + * Decode the magic & mask fields. Note: while we might have accepted + * embedded NUL bytes from above, the unescape helpers will stop at + * the first one they encounter. + */ + e->size = string_unescape_inplace(e->magic, UNESCAPE_HEX); + if (e->mask && string_unescape_inplace(e->mask, UNESCAPE_HEX) != e->size) + return NULL; + if (e->size > BINPRM_BUF_SIZE || BINPRM_BUF_SIZE - e->size < e->offset) + return NULL; + pr_debug("register: magic/mask length: %i\n", e->size); + print_hex_dump_debug( + KBUILD_MODNAME ": register: magic[decoded]: ", + DUMP_PREFIX_NONE, 16, 1, e->magic, e->size, true); + if (e->mask) + print_hex_dump_debug( + KBUILD_MODNAME ": register: mask[decoded]: ", + DUMP_PREFIX_NONE, 16, 1, e->mask, e->size, true); + return p; +} + +/* Parse the 'magic' field of an 'E' entry: the filename extension. */ +static char *parse_extension_fields(struct binfmt_misc_entry *e, char *p, + char del) +{ + /* Skip the 'offset' field. */ + p = strchr(p, del); + if (!p) + return NULL; + *p++ = '\0'; + + /* Parse the 'magic' field. */ + e->magic = p; + p = strchr(p, del); + if (!p) + return NULL; + *p++ = '\0'; + if (!e->magic[0] || strchr(e->magic, '/')) + return NULL; + pr_debug("register: extension: {%s}\n", e->magic); + + /* Skip the 'mask' field. */ + p = strchr(p, del); + if (!p) + return NULL; + *p++ = '\0'; + return p; +} + +/* + * Parse the fields of a 'B' entry: the 'offset', 'magic' and 'mask' fields + * must be empty. The handler name is carried in the 'interpreter' field. + */ +static char *parse_bpf_fields(struct binfmt_misc_entry *e, char *p, char del) +{ + /* The 'offset' field must be empty. */ + if (*p++ != del) + return NULL; + + /* The 'magic' field must be empty. */ + if (*p++ != del) + return NULL; + + /* The 'mask' field must be empty. */ + if (*p++ != del) + return NULL; return p; } @@ -354,54 +833,53 @@ static char *check_special_flags(char *sfs, Node *e) * ':name:type:offset:magic:mask:interpreter:flags' * where the ':' is the IFS, that can be chosen with the first char */ -static Node *create_entry(const char __user *buffer, size_t count) +static struct binfmt_misc_entry *create_entry(const char __user *buffer, + size_t count) { - Node *e; - int memsize, err; + struct binfmt_misc_entry *e __free(kfree) = NULL; char *buf, *p; char del; pr_debug("register: received %zu bytes\n", count); /* some sanity checks */ - err = -EINVAL; if ((count < 11) || (count > MAX_REGISTER_LENGTH)) - goto out; + return ERR_PTR(-EINVAL); - err = -ENOMEM; - memsize = sizeof(Node) + count + 8; - e = kmalloc(memsize, GFP_KERNEL_ACCOUNT); + e = kmalloc(struct_size(e, buf, count + MISC_DELIM_PAD), + GFP_KERNEL_ACCOUNT); if (!e) - goto out; + return ERR_PTR(-ENOMEM); - p = buf = (char *)e + sizeof(Node); + p = buf = e->buf; - memset(e, 0, sizeof(Node)); + memset(e, 0, sizeof(*e)); + INIT_LIST_HEAD(&e->interps); if (copy_from_user(buf, buffer, count)) - goto efault; + return ERR_PTR(-EFAULT); - del = *p++; /* delimeter */ + del = *p++; /* delimiter */ pr_debug("register: delim: %#x {%c}\n", del, del); /* A flag-char delimiter runs the flag scan off the buffer. */ - if (del == 'P' || del == 'O' || del == 'C' || del == 'F') - goto einval; + if (misc_flag_by_char(del)) + return ERR_PTR(-EINVAL); /* Pad the buffer with the delim to simplify parsing below. */ - memset(buf + count, del, 8); + memset(buf + count, del, MISC_DELIM_PAD); /* Parse the 'name' field. */ e->name = p; p = strchr(p, del); if (!p) - goto einval; + return ERR_PTR(-EINVAL); *p++ = '\0'; if (!e->name[0] || !strcmp(e->name, ".") || !strcmp(e->name, "..") || strchr(e->name, '/')) - goto einval; + return ERR_PTR(-EINVAL); pr_debug("register: name: {%s}\n", e->name); @@ -409,225 +887,213 @@ static Node *create_entry(const char __user *buffer, size_t count) switch (*p++) { case 'E': pr_debug("register: type: E (extension)\n"); - e->flags = 1 << Enabled; + e->flags = BIT(MISC_FMT_ENABLED_BIT); break; case 'M': pr_debug("register: type: M (magic)\n"); - e->flags = (1 << Enabled) | (1 << Magic); + e->flags = BIT(MISC_FMT_ENABLED_BIT) | BIT(MISC_FMT_MAGIC_BIT); + break; + case 'B': + pr_debug("register: type: B (bpf)\n"); + if (!IS_ENABLED(CONFIG_BINFMT_MISC_BPF)) + return ERR_PTR(-EINVAL); + e->flags = BIT(MISC_FMT_ENABLED_BIT) | BIT(MISC_FMT_BPF_BIT); break; default: - goto einval; + return ERR_PTR(-EINVAL); } if (*p++ != del) - goto einval; + return ERR_PTR(-EINVAL); - if (test_bit(Magic, &e->flags)) { - /* Handle the 'M' (magic) format. */ - char *s; - - /* Parse the 'offset' field. */ - s = strchr(p, del); - if (!s) - goto einval; - *s = '\0'; - if (p != s) { - int r = kstrtoint(p, 10, &e->offset); - if (r != 0 || e->offset < 0) - goto einval; - } - p = s; - if (*p++) - goto einval; - pr_debug("register: offset: %#x\n", e->offset); - - /* Parse the 'magic' field. */ - e->magic = p; - p = scanarg(p, del); - if (!p) - goto einval; - if (!e->magic[0]) - goto einval; - if (USE_DEBUG) - print_hex_dump_bytes( - KBUILD_MODNAME ": register: magic[raw]: ", - DUMP_PREFIX_NONE, e->magic, p - e->magic); - - /* Parse the 'mask' field. */ - e->mask = p; - p = scanarg(p, del); - if (!p) - goto einval; - if (!e->mask[0]) { - e->mask = NULL; - pr_debug("register: mask[raw]: none\n"); - } else if (USE_DEBUG) - print_hex_dump_bytes( - KBUILD_MODNAME ": register: mask[raw]: ", - DUMP_PREFIX_NONE, e->mask, p - e->mask); - - /* - * Decode the magic & mask fields. - * Note: while we might have accepted embedded NUL bytes from - * above, the unescape helpers here will stop at the first one - * it encounters. - */ - e->size = string_unescape_inplace(e->magic, UNESCAPE_HEX); - if (e->mask && - string_unescape_inplace(e->mask, UNESCAPE_HEX) != e->size) - goto einval; - if (e->size > BINPRM_BUF_SIZE || - BINPRM_BUF_SIZE - e->size < e->offset) - goto einval; - pr_debug("register: magic/mask length: %i\n", e->size); - if (USE_DEBUG) { - print_hex_dump_bytes( - KBUILD_MODNAME ": register: magic[decoded]: ", - DUMP_PREFIX_NONE, e->magic, e->size); - - if (e->mask) { - int i; - char *masked = kmalloc(e->size, GFP_KERNEL_ACCOUNT); - - print_hex_dump_bytes( - KBUILD_MODNAME ": register: mask[decoded]: ", - DUMP_PREFIX_NONE, e->mask, e->size); - - if (masked) { - for (i = 0; i < e->size; ++i) - masked[i] = e->magic[i] & e->mask[i]; - print_hex_dump_bytes( - KBUILD_MODNAME ": register: magic[masked]: ", - DUMP_PREFIX_NONE, masked, e->size); - - kfree(masked); - } - } - } - } else { - /* Handle the 'E' (extension) format. */ - - /* Skip the 'offset' field. */ - p = strchr(p, del); - if (!p) - goto einval; - *p++ = '\0'; - - /* Parse the 'magic' field. */ - e->magic = p; - p = strchr(p, del); - if (!p) - goto einval; - *p++ = '\0'; - if (!e->magic[0] || strchr(e->magic, '/')) - goto einval; - pr_debug("register: extension: {%s}\n", e->magic); - - /* Skip the 'mask' field. */ - p = strchr(p, del); - if (!p) - goto einval; - *p++ = '\0'; - } + if (test_bit(MISC_FMT_BPF_BIT, &e->flags)) + p = parse_bpf_fields(e, p, del); + else if (test_bit(MISC_FMT_MAGIC_BIT, &e->flags)) + p = parse_magic_fields(e, p, del); + else + p = parse_extension_fields(e, p, del); + if (!p) + return ERR_PTR(-EINVAL); /* Parse the 'interpreter' field. */ e->interpreter = p; p = strchr(p, del); if (!p) - goto einval; + return ERR_PTR(-EINVAL); *p++ = '\0'; - if (!e->interpreter[0]) - goto einval; - pr_debug("register: interpreter: {%s}\n", e->interpreter); + if (test_bit(MISC_FMT_BPF_BIT, &e->flags)) { + /* The 'interpreter' field carries the handler name. */ + e->bpf_ops_name = e->interpreter; + e->interpreter = NULL; + if (!e->bpf_ops_name[0]) + return ERR_PTR(-EINVAL); + pr_debug("register: bpf handler: {%s}\n", e->bpf_ops_name); + } else if (!e->interpreter[0]) { + return ERR_PTR(-EINVAL); + } else { + pr_debug("register: interpreter: {%s}\n", e->interpreter); + } /* Parse the 'flags' field. */ p = check_special_flags(p, e); + + /* + * A bpf handler decides the invocation flags per exec with + * bpf_binprm_set_flags() rather than fixing them at registration, and + * the interpreters it binds pre-open what 'F' would have, so a 'B' + * entry carries no invocation flags. + */ + if (test_bit(MISC_FMT_BPF_BIT, &e->flags) && + (e->flags & MISC_FMT_INVOCATION_FLAGS)) + return ERR_PTR(-EINVAL); + + /* + * 'D' is a directive for this registration rather than a lasting + * property, so consume it: the entry is created disabled and stays + * out of the search list until '1' is written to its entry file. + * Staying out is what leaves it open to being given interpreters; + * the first enable publishes it, for good. + */ + if (e->flags & MISC_FMT_DISABLED) { + e->flags &= ~MISC_FMT_DISABLED; + clear_bit(MISC_FMT_ENABLED_BIT, &e->flags); + } + + /* Transparency preserves the whole argv, argv[0] included. */ + if ((e->flags & MISC_FMT_TRANSPARENT) && + (e->flags & MISC_FMT_PRESERVE_ARGV0)) + return ERR_PTR(-EINVAL); + + /* A native exec splices no argv, passes no execfd and needs no creds. */ + if ((e->flags & MISC_FMT_LOADER) && + (e->flags & (MISC_FMT_TRANSPARENT | MISC_FMT_PRESERVE_ARGV0 | + MISC_FMT_CREDENTIALS | MISC_FMT_OPEN_BINARY))) + return ERR_PTR(-EINVAL); + if (*p == '\n') p++; if (p != buf + count) - goto einval; + return ERR_PTR(-EINVAL); - return e; + /* Non-F opens the interp at exec against the caller's cwd; require absolute. */ + if ((e->flags & (MISC_FMT_LOADER | MISC_FMT_CREDENTIALS)) && + !(e->flags & MISC_FMT_OPEN_FILE) && + e->interpreter[0] != '/') + return ERR_PTR(-EINVAL); -out: - return ERR_PTR(err); - -efault: - kfree(e); - return ERR_PTR(-EFAULT); -einval: - kfree(e); - return ERR_PTR(-EINVAL); + /* Born holding one reference; put_binfmt_handler() is the teardown. */ + refcount_set(&e->users, 1); + return no_free_ptr(e); } -/* - * Set status of entry/binfmt_misc: - * '1' enables, '0' disables and '-1' clears entry/binfmt_misc - */ -static int parse_command(const char __user *buffer, size_t count) -{ - char s[4]; +/* Commands accepted by the /status and / files. */ +enum bm_command { + BM_CMD_IGNORE, /* empty write */ + BM_CMD_DISABLE, /* "0" */ + BM_CMD_ENABLE, /* "1" */ + BM_CMD_REMOVE, /* "-1" */ +}; - if (count > 3) +/* Longest of the commands above, "-1\n". */ +#define MAX_COMMAND_LENGTH 3 + +/* + * Parse what userspace wrote to /status or an entry file: '1' enables, + * '0' disables and '-1' removes the entry or all entries. + */ +static int parse_command(const char *s, size_t count) +{ + if (count > MAX_COMMAND_LENGTH) return -EINVAL; - if (copy_from_user(s, buffer, count)) - return -EFAULT; if (!count) - return 0; + return BM_CMD_IGNORE; if (s[count - 1] == '\n') count--; if (count == 1 && s[0] == '0') - return 1; + return BM_CMD_DISABLE; if (count == 1 && s[0] == '1') - return 2; + return BM_CMD_ENABLE; if (count == 2 && s[0] == '-' && s[1] == '1') - return 3; + return BM_CMD_REMOVE; return -EINVAL; } +/* Copy in a command from a file that takes nothing else, and parse it. */ +static int read_command(const char __user *buffer, size_t count) +{ + char s[MAX_COMMAND_LENGTH + 1]; + + if (count > sizeof(s) - 1) + return -EINVAL; + if (copy_from_user(s, buffer, count)) + return -EFAULT; + return parse_command(s, count); +} + /* generic stuff */ -static void entry_status(Node *e, char *page) +/* The root directory's inode; its lock serializes configuring an instance. */ +static struct inode *bm_root_inode(struct super_block *sb) { - char *dp = page; - const char *status = "disabled"; - - if (test_bit(Enabled, &e->flags)) - status = "enabled"; - - if (!VERBOSE_STATUS) { - sprintf(page, "%s\n", status); - return; - } - - dp += sprintf(dp, "%s\ninterpreter %s\n", status, e->interpreter); - - /* print the special flags */ - dp += sprintf(dp, "flags: "); - if (e->flags & MISC_FMT_PRESERVE_ARGV0) - *dp++ = 'P'; - if (e->flags & MISC_FMT_OPEN_BINARY) - *dp++ = 'O'; - if (e->flags & MISC_FMT_CREDENTIALS) - *dp++ = 'C'; - if (e->flags & MISC_FMT_OPEN_FILE) - *dp++ = 'F'; - *dp++ = '\n'; - - if (!test_bit(Magic, &e->flags)) { - sprintf(dp, "extension .%s\n", e->magic); - } else { - dp += sprintf(dp, "offset %i\nmagic ", e->offset); - dp = bin2hex(dp, e->magic, e->size); - if (e->mask) { - dp += sprintf(dp, "\nmask "); - dp = bin2hex(dp, e->mask, e->size); - } - *dp++ = '\n'; - *dp = '\0'; - } + return d_inode(sb->s_root); } -static struct inode *bm_get_inode(struct super_block *sb, int mode) +static void bm_seq_hex(struct seq_file *m, const u8 *data, int size) +{ + for (int i = 0; i < size; i++) + seq_printf(m, "%02x", data[i]); +} + +static int bm_entry_show(struct seq_file *m, void *unused) +{ + struct binfmt_misc_entry *e = m->private; + + if (test_bit(MISC_FMT_ENABLED_BIT, &e->flags)) + seq_puts(m, "enabled\n"); + else + seq_puts(m, "disabled\n"); + + if (test_bit(MISC_FMT_BPF_BIT, &e->flags)) { + struct binfmt_misc_interp *interp; + + seq_printf(m, "bpf %s\n", e->bpf_ops->name); + /* + * A staged entry's set can still grow, so every binding is + * rcu-published. The open file pins the entry and with it + * every node, so rcu is for the tearing, not the lifetime. + */ + rcu_read_lock(); + list_for_each_entry_rcu(interp, &e->interps, list) + seq_printf(m, "bpf-interpreter %s %s\n", + interp->name, interp->path); + rcu_read_unlock(); + } else { + seq_printf(m, "interpreter %s\n", e->interpreter); + } + + /* print the special flags */ + seq_puts(m, "flags: "); + for (int i = 0; i < ARRAY_SIZE(misc_flags); i++) + if (e->flags & misc_flags[i].flag) + seq_putc(m, misc_flags[i].c); + seq_putc(m, '\n'); + + if (test_bit(MISC_FMT_BPF_BIT, &e->flags)) { + /* The program does the matching. */ + } else if (!test_bit(MISC_FMT_MAGIC_BIT, &e->flags)) { + seq_printf(m, "extension .%s\n", e->magic); + } else { + seq_printf(m, "offset %i\nmagic ", e->offset); + bm_seq_hex(m, e->magic, e->size); + if (e->mask) { + seq_puts(m, "\nmask "); + bm_seq_hex(m, e->mask, e->size); + } + seq_putc(m, '\n'); + } + return 0; +} + +static struct inode *bm_get_inode(struct super_block *sb, umode_t mode) { struct inode *inode = new_inode(sb); @@ -663,14 +1129,14 @@ static struct binfmt_misc *i_binfmt_misc(struct inode *inode) * entry is removed or the filesystem is unmounted and the super block is * shutdown. * - * If the ->evict call was not caused by a super block shutdown but by a write - * to remove the entry or all entries via bm_{entry,status}_write() the entry - * will have already been removed from the list. We keep the list_empty() check - * to make that explicit. + * If the ->evict call was not caused by a super block shutdown but by + * removing the entry via bm_{entry,status}_write() or unlink(2) the entry + * will have already been removed from the list. We keep the hlist_unhashed() + * check to make that explicit. */ static void bm_evict_inode(struct inode *inode) { - Node *e = inode->i_private; + struct binfmt_misc_entry *e = inode->i_private; clear_inode(inode); @@ -678,89 +1144,259 @@ static void bm_evict_inode(struct inode *inode) struct binfmt_misc *misc; misc = i_binfmt_misc(inode); - write_lock(&misc->entries_lock); - if (!list_empty(&e->list)) - list_del_init(&e->list); - write_unlock(&misc->entries_lock); + spin_lock(&misc->entries_lock); + if (!hlist_unhashed(&e->node)) + hlist_del_init_rcu(&e->node); + spin_unlock(&misc->entries_lock); put_binfmt_handler(e); } } +/** + * unlink_binfmt_handler - unhash a binary type handler + * @misc: handle to binfmt_misc instance + * @e: binary type handler to unhash + * + * Adding and removing entries via bm_{entry,register,status}_write() and + * unlink(2) happens under the exclusively held inode lock of the root + * dentry keeping the list stable for writers. load_misc_binary() walks it + * concurrently under SRCU. The entries_lock is only held around the actual + * unlink to serialize against bm_evict_inode() which unlinks entries + * during umount without holding the root inode lock. + */ +static void unlink_binfmt_handler(struct binfmt_misc *misc, + struct binfmt_misc_entry *e) +{ + spin_lock(&misc->entries_lock); + hlist_del_init_rcu(&e->node); + spin_unlock(&misc->entries_lock); +} + /** * remove_binfmt_handler - remove a binary type handler * @misc: handle to binfmt_misc instance * @e: binary type handler to remove * * Remove a binary type handler from the list of binary type handlers and - * remove its associated dentry. This is called from - * binfmt_{entry,status}_write(). In the future, we might want to think about - * adding a proper ->unlink() method to binfmt_misc instead of forcing caller's - * to use writes to files in order to delete binary type handlers. But it has - * worked for so long that it's not a pressing issue. + * remove its associated dentry. */ -static void remove_binfmt_handler(struct binfmt_misc *misc, Node *e) +static void remove_binfmt_handler(struct binfmt_misc *misc, + struct binfmt_misc_entry *e) { - write_lock(&misc->entries_lock); - list_del_init(&e->list); - write_unlock(&misc->entries_lock); + unlink_binfmt_handler(misc, e); locked_recursive_removal(e->dentry, NULL); } +/* Remove @e unless it was already removed. */ +static void bm_remove_entry(struct binfmt_misc_entry *e, struct super_block *sb) +{ + struct inode *root = bm_root_inode(sb); + + inode_lock_nested(root, I_MUTEX_PARENT); + /* A staged entry is not hashed; the dentry says if it was removed. */ + if (!d_unhashed(e->dentry)) + remove_binfmt_handler(i_binfmt_misc(root), e); + inode_unlock(root); +} + +/* Remove all entries of the binfmt_misc instance @misc belonging to @sb. */ +static void bm_remove_all_entries(struct binfmt_misc *misc, + struct super_block *sb) +{ + struct inode *root = bm_root_inode(sb); + struct dentry *child = NULL; + + inode_lock_nested(root, I_MUTEX_PARENT); + /* + * Walk the directory rather than the search list: a staged entry + * is in the former but not yet in the latter. The control files + * carry no entry and stay. + */ + while ((child = find_next_child(sb->s_root, child))) { + struct binfmt_misc_entry *e = d_inode(child)->i_private; + + if (e) + remove_binfmt_handler(misc, e); + } + inode_unlock(root); +} + +/** + * bm_unlink - remove a binary type handler via unlink(2) + * @dir: inode of the root directory + * @dentry: entry file to remove + * + * Removing the entry file removes its binary type handler, exactly like + * writing -1 to it does. The status and register control files can't be + * removed. The VFS calls this with the root inode lock held which + * serializes against the write based add and remove paths. + */ +static int bm_unlink(struct inode *dir, struct dentry *dentry) +{ + struct binfmt_misc_entry *e = d_inode(dentry)->i_private; + + if (!e) + return -EPERM; + + unlink_binfmt_handler(i_binfmt_misc(dir), e); + return simple_unlink(dir, dentry); +} + +static const struct inode_operations bm_dir_inode_operations = { + .lookup = simple_lookup, + .unlink = bm_unlink, +}; + /* / */ -static ssize_t -bm_entry_read(struct file *file, char __user *buf, size_t nbytes, loff_t *ppos) +static int bm_entry_open(struct inode *inode, struct file *file) { - Node *e = file_inode(file)->i_private; - ssize_t res; - char *page; + int ret; - page = kmalloc(PAGE_SIZE, GFP_KERNEL); - if (!page) - return -ENOMEM; + ret = single_open(file, bm_entry_show, inode->i_private); + if (ret) + return ret; - entry_status(e, page); + /* seq_open() clears FMODE_PWRITE, bm_entry_write() takes any offset */ + if (file->f_mode & FMODE_WRITE) + file->f_mode |= FMODE_PWRITE; + return 0; +} - res = simple_read_from_buffer(buf, nbytes, ppos, page, strlen(page)); +/* + * Longest '+ ' a write can spell, and with it the longest + * command an entry file takes: the two delimiters and a newline on top of + * the two names. + */ +#define MAX_BINDING_LENGTH (BINFMT_MISC_INTERP_NAME_MAX + PATH_MAX + 3) - kfree(page); - return res; +/** + * bm_entry_add_interp - bind another interpreter to a staged entry + * @e: the entry + * @file: the entry file being written to, for its credentials + * @buf: the '+ ' command, parsed in place and owned by the caller + * @count: its length + * + * A 'D' entry is registered outside the search list, which is what leaves + * it open to being configured: it cannot be matched, so no exec can be + * holding its interpreters and the set can still grow. Its first enable + * publishes it and ends that. One interpreter per write, up to + * BINFMT_MISC_INTERP_MAX of them, none of which has to fit in a register + * string. + * + * Return: @count on success, a negative errno on failure + */ +static ssize_t bm_entry_add_interp(struct binfmt_misc_entry *e, + struct file *file, char *buf, size_t count) +{ + struct file *f __free(close_interp_file) = NULL; + struct inode *root = bm_root_inode(file_inode(file)->i_sb); + size_t nlen, plen; + char *name, *path; + int retval; + + /* Settled before the open: type is fixed, publication is permanent. */ + if (!test_bit(MISC_FMT_BPF_BIT, &e->flags)) + return -EINVAL; + if (!hlist_unhashed_lockless(&e->node)) + return -EBUSY; + + /* '+ ': the path is everything past the first space. */ + name = buf + 1; + path = strchr(name, ' '); + if (!path) + return -EINVAL; + *path++ = '\0'; + + plen = strlen(path); + /* The command has to end at the write, like a register string. */ + if (path + plen != buf + count) + return -EINVAL; + if (plen && path[plen - 1] == '\n') + path[--plen] = '\0'; + /* Resolved now, so a relative path would name the writer's cwd. */ + if (path[0] != '/') + return -EINVAL; + + nlen = path - name - 1; + if (!nlen || nlen > BINFMT_MISC_INTERP_NAME_MAX) + return -EINVAL; + /* The name prints between delimiters, so keep it a printable word. */ + for (const char *p = name; *p; p++) + if (!isascii(*p) || !isgraph(*p)) + return -EINVAL; + + /* Opened before the lock: resolving it may walk this very filesystem. */ + f = open_interp_file(file->f_cred, path); + if (IS_ERR(f)) + return PTR_ERR(f); + + inode_lock(root); + if (d_unhashed(e->dentry)) + retval = -ENOENT; /* removed while we were opening it */ + else if (!hlist_unhashed(&e->node)) + retval = -EBUSY; /* published while we were opening it */ + else + retval = entry_attach_interpreter(e, name, path, f); + inode_unlock(root); + if (retval) + return retval; + + /* The file is owned by the entry now. */ + retain_and_null_ptr(f); + return count; } static ssize_t bm_entry_write(struct file *file, const char __user *buffer, size_t count, loff_t *ppos) { struct inode *inode = file_inode(file); - Node *e = inode->i_private; - int res = parse_command(buffer, count); + struct binfmt_misc_entry *e = inode->i_private; + char *buf __free(kfree) = NULL; + int res; + + /* A binding is the longest command this file takes. */ + if (count > MAX_BINDING_LENGTH) + return -E2BIG; + + buf = memdup_user_nul(buffer, count); + if (IS_ERR(buf)) + return PTR_ERR(buf); + + /* '+ ' binds an interpreter, everything else toggles. */ + if (buf[0] == '+') + return bm_entry_add_interp(e, file, buf, count); + + res = parse_command(buf, count); switch (res) { - case 1: - /* Disable this handler. */ - clear_bit(Enabled, &e->flags); + case BM_CMD_DISABLE: + clear_bit(MISC_FMT_ENABLED_BIT, &e->flags); break; - case 2: - /* Enable this handler. */ - set_bit(Enabled, &e->flags); - break; - case 3: - /* Delete this handler. */ - inode = d_inode(inode->i_sb->s_root); - inode_lock_nested(inode, I_MUTEX_PARENT); + case BM_CMD_ENABLE: { + struct inode *root = bm_root_inode(inode->i_sb); /* - * In order to add new element or remove elements from the list - * via bm_{entry,register,status}_write() inode_lock() on the - * root inode must be held. - * The lock is exclusive ensuring that the list can't be - * modified. Only load_misc_binary() can access but does so - * read-only. So we only need to take the write lock when we - * actually remove the entry from the list. + * The first enable publishes a 'D' entry into the search + * list, whole. The lock keeps that ordered against a second + * enable, against removal - a removed entry has nothing left + * to publish - and against binding: what can be matched can + * no longer be configured. */ - if (!list_empty(&e->list)) - remove_binfmt_handler(i_binfmt_misc(inode), e); + inode_lock(root); + set_bit(MISC_FMT_ENABLED_BIT, &e->flags); + if (hlist_unhashed(&e->node) && !d_unhashed(e->dentry)) { + struct binfmt_misc *misc = i_binfmt_misc(inode); - inode_unlock(inode); + spin_lock(&misc->entries_lock); + hlist_add_head_rcu(&e->node, &misc->entries); + spin_unlock(&misc->entries_lock); + } + inode_unlock(root); + break; + } + case BM_CMD_REMOVE: + bm_remove_entry(e, inode->i_sb); break; default: return res; @@ -770,15 +1406,17 @@ static ssize_t bm_entry_write(struct file *file, const char __user *buffer, } static const struct file_operations bm_entry_operations = { - .read = bm_entry_read, + .open = bm_entry_open, + .read = seq_read, .write = bm_entry_write, - .llseek = default_llseek, + .llseek = seq_lseek, + .release = single_release, }; /* /register */ /* add to filesystem */ -static int add_entry(Node *e, struct super_block *sb) +static int add_entry(struct binfmt_misc_entry *e, struct super_block *sb) { struct dentry *dentry = simple_start_creating(sb->s_root, e->name); struct inode *inode; @@ -793,16 +1431,18 @@ static int add_entry(Node *e, struct super_block *sb) return -ENOMEM; } - refcount_set(&e->users, 1); e->dentry = dentry; inode->i_private = e; inode->i_fop = &bm_entry_operations; d_make_persistent(dentry, inode); - misc = i_binfmt_misc(inode); - write_lock(&misc->entries_lock); - list_add(&e->list, &misc->entries); - write_unlock(&misc->entries_lock); + /* A 'D' entry stays out of the search list until its first enable. */ + if (test_bit(MISC_FMT_ENABLED_BIT, &e->flags)) { + misc = i_binfmt_misc(inode); + spin_lock(&misc->entries_lock); + hlist_add_head_rcu(&e->node, &misc->entries); + spin_unlock(&misc->entries_lock); + } simple_done_creating(dentry); return 0; } @@ -810,44 +1450,41 @@ static int add_entry(Node *e, struct super_block *sb) static ssize_t bm_register_write(struct file *file, const char __user *buffer, size_t count, loff_t *ppos) { - Node *e; + struct binfmt_misc_entry *e __free(put_binfmt_handler) = NULL; struct super_block *sb = file_inode(file)->i_sb; - int err = 0; - struct file *f = NULL; + int err; e = create_entry(buffer, count); - if (IS_ERR(e)) return PTR_ERR(e); + if (test_bit(MISC_FMT_BPF_BIT, &e->flags)) { + e->bpf_ops = binfmt_misc_get_ops(sb->s_user_ns, e->bpf_ops_name); + if (!e->bpf_ops) { + pr_notice("register: no bpf handler named %s\n", + e->bpf_ops_name); + return -ENOENT; + } + } + if (e->flags & MISC_FMT_OPEN_FILE) { - /* - * Now that we support unprivileged binfmt_misc mounts make - * sure we use the credentials that the register @file was - * opened with to also open the interpreter. Before that this - * didn't matter much as only a privileged process could open - * the register file. - */ - scoped_with_creds(file->f_cred) - f = open_exec(e->interpreter); - if (IS_ERR(f)) { - pr_notice("register: failed to install interpreter file %s\n", - e->interpreter); - kfree(e); + struct file *f = open_interp_file(file->f_cred, e->interpreter); + + if (IS_ERR(f)) return PTR_ERR(f); + err = entry_attach_interpreter(e, "", e->interpreter, f); + if (err) { + close_interp_file(f); + return err; } - e->interp_file = f; } err = add_entry(e, sb); - if (err) { - if (f) { - exe_file_allow_write_access(f); - filp_close(f, NULL); - } - kfree(e); + if (err) return err; - } + + /* The entry is owned by its inode now. */ + retain_and_null_ptr(e); return count; } @@ -862,10 +1499,10 @@ static ssize_t bm_status_read(struct file *file, char __user *buf, size_t nbytes, loff_t *ppos) { struct binfmt_misc *misc; - char *s; + const char *s; misc = i_binfmt_misc(file_inode(file)); - s = misc->enabled ? "enabled\n" : "disabled\n"; + s = READ_ONCE(misc->enabled) ? "enabled\n" : "disabled\n"; return simple_read_from_buffer(buf, nbytes, ppos, s, strlen(s)); } @@ -873,38 +1510,18 @@ static ssize_t bm_status_write(struct file *file, const char __user *buffer, size_t count, loff_t *ppos) { struct binfmt_misc *misc; - int res = parse_command(buffer, count); - Node *e, *next; - struct inode *inode; + int res = read_command(buffer, count); misc = i_binfmt_misc(file_inode(file)); switch (res) { - case 1: - /* Disable all handlers. */ - misc->enabled = false; + case BM_CMD_DISABLE: + WRITE_ONCE(misc->enabled, false); break; - case 2: - /* Enable all handlers. */ - misc->enabled = true; + case BM_CMD_ENABLE: + WRITE_ONCE(misc->enabled, true); break; - case 3: - /* Delete all handlers. */ - inode = d_inode(file_inode(file)->i_sb->s_root); - inode_lock_nested(inode, I_MUTEX_PARENT); - - /* - * In order to add new element or remove elements from the list - * via bm_{entry,register,status}_write() inode_lock() on the - * root inode must be held. - * The lock is exclusive ensuring that the list can't be - * modified. Only load_misc_binary() can access but does so - * read-only. So we only need to take the write lock when we - * actually remove the entry from the list. - */ - list_for_each_entry_safe(e, next, &misc->entries, list) - remove_binfmt_handler(misc, e); - - inode_unlock(inode); + case BM_CMD_REMOVE: + bm_remove_all_entries(misc, file_inode(file)->i_sb); break; default: return res; @@ -921,7 +1538,7 @@ static const struct file_operations bm_status_operations = { /* Superblock handling */ -static const struct super_operations s_ops = { +static const struct super_operations bm_super_ops = { .statfs = simple_statfs, .evict_inode = bm_evict_inode, }; @@ -971,10 +1588,10 @@ static int bm_fill_super(struct super_block *sb, struct fs_context *fc) if (!misc) return -ENOMEM; - INIT_LIST_HEAD(&misc->entries); - rwlock_init(&misc->entries_lock); + INIT_HLIST_HEAD(&misc->entries); + spin_lock_init(&misc->entries_lock); - /* Pairs with smp_load_acquire() in load_binfmt_misc(). */ + /* Pairs with smp_load_acquire() in current_binfmt_misc(). */ smp_store_release(&user_ns->binfmt_misc, misc); } @@ -988,12 +1605,15 @@ static int bm_fill_super(struct super_block *sb, struct fs_context *fc) * is true. Instead, if someone mounts binfmt_misc for the first time or * again we simply reset ->enabled to true. */ - misc->enabled = true; + WRITE_ONCE(misc->enabled, true); err = simple_fill_super(sb, BINFMTFS_MAGIC, bm_files); - if (!err) - sb->s_op = &s_ops; - return err; + if (err) + return err; + + sb->s_op = &bm_super_ops; + d_inode(sb->s_root)->i_op = &bm_dir_inode_operations; + return 0; } static void bm_free(struct fs_context *fc) @@ -1052,6 +1672,8 @@ static void __exit exit_misc_binfmt(void) { unregister_binfmt(&misc_format); unregister_filesystem(&bm_fs_type); + /* Flush pending bm_entry_free_rcu() callbacks before the text goes. */ + srcu_barrier(&bm_entries_srcu); } core_initcall(init_misc_binfmt); diff --git a/fs/binfmt_misc_bpf.c b/fs/binfmt_misc_bpf.c new file mode 100644 index 000000000000..b246d886437b --- /dev/null +++ b/fs/binfmt_misc_bpf.c @@ -0,0 +1,437 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * BPF-backed binary type handlers for binfmt_misc. + * + * A handler is a struct binfmt_misc_ops struct_ops map. Loading and + * registering it makes the handler available under its name in the user + * namespace it was registered in. A binfmt_misc 'B' entry activates it: + * + * echo ':entry:B:::::' > /register + * + * The entry can bind the interpreters the handler may run its binaries + * with, each opened by the write that binds it and selected by name per + * exec. An entry registered with 'D' is not matchable yet, which is what + * leaves it open to being given them: + * + * echo ':entry:B:::::D' > /register + * echo '+ ' > /entry + * echo 1 > /entry + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +struct bm_bpf_ops_reg { + struct list_head list; + const struct binfmt_misc_ops *ops; + struct bpf_link *link; + struct user_namespace *user_ns; +}; + +static DEFINE_SPINLOCK(bm_bpf_ops_lock); +static LIST_HEAD(bm_bpf_ops_list); + +static struct bpf_struct_ops bpf_binfmt_misc_ops; + +static struct bm_bpf_ops_reg *bm_bpf_ops_find(const struct user_namespace *user_ns, + const char *name) +{ + struct bm_bpf_ops_reg *reg; + + lockdep_assert_held(&bm_bpf_ops_lock); + + list_for_each_entry(reg, &bm_bpf_ops_list, list) { + if (reg->user_ns == user_ns && !strcmp(reg->ops->name, name)) + return reg; + } + return NULL; +} + +/** + * binfmt_misc_get_ops - look up a bpf binary type handler by name + * @user_ns: user namespace of the binfmt_misc instance + * @name: name the handler was registered under + * + * Search @user_ns and its ancestors for a handler named @name, mirroring + * the instance lookup in current_binfmt_misc(). The returned handler stays + * callable until binfmt_misc_put_ops() even if the backing struct_ops map + * is detached or deleted in the meantime. + * + * Return: the handler on success, NULL on failure + */ +const struct binfmt_misc_ops *binfmt_misc_get_ops(struct user_namespace *user_ns, + const char *name) +{ + const struct user_namespace *ns; + struct bm_bpf_ops_reg *reg; + + guard(spinlock)(&bm_bpf_ops_lock); + + for (ns = user_ns; ns; ns = ns->parent) { + reg = bm_bpf_ops_find(ns, name); + if (!reg) + continue; + if (!bpf_struct_ops_get(reg->ops)) + return NULL; + return reg->ops; + } + return NULL; +} + +void binfmt_misc_put_ops(const struct binfmt_misc_ops *ops) +{ + bpf_struct_ops_put(ops); +} + +bool bpf_prog_is_binfmt_misc_ops(const struct bpf_prog *prog) +{ + return prog->type == BPF_PROG_TYPE_STRUCT_OPS && + prog->aux->st_ops == &bpf_binfmt_misc_ops; +} + +/* + * Replace the staged interpreter selection: naming a path drops a bound + * file, selecting a bound interpreter carries its file along. + */ +static void bm_bpf_stage_selection(struct linux_binprm *bprm, char *path, + struct file *f) +{ + if (bprm->bpf_interp_file) + fput(bprm->bpf_interp_file); + kfree(bprm->bpf_interp); + bprm->bpf_interp = path; + bprm->bpf_interp_file = f; +} + +__bpf_kfunc_start_defs(); + +/** + * bpf_binprm_set_interp - select the interpreter for the current exec + * @bprm: binary that is being executed + * @path: absolute path to the interpreter + * @path__sz: size of the @path buffer, including the terminating NUL + * + * To be called from the load program of a struct binfmt_misc_ops handler + * before returning zero; the verifier rejects the call from any other + * program, including the handler's own match program. The path is opened + * with the credentials of the task doing the exec after the program + * returns. Calling it again replaces the selection, as does selecting an + * interpreter the entry bound with bpf_binprm_select_interp(). + * + * Return: 0 on success, a negative errno on failure + */ +__bpf_kfunc int bpf_binprm_set_interp(struct linux_binprm *bprm, + const char *path, size_t path__sz) +{ + size_t len; + char *interp; + + if (!path__sz) + return -EINVAL; + len = strnlen(path, path__sz); + if (len == path__sz) + return -EINVAL; + if (path[0] != '/') + return -EINVAL; + if (len >= PATH_MAX) + return -ENAMETOOLONG; + + interp = kmemdup_nul(path, len, GFP_KERNEL); + if (!interp) + return -ENOMEM; + + bm_bpf_stage_selection(bprm, interp, NULL); + return 0; +} + +/** + * bpf_binprm_select_interp - run this exec under an interpreter the entry bound + * @bprm: binary that is being executed + * @name: name the interpreter was registered under + * @name__sz: size of the @name buffer, including the terminating NUL + * + * To be called from the load program of a struct binfmt_misc_ops handler + * instead of bpf_binprm_set_interp(). It selects one of the interpreters + * the matched entry was registered with, each of which was opened once when + * the entry was registered. Nothing is resolved at exec time, so no + * filesystem view can redirect the interpreter. + * + * The interpreter runs under the path the entry registered it under. + * Calling it again replaces the selection. + * + * Return: 0 on success, -ENOENT if the matched entry bound no interpreter + * of that name, a negative errno on failure + */ +__bpf_kfunc int bpf_binprm_select_interp(struct linux_binprm *bprm, + const char *name, size_t name__sz) +{ + const struct binfmt_misc_interp *interp; + size_t len; + char *path; + + if (!name__sz) + return -EINVAL; + len = strnlen(name, name__sz); + if (len == name__sz || !len) + return -EINVAL; + + interp = binfmt_misc_find_interp(bprm->bpf_interps, name); + if (!interp) + return -ENOENT; + + path = kstrdup(interp->path, GFP_KERNEL); + if (!path) + return -ENOMEM; + + bm_bpf_stage_selection(bprm, path, get_file(interp->file)); + return 0; +} + +/** + * bpf_binprm_set_interp_arg - set a single argument for the interpreter + * @bprm: binary that is being executed + * @arg: argument to pass to the interpreter + * @arg__sz: size of the @arg buffer, including the terminating NUL + * + * To be called from the load program of a struct binfmt_misc_ops handler. The + * argument is passed to the interpreter ahead of the binary, mirroring the + * single optional argument of a #! interpreter line. Calling it again + * replaces the argument. + * + * Return: 0 on success, a negative errno on failure + */ +__bpf_kfunc int bpf_binprm_set_interp_arg(struct linux_binprm *bprm, + const char *arg, size_t arg__sz) +{ + size_t len; + char *val; + + if (!arg__sz) + return -EINVAL; + len = strnlen(arg, arg__sz); + if (len == arg__sz) + return -EINVAL; + if (!len) + return -EINVAL; + + val = kmemdup_nul(arg, len, GFP_KERNEL); + if (!val) + return -ENOMEM; + + kfree(bprm->bpf_interp_arg); + bprm->bpf_interp_arg = val; + return 0; +} + +/** + * bpf_binprm_set_flags - choose the interpreter invocation flags for this exec + * @bprm: binary that is being executed + * @flags: an OR of enum bpf_binprm_flags values + * + * To be called from the load program of a struct binfmt_misc_ops handler. It + * decides per exec what a static entry fixes at registration with the P, C, + * O, T and L flags: BPF_BINPRM_PRESERVE_ARGV0 keeps the caller's argv[0], + * BPF_BINPRM_CREDENTIALS computes credentials from the binary, and + * BPF_BINPRM_EXECFD hands the binary to the interpreter through AT_EXECFD. + * BPF_BINPRM_TRANSPARENT additionally leaves the argument vector untouched, + * making the exec look like a direct execution of the binary. + * BPF_BINPRM_LOADER substitutes the interpreter for the binary's PT_INTERP + * and runs the binary as a native exec; it excludes every other flag. + * Calling it again replaces the flags, passing zero clears them again. + * + * Return: 0 on success, -EINVAL if @flags contains an unknown bit or an + * invalid combination + */ +__bpf_kfunc int bpf_binprm_set_flags(struct linux_binprm *bprm, + enum bpf_binprm_flags flags) +{ + if (flags & ~(BPF_BINPRM_PRESERVE_ARGV0 | BPF_BINPRM_CREDENTIALS | + BPF_BINPRM_EXECFD | BPF_BINPRM_TRANSPARENT | + BPF_BINPRM_LOADER)) + return -EINVAL; + + /* Loader substitution is a native exec: no splice, execfd or creds work. */ + if ((flags & BPF_BINPRM_LOADER) && (flags & ~BPF_BINPRM_LOADER)) + return -EINVAL; + + /* Transparency preserves the whole argv, argv[0] included. */ + if ((flags & BPF_BINPRM_TRANSPARENT) && (flags & BPF_BINPRM_PRESERVE_ARGV0)) + return -EINVAL; + + bprm->bpf_flags = flags; + return 0; +} + +__bpf_kfunc_end_defs(); + +BTF_KFUNCS_START(bm_bpf_kfunc_ids) +BTF_ID_FLAGS(func, bpf_binprm_set_interp, KF_SLEEPABLE) +BTF_ID_FLAGS(func, bpf_binprm_select_interp, KF_SLEEPABLE) +BTF_ID_FLAGS(func, bpf_binprm_set_interp_arg, KF_SLEEPABLE) +BTF_ID_FLAGS(func, bpf_binprm_set_flags, KF_SLEEPABLE) +BTF_KFUNCS_END(bm_bpf_kfunc_ids) + +static int bm_bpf_kfunc_filter(const struct bpf_prog *prog, u32 kfunc_id) +{ + if (!btf_id_set8_contains(&bm_bpf_kfunc_ids, kfunc_id)) + return 0; + if (prog->type != BPF_PROG_TYPE_STRUCT_OPS) + return -EACCES; + /* ->st_ops is unset during the cfg pass; enforced once it is set. */ + if (!prog->aux->st_ops) + return 0; + /* Only the load program decides how a binary is run. */ + if (bpf_prog_is_binfmt_misc_ops(prog) && + prog->aux->attach_st_ops_member_off == offsetof(struct binfmt_misc_ops, load)) + return 0; + return -EACCES; +} + +static const struct btf_kfunc_id_set bm_bpf_kfunc_set = { + .owner = THIS_MODULE, + .set = &bm_bpf_kfunc_ids, + .filter = bm_bpf_kfunc_filter, +}; + +static bool bm_bpf_ops__match(struct linux_binprm *bprm) +{ + return false; +} + +static int bm_bpf_ops__load(struct linux_binprm *bprm) +{ + return 0; +} + +static struct binfmt_misc_ops bm_bpf_ops_stubs = { + .match = bm_bpf_ops__match, + .load = bm_bpf_ops__load, +}; + +static int bm_bpf_init(struct btf *btf) +{ + return register_btf_kfunc_id_set(BPF_PROG_TYPE_STRUCT_OPS, + &bm_bpf_kfunc_set); +} + +static int bm_bpf_check_member(const struct btf_type *t, + const struct btf_member *member, + const struct bpf_prog *prog) +{ + u32 moff = __btf_member_bit_offset(t, member) / 8; + + switch (moff) { + case offsetof(struct binfmt_misc_ops, match): + case offsetof(struct binfmt_misc_ops, load): + /* Reliable file reads at exec time require sleeping. */ + if (!prog->sleepable) + return -EINVAL; + break; + } + return 0; +} + +static int bm_bpf_init_member(const struct btf_type *t, + const struct btf_member *member, + void *kdata, const void *udata) +{ + const struct binfmt_misc_ops *uops = udata; + struct binfmt_misc_ops *ops = kdata; + u32 moff = __btf_member_bit_offset(t, member) / 8; + + switch (moff) { + case offsetof(struct binfmt_misc_ops, name): + if (bpf_obj_name_cpy(ops->name, uops->name, + sizeof(ops->name)) <= 0) + return -EINVAL; + return 1; + } + return 0; +} + +static int bm_bpf_validate(void *kdata) +{ + struct binfmt_misc_ops *ops = kdata; + + if (!ops->match || !ops->load) + return -EINVAL; + return 0; +} + +static int bm_bpf_reg(void *kdata, struct bpf_link *link) +{ + struct binfmt_misc_ops *ops = kdata; + struct bm_bpf_ops_reg *reg; + + reg = kzalloc_obj(*reg, GFP_KERNEL_ACCOUNT); + if (!reg) + return -ENOMEM; + + reg->ops = ops; + reg->link = link; + reg->user_ns = get_user_ns(current_user_ns()); + + guard(spinlock)(&bm_bpf_ops_lock); + + if (bm_bpf_ops_find(reg->user_ns, ops->name)) { + put_user_ns(reg->user_ns); + kfree(reg); + return -EEXIST; + } + + list_add(®->list, &bm_bpf_ops_list); + return 0; +} + +static void bm_bpf_unreg(void *kdata, struct bpf_link *link) +{ + struct bm_bpf_ops_reg *reg; + + guard(spinlock)(&bm_bpf_ops_lock); + + list_for_each_entry(reg, &bm_bpf_ops_list, list) { + if (reg->ops == kdata && reg->link == link) { + list_del(®->list); + put_user_ns(reg->user_ns); + kfree(reg); + return; + } + } +} + +static const struct bpf_verifier_ops bm_bpf_verifier_ops = { + .get_func_proto = bpf_base_func_proto, + .is_valid_access = bpf_tracing_btf_ctx_access, +}; + +static struct bpf_struct_ops bpf_binfmt_misc_ops = { + .verifier_ops = &bm_bpf_verifier_ops, + .init = bm_bpf_init, + .check_member = bm_bpf_check_member, + .init_member = bm_bpf_init_member, + .validate = bm_bpf_validate, + .reg = bm_bpf_reg, + .unreg = bm_bpf_unreg, + .cfi_stubs = &bm_bpf_ops_stubs, + .name = "binfmt_misc_ops", + .owner = THIS_MODULE, +}; + +static int __init bm_bpf_struct_ops_init(void) +{ + return register_bpf_struct_ops(&bpf_binfmt_misc_ops, binfmt_misc_ops); +} +late_initcall(bm_bpf_struct_ops_init); diff --git a/fs/bpf_fs_kfuncs.c b/fs/bpf_fs_kfuncs.c index f1863a891db6..6cb877267978 100644 --- a/fs/bpf_fs_kfuncs.c +++ b/fs/bpf_fs_kfuncs.c @@ -1,6 +1,7 @@ // SPDX-License-Identifier: GPL-2.0 /* Copyright (c) 2024 Google LLC. */ +#include #include #include #include @@ -11,6 +12,7 @@ #include #include #include +#include #include __bpf_kfunc_start_defs(); @@ -359,6 +361,39 @@ __bpf_kfunc int bpf_cgroup_read_xattr(struct cgroup *cgroup, const char *name__s } #endif /* CONFIG_CGROUPS */ +#ifdef CONFIG_NET +/** + * bpf_sock_read_xattr - read xattr of a socket's inode in sockfs + * @sock: socket to get xattr from + * @name__str: name of the xattr + * @value_p: output buffer of the xattr value + * + * Get xattr *name__str* of *sock* and store the output in *value_p*. + * + * For security reasons, only *name__str* with prefix "user." is allowed. + * + * Return: length of the xattr value on success, a negative value on error. + */ +__bpf_kfunc int bpf_sock_read_xattr(struct socket *sock, const char *name__str, + struct bpf_dynptr *value_p) +{ + struct bpf_dynptr_kern *value_ptr = (struct bpf_dynptr_kern *)value_p; + u32 value_len; + void *value; + + /* Only allow reading "user.*" xattrs */ + if (strncmp(name__str, XATTR_USER_PREFIX, XATTR_USER_PREFIX_LEN)) + return -EPERM; + + value_len = __bpf_dynptr_size(value_ptr); + value = __bpf_dynptr_data_rw(value_ptr, value_len); + if (!value) + return -EINVAL; + + return sock_read_xattr(sock, name__str, value, value_len); +} +#endif /* CONFIG_NET */ + /** * bpf_real_data_inode - get the real inode hosting a file's data * @file: file to resolve @@ -390,12 +425,30 @@ BTF_ID_FLAGS(func, bpf_get_file_xattr, KF_SLEEPABLE) BTF_ID_FLAGS(func, bpf_set_dentry_xattr, KF_SLEEPABLE) BTF_ID_FLAGS(func, bpf_remove_dentry_xattr, KF_SLEEPABLE) BTF_ID_FLAGS(func, bpf_real_data_inode, KF_SLEEPABLE | KF_RET_NULL) +#ifdef CONFIG_NET +BTF_ID_FLAGS(func, bpf_sock_read_xattr, KF_RCU) +#endif BTF_KFUNCS_END(bpf_fs_kfunc_set_ids) +/* Side-effecting kfuncs that stay exclusive to LSM programs. */ +BTF_SET_START(bpf_fs_kfunc_lsm_only_ids) +BTF_ID(func, bpf_set_dentry_xattr) +BTF_ID(func, bpf_remove_dentry_xattr) +BTF_SET_END(bpf_fs_kfunc_lsm_only_ids) + static int bpf_fs_kfuncs_filter(const struct bpf_prog *prog, u32 kfunc_id) { - if (!btf_id_set8_contains(&bpf_fs_kfunc_set_ids, kfunc_id) || - prog->type == BPF_PROG_TYPE_LSM) + if (!btf_id_set8_contains(&bpf_fs_kfunc_set_ids, kfunc_id)) + return 0; + if (prog->type == BPF_PROG_TYPE_LSM) + return 0; + if (prog->type != BPF_PROG_TYPE_STRUCT_OPS) + return -EACCES; + /* ->st_ops is unset during the cfg pass; enforced once it is set. */ + if (!prog->aux->st_ops) + return 0; + if (bpf_prog_is_binfmt_misc_ops(prog) && + !btf_id_set_contains(&bpf_fs_kfunc_lsm_only_ids, kfunc_id)) return 0; return -EACCES; } @@ -438,7 +491,13 @@ static const struct btf_kfunc_id_set bpf_fs_kfunc_set = { static int __init bpf_fs_kfuncs_init(void) { - return register_btf_kfunc_id_set(BPF_PROG_TYPE_LSM, &bpf_fs_kfunc_set); + int ret; + + ret = register_btf_kfunc_id_set(BPF_PROG_TYPE_LSM, &bpf_fs_kfunc_set); + if (ret || !IS_ENABLED(CONFIG_BINFMT_MISC_BPF)) + return ret; + return register_btf_kfunc_id_set(BPF_PROG_TYPE_STRUCT_OPS, + &bpf_fs_kfunc_set); } late_initcall(bpf_fs_kfuncs_init); diff --git a/fs/btrfs/dev-replace.c b/fs/btrfs/dev-replace.c index 318ddb790429..dc0834f920c3 100644 --- a/fs/btrfs/dev-replace.c +++ b/fs/btrfs/dev-replace.c @@ -247,8 +247,8 @@ static int btrfs_init_dev_replace_tgtdev(struct btrfs_fs_info *fs_info, return -EINVAL; } - bdev_file = bdev_file_open_by_path(device_path, BLK_OPEN_WRITE, - fs_info->sb, &fs_holder_ops); + /* Unfreezable for the whole replace; see btrfs_dev_replace_start(). */ + bdev_file = btrfs_open_device_deny_freeze(device_path, fs_info->sb); if (IS_ERR(bdev_file)) { btrfs_err(fs_info, "target device %s is invalid!", device_path); return PTR_ERR(bdev_file); @@ -327,7 +327,8 @@ static int btrfs_init_dev_replace_tgtdev(struct btrfs_fs_info *fs_info, return 0; error: - bdev_fput(bdev_file); + /* Undo the open-time freeze deny. */ + btrfs_release_device_allow_freeze(bdev_file); return ret; } @@ -624,6 +625,15 @@ static int btrfs_dev_replace_start(struct btrfs_fs_info *fs_info, if (ret) return ret; + /* Deny the source before mark, so every 'leave' unwinds both denied. */ + if (src_device->bdev) { + ret = bdev_deny_freeze(src_device->bdev); + if (ret) { + btrfs_destroy_dev_replace_tgtdev(tgt_device, true); + return ret; + } + } + ret = mark_block_group_to_copy(fs_info, src_device); if (ret) return ret; @@ -708,7 +718,9 @@ static int btrfs_dev_replace_start(struct btrfs_fs_info *fs_info, return ret; leave: - btrfs_destroy_dev_replace_tgtdev(tgt_device); + if (src_device->bdev) + bdev_allow_freeze(src_device->bdev); + btrfs_destroy_dev_replace_tgtdev(tgt_device, true); return ret; } @@ -889,6 +901,7 @@ static int btrfs_dev_replace_finishing(struct btrfs_fs_info *fs_info, */ ret = btrfs_start_delalloc_roots(fs_info, LONG_MAX, false); if (ret) { + /* Stays started/resumable; keep both denied. */ mutex_unlock(&dev_replace->lock_finishing_cancel_unmount); return ret; } @@ -902,6 +915,7 @@ static int btrfs_dev_replace_finishing(struct btrfs_fs_info *fs_info, while (1) { trans = btrfs_start_transaction(root, 0); if (IS_ERR(trans)) { + /* Stays started/resumable; keep both denied. */ mutex_unlock(&dev_replace->lock_finishing_cancel_unmount); return PTR_ERR(trans); } @@ -954,7 +968,10 @@ error: mutex_unlock(&fs_devices->device_list_mutex); btrfs_rm_dev_replace_blocked(fs_info); if (tgt_device) - btrfs_destroy_dev_replace_tgtdev(tgt_device); + btrfs_destroy_dev_replace_tgtdev(tgt_device, true); + /* The source stays a member; re-allow freezing it. */ + if (src_device->bdev) + bdev_allow_freeze(src_device->bdev); btrfs_rm_dev_replace_unblocked(fs_info); mutex_unlock(&dev_replace->lock_finishing_cancel_unmount); @@ -1027,6 +1044,8 @@ error: mutex_unlock(&dev_replace->lock_finishing_cancel_unmount); + /* The target is now a member; the source is freed (allow + release). */ + bdev_allow_freeze(tgt_device->bdev); btrfs_rm_dev_replace_free_srcdev(src_device); return 0; @@ -1155,8 +1174,9 @@ int btrfs_dev_replace_cancel(struct btrfs_fs_info *fs_info) btrfs_dev_name(src_device), src_device->devid, btrfs_dev_name(tgt_device)); + /* A suspended replace never re-denied freezing; do not allow. */ if (tgt_device) - btrfs_destroy_dev_replace_tgtdev(tgt_device); + btrfs_destroy_dev_replace_tgtdev(tgt_device, false); break; default: up_write(&dev_replace->rwsem); @@ -1186,6 +1206,11 @@ void btrfs_dev_replace_suspend_for_unmount(struct btrfs_fs_info *fs_info) dev_replace->time_stopped = ktime_get_real_seconds(); dev_replace->item_needs_writeback = 1; btrfs_info(fs_info, "suspending dev_replace for unmount"); + /* Reopened freezable next mount; resume re-denies. */ + if (dev_replace->srcdev && dev_replace->srcdev->bdev) + bdev_allow_freeze(dev_replace->srcdev->bdev); + if (dev_replace->tgtdev && dev_replace->tgtdev->bdev) + bdev_allow_freeze(dev_replace->tgtdev->bdev); break; } @@ -1198,6 +1223,7 @@ int btrfs_resume_dev_replace_async(struct btrfs_fs_info *fs_info) { struct task_struct *task; struct btrfs_dev_replace *dev_replace = &fs_info->dev_replace; + int ret = 0; down_write(&dev_replace->rwsem); @@ -1241,8 +1267,33 @@ int btrfs_resume_dev_replace_async(struct btrfs_fs_info *fs_info) return 0; } + /* Re-deny for the resumed replace; stay suspended if frozen now. */ + if (dev_replace->srcdev->bdev && + bdev_deny_freeze(dev_replace->srcdev->bdev)) + goto suspend; + if (bdev_deny_freeze(dev_replace->tgtdev->bdev)) { + if (dev_replace->srcdev->bdev) + bdev_allow_freeze(dev_replace->srcdev->bdev); + goto suspend; + } + task = kthread_run(btrfs_dev_replace_kthread, fs_info, "btrfs-devrepl"); - return PTR_ERR_OR_ZERO(task); + if (IS_ERR(task)) { + bdev_allow_freeze(dev_replace->tgtdev->bdev); + if (dev_replace->srcdev->bdev) + bdev_allow_freeze(dev_replace->srcdev->bdev); + /* Undo the deny and suspend, but still fail the mount. */ + ret = PTR_ERR(task); + goto suspend; + } + return 0; + +suspend: + btrfs_exclop_finish(fs_info); + down_write(&dev_replace->rwsem); + dev_replace->replace_state = BTRFS_IOCTL_DEV_REPLACE_STATE_SUSPENDED; + up_write(&dev_replace->rwsem); + return ret; } static int btrfs_dev_replace_kthread(void *data) diff --git a/fs/btrfs/direct-io.c b/fs/btrfs/direct-io.c index ed1779ccb4de..3075d7992713 100644 --- a/fs/btrfs/direct-io.c +++ b/fs/btrfs/direct-io.c @@ -804,9 +804,11 @@ static void btrfs_dio_submit_io(const struct iomap_iter *iter, struct bio *bio, btrfs_submit_bbio(bbio, 0); } +static DEFINE_IOMAP_ITER_NEXT_END(btrfs_dio_iomap_next, btrfs_dio_iomap_begin, + btrfs_dio_iomap_end); + static const struct iomap_ops btrfs_dio_iomap_ops = { - .iomap_begin = btrfs_dio_iomap_begin, - .iomap_end = btrfs_dio_iomap_end, + .iomap_next = btrfs_dio_iomap_next, }; static const struct iomap_dio_ops btrfs_dio_ops = { diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 50c6640543b9..3c10a0ef0002 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -7019,7 +7019,7 @@ static int btrfs_mknod(struct mnt_idmap *idmap, struct inode *dir, } static int btrfs_create(struct mnt_idmap *idmap, struct inode *dir, - struct dentry *dentry, umode_t mode, bool excl) + struct dentry *dentry, umode_t mode) { struct inode *inode; @@ -7123,7 +7123,7 @@ static struct dentry *btrfs_mkdir(struct mnt_idmap *idmap, struct inode *dir, inode = new_inode(dir->i_sb); if (!inode) return ERR_PTR(-ENOMEM); - inode_init_owner(idmap, inode, dir, S_IFDIR | mode); + inode_init_owner(idmap, inode, dir, mode); inode->i_op = &btrfs_dir_inode_operations; inode->i_fop = &btrfs_dir_file_operations; return ERR_PTR(btrfs_create_common(dir, dentry, inode)); diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c index ebfb258161c8..72bc9d4f7708 100644 --- a/fs/btrfs/ioctl.c +++ b/fs/btrfs/ioctl.c @@ -2627,7 +2627,7 @@ static long btrfs_ioctl_rm_dev_v2(struct file *file, void __user *arg) err_drop: mnt_drop_write_file(file); if (bdev_file) - bdev_fput(bdev_file); + btrfs_release_device_allow_freeze(bdev_file); out: btrfs_put_dev_args_from_path(&args); return ret; @@ -2677,7 +2677,7 @@ static long btrfs_ioctl_rm_dev(struct file *file, void __user *arg) mnt_drop_write_file(file); if (bdev_file) - bdev_fput(bdev_file); + btrfs_release_device_allow_freeze(bdev_file); out: btrfs_put_dev_args_from_path(&args); return ret; diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c index a8e27db8e4bc..9b66eb584ece 100644 --- a/fs/btrfs/volumes.c +++ b/fs/btrfs/volumes.c @@ -12,6 +12,7 @@ #include #include #include +#include #include "misc.h" #include "disk-io.h" #include "extent-tree.h" @@ -480,7 +481,12 @@ btrfs_get_bdev_and_sb(const char *device_path, blk_mode_t flags, void *holder, struct block_device *bdev; int ret; - *bdev_file = bdev_file_open_by_path(device_path, flags, holder, &fs_holder_ops); + if (holder) + *bdev_file = fs_bdev_file_open_by_path(device_path, flags, + holder, holder); + else + *bdev_file = bdev_file_open_by_path(device_path, flags, NULL, + NULL); if (IS_ERR(*bdev_file)) { ret = PTR_ERR(*bdev_file); @@ -495,7 +501,7 @@ btrfs_get_bdev_and_sb(const char *device_path, blk_mode_t flags, void *holder, if (holder) { ret = set_blocksize(*bdev_file, BTRFS_BDEV_BLOCKSIZE); if (ret) { - bdev_fput(*bdev_file); + fs_bdev_file_release(*bdev_file, holder); goto error; } } @@ -503,7 +509,10 @@ btrfs_get_bdev_and_sb(const char *device_path, blk_mode_t flags, void *holder, *disk_super = btrfs_read_disk_super(bdev, 0, false); if (IS_ERR(*disk_super)) { ret = PTR_ERR(*disk_super); - bdev_fput(*bdev_file); + if (holder) + fs_bdev_file_release(*bdev_file, holder); + else + bdev_fput(*bdev_file); goto error; } @@ -727,7 +736,7 @@ static int btrfs_open_one_device(struct btrfs_fs_devices *fs_devices, error_free_page: btrfs_release_disk_super(disk_super); - bdev_fput(bdev_file); + fs_bdev_file_release(bdev_file, holder); return -EINVAL; } @@ -1052,7 +1061,7 @@ static void __btrfs_free_extra_devids(struct btrfs_fs_devices *fs_devices, continue; if (device->bdev_file) { - bdev_fput(device->bdev_file); + fs_bdev_file_release(device->bdev_file, device->bdev_file->private_data); device->bdev = NULL; device->bdev_file = NULL; fs_devices->open_devices--; @@ -1089,7 +1098,18 @@ void btrfs_free_extra_devids(struct btrfs_fs_devices *fs_devices) mutex_unlock(&uuid_mutex); } -static void btrfs_close_bdev(struct btrfs_device *device) +/* Release a device that was made unfreezable for a membership change. */ +void btrfs_release_device_allow_freeze(struct file *bdev_file) +{ + struct super_block *sb = bdev_file->private_data; + + /* Unregister before re-allowing (strand-safe); file still open (UAF-safe). */ + fs_bdev_unregister(bdev_file, sb); + bdev_allow_freeze(file_bdev(bdev_file)); + bdev_fput(bdev_file); +} + +static void btrfs_close_bdev(struct btrfs_device *device, bool allow_freeze) { if (!device->bdev) return; @@ -1099,7 +1119,12 @@ static void btrfs_close_bdev(struct btrfs_device *device) invalidate_bdev(device->bdev); } - bdev_fput(device->bdev_file); + /* @allow_freeze undoes a replace-time deny; unmount-close was never denied. */ + if (allow_freeze) + btrfs_release_device_allow_freeze(device->bdev_file); + else + fs_bdev_file_release(device->bdev_file, + device->bdev_file->private_data); } static void btrfs_close_one_device(struct btrfs_device *device) @@ -1120,7 +1145,7 @@ static void btrfs_close_one_device(struct btrfs_device *device) fs_devices->missing_devices--; } - btrfs_close_bdev(device); + btrfs_close_bdev(device, false); if (device->bdev) { fs_devices->open_devices--; device->bdev = NULL; @@ -2090,8 +2115,16 @@ static int btrfs_add_dev_item(struct btrfs_trans_handle *trans, static void update_dev_time(const char *device_path) { struct path path; + int err; - if (!kern_path(device_path, LOOKUP_FOLLOW, &path)) { + if (tsk_is_kthread(current)) { + scoped_with_init_fs() + err = kern_path(device_path, LOOKUP_FOLLOW, &path); + } else { + err = kern_path(device_path, LOOKUP_FOLLOW, &path); + } + + if (!err) { vfs_utimes(&path, NULL); path_put(&path); } @@ -2338,6 +2371,13 @@ int btrfs_rm_device(struct btrfs_fs_info *fs_info, fs_info->fs_devices->rw_devices == 1) return BTRFS_ERROR_DEV_ONLY_WRITABLE; + /* Removal and freezing are mutually exclusive; refuse if frozen now. */ + if (device->bdev) { + ret = bdev_deny_freeze(device->bdev); + if (ret) + return ret; + } + if (test_bit(BTRFS_DEV_STATE_WRITEABLE, &device->dev_state)) { mutex_lock(&fs_info->chunk_mutex); list_del_init(&device->dev_alloc_list); @@ -2364,6 +2404,8 @@ int btrfs_rm_device(struct btrfs_fs_info *fs_info, device->devid, ret); btrfs_abort_transaction(trans, ret); btrfs_end_transaction(trans); + if (device->bdev) + bdev_allow_freeze(device->bdev); return ret; } @@ -2455,6 +2497,8 @@ int btrfs_rm_device(struct btrfs_fs_info *fs_info, return btrfs_commit_transaction(trans); error_undo: + if (device->bdev) + bdev_allow_freeze(device->bdev); if (test_bit(BTRFS_DEV_STATE_WRITEABLE, &device->dev_state)) { mutex_lock(&fs_info->chunk_mutex); list_add(&device->dev_alloc_list, @@ -2499,7 +2543,8 @@ void btrfs_rm_dev_replace_free_srcdev(struct btrfs_device *srcdev) mutex_lock(&uuid_mutex); - btrfs_close_bdev(srcdev); + /* The source was made unfreezable for the replace; undo it. */ + btrfs_close_bdev(srcdev, true); synchronize_rcu(); btrfs_free_device(srcdev); @@ -2520,7 +2565,8 @@ void btrfs_rm_dev_replace_free_srcdev(struct btrfs_device *srcdev) mutex_unlock(&uuid_mutex); } -void btrfs_destroy_dev_replace_tgtdev(struct btrfs_device *tgtdev) +void btrfs_destroy_dev_replace_tgtdev(struct btrfs_device *tgtdev, + bool allow_freeze) { struct btrfs_fs_devices *fs_devices = tgtdev->fs_info->fs_devices; @@ -2541,7 +2587,7 @@ void btrfs_destroy_dev_replace_tgtdev(struct btrfs_device *tgtdev) btrfs_scratch_superblocks(tgtdev->fs_info, tgtdev); - btrfs_close_bdev(tgtdev); + btrfs_close_bdev(tgtdev, allow_freeze); synchronize_rcu(); btrfs_free_device(tgtdev); } @@ -2810,6 +2856,37 @@ next_slot: return 0; } +/* + * Open @path for @sb with freezing denied before the holder claim is published, + * so a racing bdev_freeze() can never reach a claim a device add or replace may + * still abort. The deny is taken on a throwaway non-holder probe open, then the + * holder is opened by the probe's dev_t. Balanced by the caller. + */ +struct file *btrfs_open_device_deny_freeze(const char *path, + struct super_block *sb) +{ + struct file *probe_file, *bdev_file; + int ret; + + /* WRITE so bdev_file_open_by_path() rejects a read-only device. */ + probe_file = bdev_file_open_by_path(path, BLK_OPEN_WRITE, NULL, NULL); + if (IS_ERR(probe_file)) + return probe_file; + + ret = bdev_deny_freeze(file_bdev(probe_file)); + if (ret) { + bdev_fput(probe_file); + return ERR_PTR(ret); + } + + bdev_file = fs_bdev_file_open_by_dev(file_bdev(probe_file)->bd_dev, + BLK_OPEN_WRITE, sb, sb); + if (IS_ERR(bdev_file)) + bdev_allow_freeze(file_bdev(probe_file)); + bdev_fput(probe_file); + return bdev_file; +} + int btrfs_init_new_device(struct btrfs_fs_info *fs_info, const char *device_path) { struct btrfs_root *root = fs_info->dev_root; @@ -2828,8 +2905,8 @@ int btrfs_init_new_device(struct btrfs_fs_info *fs_info, const char *device_path if (sb_rdonly(sb) && !fs_devices->seeding) return -EROFS; - bdev_file = bdev_file_open_by_path(device_path, BLK_OPEN_WRITE, - fs_info->sb, &fs_holder_ops); + /* Forbid freezing until the device is a committed member (or unwound). */ + bdev_file = btrfs_open_device_deny_freeze(device_path, fs_info->sb); if (IS_ERR(bdev_file)) return PTR_ERR(bdev_file); @@ -3000,8 +3077,10 @@ int btrfs_init_new_device(struct btrfs_fs_info *fs_info, const char *device_path up_write(&sb->s_umount); locked = false; - if (ret) /* transaction commit */ + if (ret) { /* transaction commit */ + bdev_allow_freeze(file_bdev(bdev_file)); return ret; + } ret = btrfs_relocate_sys_chunks(fs_info); if (ret < 0) @@ -3009,8 +3088,10 @@ int btrfs_init_new_device(struct btrfs_fs_info *fs_info, const char *device_path "Failed to relocate sys chunks after device initialization. This can be fixed using the \"btrfs balance\" command."); trans = btrfs_attach_transaction(root); if (IS_ERR(trans)) { - if (PTR_ERR(trans) == -ENOENT) + if (PTR_ERR(trans) == -ENOENT) { + bdev_allow_freeze(file_bdev(bdev_file)); return 0; + } ret = PTR_ERR(trans); trans = NULL; goto error_sysfs; @@ -3030,6 +3111,7 @@ int btrfs_init_new_device(struct btrfs_fs_info *fs_info, const char *device_path /* Update ctime/mtime for blkid or udev */ update_dev_time(device_path); + bdev_allow_freeze(file_bdev(bdev_file)); return ret; error_sysfs: @@ -3059,7 +3141,7 @@ error_free_zone: error_free_device: btrfs_free_device(device); error: - bdev_fput(bdev_file); + btrfs_release_device_allow_freeze(bdev_file); if (locked) { mutex_unlock(&uuid_mutex); up_write(&sb->s_umount); diff --git a/fs/btrfs/volumes.h b/fs/btrfs/volumes.h index eaf23c0dcbf6..0415d74cad9b 100644 --- a/fs/btrfs/volumes.h +++ b/fs/btrfs/volumes.h @@ -750,6 +750,7 @@ int btrfs_open_devices(struct btrfs_fs_devices *fs_devices, struct btrfs_device *btrfs_scan_one_device(const char *path, bool mount_arg_dev); int btrfs_forget_devices(dev_t devt); void btrfs_close_devices(struct btrfs_fs_devices *fs_devices); +void btrfs_release_device_allow_freeze(struct file *bdev_file); void btrfs_free_extra_devids(struct btrfs_fs_devices *fs_devices); void btrfs_assign_next_active_device(struct btrfs_device *device, struct btrfs_device *this_dev); @@ -774,6 +775,8 @@ struct btrfs_device *btrfs_find_device(const struct btrfs_fs_devices *fs_devices const struct btrfs_dev_lookup_args *args); int btrfs_shrink_device(struct btrfs_device *device, u64 new_size); int btrfs_init_new_device(struct btrfs_fs_info *fs_info, const char *path); +struct file *btrfs_open_device_deny_freeze(const char *path, + struct super_block *sb); int btrfs_balance(struct btrfs_fs_info *fs_info, struct btrfs_balance_control *bctl, struct btrfs_ioctl_balance_args *bargs); @@ -794,7 +797,8 @@ int btrfs_init_writeback_bio_size(struct btrfs_fs_info *fs_info); int btrfs_run_dev_stats(struct btrfs_trans_handle *trans); void btrfs_rm_dev_replace_remove_srcdev(struct btrfs_device *srcdev); void btrfs_rm_dev_replace_free_srcdev(struct btrfs_device *srcdev); -void btrfs_destroy_dev_replace_tgtdev(struct btrfs_device *tgtdev); +void btrfs_destroy_dev_replace_tgtdev(struct btrfs_device *tgtdev, + bool allow_freeze); unsigned long btrfs_full_stripe_len(struct btrfs_fs_info *fs_info, u64 logical); u64 btrfs_calc_stripe_length(const struct btrfs_chunk_map *map); diff --git a/fs/buffer.c b/fs/buffer.c index b62e7de421b5..dd50d17b8907 100644 --- a/fs/buffer.c +++ b/fs/buffer.c @@ -599,79 +599,6 @@ int mmb_sync(struct mapping_metadata_bhs *mmb) } EXPORT_SYMBOL(mmb_sync); -/** - * mmb_fsync_noflush - fsync implementation for simple filesystems with - * metadata buffers list - * - * @file: file to synchronize - * @mmb: list of metadata bhs to flush - * @start: start offset in bytes - * @end: end offset in bytes (inclusive) - * @datasync: only synchronize essential metadata if true - * - * This is an implementation of the fsync method for simple filesystems which - * track all non-inode metadata in the buffers list hanging off the @mmb - * structure. - */ -int mmb_fsync_noflush(struct file *file, struct mapping_metadata_bhs *mmb, - loff_t start, loff_t end, bool datasync) -{ - struct inode *inode = file->f_mapping->host; - int err; - int ret = 0; - - err = file_write_and_wait_range(file, start, end); - if (err) - return err; - - if (mmb) - ret = mmb_sync(mmb); - if (!(inode_state_read_once(inode) & I_DIRTY_ALL)) - goto out; - if (datasync && !(inode_state_read_once(inode) & I_DIRTY_DATASYNC)) - goto out; - - err = sync_inode_metadata(inode, 1); - if (ret == 0) - ret = err; - -out: - /* check and advance again to catch errors after syncing out buffers */ - err = file_check_and_advance_wb_err(file); - if (ret == 0) - ret = err; - return ret; -} -EXPORT_SYMBOL(mmb_fsync_noflush); - -/** - * mmb_fsync - fsync implementation for simple filesystems with metadata - * buffers list - * - * @file: file to synchronize - * @mmb: list of metadata bhs to flush - * @start: start offset in bytes - * @end: end offset in bytes (inclusive) - * @datasync: only synchronize essential metadata if true - * - * This is an implementation of the fsync method for simple filesystems which - * track all non-inode metadata in the buffers list hanging off the @mmb - * structure. This also makes sure that a device cache flush operation is - * called at the end. - */ -int mmb_fsync(struct file *file, struct mapping_metadata_bhs *mmb, - loff_t start, loff_t end, bool datasync) -{ - struct inode *inode = file->f_mapping->host; - int ret; - - ret = mmb_fsync_noflush(file, mmb, start, end, datasync); - if (!ret) - ret = blkdev_issue_flush(inode->i_sb->s_bdev); - return ret; -} -EXPORT_SYMBOL(mmb_fsync); - /* * Called when we've recently written block `bblock', and it is known that * `bblock' was for a buffer_boundary() buffer. This means that the block at @@ -1094,12 +1021,18 @@ EXPORT_SYMBOL(mark_buffer_dirty); void mark_buffer_write_io_error(struct buffer_head *bh) { + struct mapping_metadata_bhs *mmb; + set_buffer_write_io_error(bh); /* FIXME: do we need to set this in both places? */ if (bh->b_folio && bh->b_folio->mapping) mapping_set_error(bh->b_folio->mapping, -EIO); - if (bh->b_mmb) - mapping_set_error(bh->b_mmb->mapping, -EIO); + /* Protect us from mmb & inode getting freed while we work on it */ + rcu_read_lock(); + mmb = READ_ONCE(bh->b_mmb); + if (mmb) + mapping_set_error(mmb->mapping, -EIO); + rcu_read_unlock(); } EXPORT_SYMBOL(mark_buffer_write_io_error); diff --git a/fs/ceph/dir.c b/fs/ceph/dir.c index ef9e92e362d3..4a5e0290f2e3 100644 --- a/fs/ceph/dir.c +++ b/fs/ceph/dir.c @@ -983,7 +983,7 @@ out: } static int ceph_create(struct mnt_idmap *idmap, struct inode *dir, - struct dentry *dentry, umode_t mode, bool excl) + struct dentry *dentry, umode_t mode) { return ceph_mknod(idmap, dir, dentry, mode, 0); } @@ -1147,7 +1147,6 @@ static struct dentry *ceph_mkdir(struct mnt_idmap *idmap, struct inode *dir, goto out; } - mode |= S_IFDIR; req->r_new_inode = ceph_new_inode(dir, dentry, &mode, &as_ctx); if (IS_ERR(req->r_new_inode)) { ret = ERR_CAST(req->r_new_inode); @@ -1673,7 +1672,7 @@ __dentry_leases_walk(struct ceph_mds_client *mdsc, if (!spin_trylock(&dentry->d_lock)) continue; - if (__lockref_is_dead(&dentry->d_lockref)) { + if (lockref_is_dead(&dentry->d_lockref)) { list_del_init(&di->lease_list); goto next; } diff --git a/fs/coda/dir.c b/fs/coda/dir.c index 835eb7fdfdad..67148edfadee 100644 --- a/fs/coda/dir.c +++ b/fs/coda/dir.c @@ -134,7 +134,7 @@ static inline void coda_dir_drop_nlink(struct inode *dir) /* creation routines: create, mknod, mkdir, link, symlink */ static int coda_create(struct mnt_idmap *idmap, struct inode *dir, - struct dentry *de, umode_t mode, bool excl) + struct dentry *de, umode_t mode) { int error; const char *name=de->d_name.name; @@ -179,7 +179,12 @@ static struct dentry *coda_mkdir(struct mnt_idmap *idmap, struct inode *dir, if (is_root_inode(dir) && coda_iscontrol(name, len)) return ERR_PTR(-EPERM); - attrs.va_mode = mode; + /* + * vfs_mkdir() now passes S_IFDIR in @mode, but @mode is forwarded + * verbatim to userspace, which has only ever been given the permission + * bits. Strip the type bit until venus is known to cope with it. + */ + attrs.va_mode = mode & ~S_IFDIR; error = venus_mkdir(dir->i_sb, coda_i2f(dir), name, len, &newfid, &attrs); if (error) diff --git a/fs/coredump.c b/fs/coredump.c index e68a76ff92a3..ac3cd74808c6 100644 --- a/fs/coredump.c +++ b/fs/coredump.c @@ -921,15 +921,10 @@ static bool coredump_file(struct core_name *cn, struct coredump_params *cprm, * with a fully qualified path" rule is to control where * coredumps may be placed using root privileges, * current->fs->root must not be used. Instead, use the - * root directory of init_task. + * root directory of PID 1. */ - struct path root; - - task_lock(&init_task); - get_fs_root(init_task.fs, &root); - task_unlock(&init_task); - file = file_open_root(&root, cn->corename, open_flags, 0600); - path_put(&root); + scoped_with_init_fs() + file = filp_open(cn->corename, open_flags, 0600); } else { file = filp_open(cn->corename, open_flags, 0600); } diff --git a/fs/cramfs/inode.c b/fs/cramfs/inode.c index 4edbfccd0bbe..d4cd03f4f60d 100644 --- a/fs/cramfs/inode.c +++ b/fs/cramfs/inode.c @@ -504,7 +504,7 @@ static void cramfs_kill_sb(struct super_block *sb) sb->s_mtd = NULL; } else if (IS_ENABLED(CONFIG_CRAMFS_BLOCKDEV) && sb->s_bdev) { sync_blockdev(sb->s_bdev); - bdev_fput(sb->s_bdev_file); + fs_bdev_file_release(sb->s_bdev_file, sb); } kfree(sbi); } diff --git a/fs/d_path.c b/fs/d_path.c index a48957c0971e..c25309006d5d 100644 --- a/fs/d_path.c +++ b/fs/d_path.c @@ -279,7 +279,8 @@ char *d_path(const struct path *path, char *buf, int buflen) * and instead have d_path return the mounted path. */ if (path->dentry->d_op && path->dentry->d_op->d_dname && - (!IS_ROOT(path->dentry) || path->dentry != path->mnt->mnt_root)) + (!IS_ROOT(path->dentry) || path->dentry != path->mnt->mnt_root || + failfs_mnt(path->mnt))) return path->dentry->d_op->d_dname(path->dentry, buf, buflen); rcu_read_lock(); diff --git a/fs/dcache.c b/fs/dcache.c index 3e9af9de7074..1b1a81f10da6 100644 --- a/fs/dcache.c +++ b/fs/dcache.c @@ -434,7 +434,7 @@ static inline void __d_clear_type_and_inode(struct dentry *dentry) static void dentry_free(struct dentry *dentry) { DENTRY_WARN_ONCE(d_really_is_positive(dentry), dentry); - DENTRY_WARN_ONCE(dentry->d_lockref.count >= 0, dentry); + DENTRY_WARN_ONCE(!lockref_is_dead(&dentry->d_lockref), dentry); D_FLAG_VERIFY(dentry, 0); if (unlikely(dname_external(dentry))) { struct external_name *p = external_name(dentry); @@ -782,7 +782,7 @@ static bool lock_for_kill(struct dentry *dentry) * * If @dentry is idle and remains such after we assemble the full * locking environment for eviction (see lock_for_kill() for details) - * we mark it doomed (->d_lockref.count < 0) and proceed to detaching + * we mark it doomed (see lockref_mark_dead()) and proceed to detaching * it from any filesystem objects. Otherwise we drop ->d_lock and * return %NULL. * @@ -946,7 +946,7 @@ static inline bool fast_dput(struct dentry *dentry) if (unlikely(ret < 0)) { spin_lock(&dentry->d_lock); rcu_read_unlock(); - if (WARN_ON_ONCE(dentry->d_lockref.count <= 0)) { + if (WARN_ON_ONCE(lockref_is_dead_or_zero(&dentry->d_lockref))) { spin_unlock(&dentry->d_lock); return true; } @@ -1644,7 +1644,7 @@ static enum d_walk_ret select_collect(void *_data, struct dentry *dentry) if (data->start == dentry) goto out; - if (dentry->d_lockref.count <= 0) { + if (lockref_is_dead_or_zero(&dentry->d_lockref)) { __move_to_shrink_list(dentry, &data->dispose); data->found++; } @@ -1676,7 +1676,7 @@ static enum d_walk_ret select_collect2(void *_data, struct dentry *dentry) if (data->start == dentry) goto out; - if (dentry->d_lockref.count <= 0) { + if (lockref_is_dead_or_zero(&dentry->d_lockref)) { if (!__move_to_shrink_list(dentry, &data->dispose)) { /* * We need an enter RCU read-side critical area that @@ -1747,7 +1747,7 @@ static void shrink_dcache_tree(struct dentry *parent, bool for_umount) spin_lock(&v->d_lock); rcu_read_unlock(); - if (unlikely(v->d_lockref.count < 0)) { + if (unlikely(lockref_is_dead(&v->d_lockref))) { // It's doomed; if it isn't dead yet, notify us // once it becomes invisible to d_walk(). need_wait = d_add_waiter(v, &wait); @@ -1794,7 +1794,12 @@ static void do_one_tree(struct dentry *dentry) { shrink_dcache_tree(dentry, true); d_walk(dentry, dentry, umount_check); - d_drop(dentry); + spin_lock(&dentry->d_lock); + __d_drop(dentry); + /* A busy root survives the dput() below so don't leave it on ->s_roots. */ + if (unlikely(!hlist_unhashed(&dentry->d_sib))) + unlink_secondary_root(dentry); + spin_unlock(&dentry->d_lock); dput(dentry); } @@ -1823,7 +1828,7 @@ void shrink_dcache_for_umount(struct super_block *sb) spin_unlock(&sb->s_roots_lock); spin_lock(&dentry->d_lock); rcu_read_unlock(); - if (unlikely(dentry->d_lockref.count < 0)) { + if (unlikely(lockref_is_dead(&dentry->d_lockref))) { struct completion_list wait; bool need_wait = d_add_waiter(dentry, &wait); @@ -2822,7 +2827,7 @@ retry: spin_lock(&dentry->d_lock); rcu_read_unlock(); /* now we can try to grab a reference */ - if (unlikely(dentry->d_lockref.count < 0)) { + if (unlikely(lockref_is_dead(&dentry->d_lockref))) { spin_unlock(&dentry->d_lock); goto retry; } diff --git a/fs/ecryptfs/inode.c b/fs/ecryptfs/inode.c index 7aaf1913f9c6..525297c7ebd8 100644 --- a/fs/ecryptfs/inode.c +++ b/fs/ecryptfs/inode.c @@ -268,7 +268,7 @@ out: static int ecryptfs_create(struct mnt_idmap *idmap, struct inode *directory_inode, struct dentry *ecryptfs_dentry, - umode_t mode, bool excl) + umode_t mode) { struct inode *ecryptfs_inode; int rc; diff --git a/fs/efivarfs/inode.c b/fs/efivarfs/inode.c index 95dcad83da11..f0d009555fc6 100644 --- a/fs/efivarfs/inode.c +++ b/fs/efivarfs/inode.c @@ -75,7 +75,7 @@ static bool efivarfs_valid_name(const char *str, int len) } static int efivarfs_create(struct mnt_idmap *idmap, struct inode *dir, - struct dentry *dentry, umode_t mode, bool excl) + struct dentry *dentry, umode_t mode) { struct inode *inode = NULL; struct efivar_entry *var; diff --git a/fs/efs/Kconfig b/fs/efs/Kconfig deleted file mode 100644 index 0833e533df9d..000000000000 --- a/fs/efs/Kconfig +++ /dev/null @@ -1,16 +0,0 @@ -# SPDX-License-Identifier: GPL-2.0-only -config EFS_FS - tristate "EFS file system support (read only)" - depends on BLOCK - select BUFFER_HEAD - help - EFS is an older file system used for non-ISO9660 CD-ROMs and hard - disk partitions by SGI's IRIX operating system (IRIX 6.0 and newer - uses the XFS file system for hard disk partitions however). - - This implementation only offers read-only access. If you don't know - what all this is about, it's safe to say N. For more information - about EFS see its home page at . - - To compile the EFS file system support as a module, choose M here: the - module will be called efs. diff --git a/fs/efs/Makefile b/fs/efs/Makefile deleted file mode 100644 index 85e5b88f9471..000000000000 --- a/fs/efs/Makefile +++ /dev/null @@ -1,8 +0,0 @@ -# SPDX-License-Identifier: GPL-2.0-only -# -# Makefile for the linux efs-filesystem routines. -# - -obj-$(CONFIG_EFS_FS) += efs.o - -efs-objs := super.o inode.o namei.o dir.o file.o symlink.o diff --git a/fs/efs/dir.c b/fs/efs/dir.c deleted file mode 100644 index 35ad0092c115..000000000000 --- a/fs/efs/dir.c +++ /dev/null @@ -1,105 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * dir.c - * - * Copyright (c) 1999 Al Smith - */ - -#include -#include -#include "efs.h" - -static int efs_readdir(struct file *, struct dir_context *); - -const struct file_operations efs_dir_operations = { - .llseek = generic_file_llseek, - .read = generic_read_dir, - .iterate_shared = efs_readdir, - .setlease = generic_setlease, -}; - -const struct inode_operations efs_dir_inode_operations = { - .lookup = efs_lookup, -}; - -static int efs_readdir(struct file *file, struct dir_context *ctx) -{ - struct inode *inode = file_inode(file); - efs_block_t block; - int slot; - - if (inode->i_size & (EFS_DIRBSIZE-1)) - pr_warn("%s(): directory size not a multiple of EFS_DIRBSIZE\n", - __func__); - - /* work out where this entry can be found */ - block = ctx->pos >> EFS_DIRBSIZE_BITS; - - /* each block contains at most 256 slots */ - slot = ctx->pos & 0xff; - - /* look at all blocks */ - while (block < inode->i_blocks) { - struct efs_dir *dirblock; - struct buffer_head *bh; - - /* read the dir block */ - bh = sb_bread(inode->i_sb, efs_bmap(inode, block)); - - if (!bh) { - pr_err("%s(): failed to read dir block %d\n", - __func__, block); - break; - } - - dirblock = (struct efs_dir *) bh->b_data; - - if (be16_to_cpu(dirblock->magic) != EFS_DIRBLK_MAGIC) { - pr_err("%s(): invalid directory block\n", __func__); - brelse(bh); - break; - } - - for (; slot < dirblock->slots; slot++) { - struct efs_dentry *dirslot; - efs_ino_t inodenum; - const char *nameptr; - int namelen; - - if (dirblock->space[slot] == 0) - continue; - - dirslot = (struct efs_dentry *) (((char *) bh->b_data) + EFS_SLOTAT(dirblock, slot)); - - inodenum = be32_to_cpu(dirslot->inode); - namelen = dirslot->namelen; - nameptr = dirslot->name; - pr_debug("%s(): block %d slot %d/%d: inode %u, name \"%s\", namelen %u\n", - __func__, block, slot, dirblock->slots-1, - inodenum, nameptr, namelen); - if (!namelen) - continue; - /* found the next entry */ - ctx->pos = (block << EFS_DIRBSIZE_BITS) | slot; - - /* sanity check */ - if (nameptr - (char *) dirblock + namelen > EFS_DIRBSIZE) { - pr_warn("directory entry %d exceeds directory block\n", - slot); - continue; - } - - /* copy filename and data in dirslot */ - if (!dir_emit(ctx, nameptr, namelen, inodenum, DT_UNKNOWN)) { - brelse(bh); - return 0; - } - } - brelse(bh); - - slot = 0; - block++; - } - ctx->pos = (block << EFS_DIRBSIZE_BITS) | slot; - return 0; -} diff --git a/fs/efs/efs.h b/fs/efs/efs.h deleted file mode 100644 index 918d2b9abb76..000000000000 --- a/fs/efs/efs.h +++ /dev/null @@ -1,144 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 */ -/* - * Copyright (c) 1999 Al Smith, - * - * Portions derived from work (c) 1995,1996 Christian Vogelgsang. - * Portions derived from IRIX header files (c) 1988 Silicon Graphics - */ -#ifndef _EFS_EFS_H_ -#define _EFS_EFS_H_ - -#ifdef pr_fmt -#undef pr_fmt -#endif - -#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt - -#include -#include - -#define EFS_VERSION "1.0a" - -/* 1 block is 512 bytes */ -#define EFS_BLOCKSIZE_BITS 9 -#define EFS_BLOCKSIZE (1 << EFS_BLOCKSIZE_BITS) - -typedef int32_t efs_block_t; -typedef uint32_t efs_ino_t; - -#define EFS_DIRECTEXTENTS 12 - -/* - * layout of an extent, in memory and on disk. 8 bytes exactly. - */ -typedef union extent_u { - unsigned char raw[8]; - struct extent_s { - unsigned int ex_magic:8; /* magic # (zero) */ - unsigned int ex_bn:24; /* basic block */ - unsigned int ex_length:8; /* numblocks in this extent */ - unsigned int ex_offset:24; /* logical offset into file */ - } cooked; -} efs_extent; - -typedef struct edevs { - __be16 odev; - __be32 ndev; -} efs_devs; - -/* - * extent based filesystem inode as it appears on disk. The efs inode - * is exactly 128 bytes long. - */ -struct efs_dinode { - __be16 di_mode; /* mode and type of file */ - __be16 di_nlink; /* number of links to file */ - __be16 di_uid; /* owner's user id */ - __be16 di_gid; /* owner's group id */ - __be32 di_size; /* number of bytes in file */ - __be32 di_atime; /* time last accessed */ - __be32 di_mtime; /* time last modified */ - __be32 di_ctime; /* time created */ - __be32 di_gen; /* generation number */ - __be16 di_numextents; /* # of extents */ - u_char di_version; /* version of inode */ - u_char di_spare; /* spare - used by AFS */ - union di_addr { - efs_extent di_extents[EFS_DIRECTEXTENTS]; - efs_devs di_dev; /* device for IFCHR/IFBLK */ - } di_u; -}; - -/* efs inode storage in memory */ -struct efs_inode_info { - int numextents; - int lastextent; - - efs_extent extents[EFS_DIRECTEXTENTS]; - struct inode vfs_inode; -}; - -#include - -#define EFS_DIRBSIZE_BITS EFS_BLOCKSIZE_BITS -#define EFS_DIRBSIZE (1 << EFS_DIRBSIZE_BITS) - -struct efs_dentry { - __be32 inode; - unsigned char namelen; - char name[3]; -}; - -#define EFS_DENTSIZE (sizeof(struct efs_dentry) - 3 + 1) -#define EFS_MAXNAMELEN ((1 << (sizeof(char) * 8)) - 1) - -#define EFS_DIRBLK_HEADERSIZE 4 -#define EFS_DIRBLK_MAGIC 0xbeef /* moo */ - -struct efs_dir { - __be16 magic; - unsigned char firstused; - unsigned char slots; - - unsigned char space[EFS_DIRBSIZE - EFS_DIRBLK_HEADERSIZE]; -}; - -#define EFS_MAXENTS \ - ((EFS_DIRBSIZE - EFS_DIRBLK_HEADERSIZE) / \ - (EFS_DENTSIZE + sizeof(char))) - -#define EFS_SLOTAT(dir, slot) EFS_REALOFF((dir)->space[slot]) - -#define EFS_REALOFF(offset) ((offset << 1)) - - -static inline struct efs_inode_info *INODE_INFO(struct inode *inode) -{ - return container_of(inode, struct efs_inode_info, vfs_inode); -} - -static inline struct efs_sb_info *SUPER_INFO(struct super_block *sb) -{ - return sb->s_fs_info; -} - -struct statfs; -struct fid; - -extern const struct inode_operations efs_dir_inode_operations; -extern const struct file_operations efs_dir_operations; -extern const struct address_space_operations efs_symlink_aops; - -extern struct inode *efs_iget(struct super_block *, unsigned long); -extern efs_block_t efs_map_block(struct inode *, efs_block_t); -extern int efs_get_block(struct inode *, sector_t, struct buffer_head *, int); - -extern struct dentry *efs_lookup(struct inode *, struct dentry *, unsigned int); -extern struct dentry *efs_fh_to_dentry(struct super_block *sb, struct fid *fid, - int fh_len, int fh_type); -extern struct dentry *efs_fh_to_parent(struct super_block *sb, struct fid *fid, - int fh_len, int fh_type); -extern struct dentry *efs_get_parent(struct dentry *); -extern int efs_bmap(struct inode *, int); - -#endif /* _EFS_EFS_H_ */ diff --git a/fs/efs/file.c b/fs/efs/file.c deleted file mode 100644 index 9153dfe79bbc..000000000000 --- a/fs/efs/file.c +++ /dev/null @@ -1,42 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * file.c - * - * Copyright (c) 1999 Al Smith - * - * Portions derived from work (c) 1995,1996 Christian Vogelgsang. - */ - -#include -#include "efs.h" - -int efs_get_block(struct inode *inode, sector_t iblock, - struct buffer_head *bh_result, int create) -{ - int error = -EROFS; - long phys; - - if (create) - return error; - if (iblock >= inode->i_blocks) - return 0; - - phys = efs_map_block(inode, iblock); - if (phys) - map_bh(bh_result, inode->i_sb, phys); - return 0; -} - -int efs_bmap(struct inode *inode, efs_block_t block) { - - if (block < 0) { - pr_warn("%s(): block < 0\n", __func__); - return 0; - } - - /* are we about to read past the end of a file ? */ - if (!(block < inode->i_blocks)) - return 0; - - return efs_map_block(inode, block); -} diff --git a/fs/efs/inode.c b/fs/efs/inode.c deleted file mode 100644 index 4b132729e638..000000000000 --- a/fs/efs/inode.c +++ /dev/null @@ -1,315 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0-only -/* - * inode.c - * - * Copyright (c) 1999 Al Smith - * - * Portions derived from work (c) 1995,1996 Christian Vogelgsang, - * and from work (c) 1998 Mike Shaver. - */ - -#include -#include -#include -#include "efs.h" -#include - -static int efs_read_folio(struct file *file, struct folio *folio) -{ - return block_read_full_folio(folio, efs_get_block); -} - -static sector_t _efs_bmap(struct address_space *mapping, sector_t block) -{ - return generic_block_bmap(mapping,block,efs_get_block); -} - -static const struct address_space_operations efs_aops = { - .read_folio = efs_read_folio, - .bmap = _efs_bmap -}; - -static inline void extent_copy(efs_extent *src, efs_extent *dst) { - /* - * this is slightly evil. it doesn't just copy - * efs_extent from src to dst, it also mangles - * the bits so that dst ends up in cpu byte-order. - */ - - dst->cooked.ex_magic = (unsigned int) src->raw[0]; - dst->cooked.ex_bn = ((unsigned int) src->raw[1] << 16) | - ((unsigned int) src->raw[2] << 8) | - ((unsigned int) src->raw[3] << 0); - dst->cooked.ex_length = (unsigned int) src->raw[4]; - dst->cooked.ex_offset = ((unsigned int) src->raw[5] << 16) | - ((unsigned int) src->raw[6] << 8) | - ((unsigned int) src->raw[7] << 0); - return; -} - -struct inode *efs_iget(struct super_block *super, unsigned long ino) -{ - int i, inode_index; - dev_t device; - u32 rdev; - struct buffer_head *bh; - struct efs_sb_info *sb = SUPER_INFO(super); - struct efs_inode_info *in; - efs_block_t block, offset; - struct efs_dinode *efs_inode; - struct inode *inode; - - inode = iget_locked(super, ino); - if (!inode) - return ERR_PTR(-ENOMEM); - if (!(inode_state_read_once(inode) & I_NEW)) - return inode; - - in = INODE_INFO(inode); - - /* - ** EFS layout: - ** - ** | cylinder group | cylinder group | cylinder group ..etc - ** |inodes|data |inodes|data |inodes|data ..etc - ** - ** work out the inode block index, (considering initially that the - ** inodes are stored as consecutive blocks). then work out the block - ** number of that inode given the above layout, and finally the - ** offset of the inode within that block. - */ - - inode_index = inode->i_ino / - (EFS_BLOCKSIZE / sizeof(struct efs_dinode)); - - block = sb->fs_start + sb->first_block + - (sb->group_size * (inode_index / sb->inode_blocks)) + - (inode_index % sb->inode_blocks); - - offset = (inode->i_ino % - (EFS_BLOCKSIZE / sizeof(struct efs_dinode))) * - sizeof(struct efs_dinode); - - bh = sb_bread(inode->i_sb, block); - if (!bh) { - pr_warn("%s() failed at block %d\n", __func__, block); - goto read_inode_error; - } - - efs_inode = (struct efs_dinode *) (bh->b_data + offset); - - inode->i_mode = be16_to_cpu(efs_inode->di_mode); - set_nlink(inode, be16_to_cpu(efs_inode->di_nlink)); - i_uid_write(inode, (uid_t)be16_to_cpu(efs_inode->di_uid)); - i_gid_write(inode, (gid_t)be16_to_cpu(efs_inode->di_gid)); - inode->i_size = be32_to_cpu(efs_inode->di_size); - inode_set_atime(inode, be32_to_cpu(efs_inode->di_atime), 0); - inode_set_mtime(inode, be32_to_cpu(efs_inode->di_mtime), 0); - inode_set_ctime(inode, be32_to_cpu(efs_inode->di_ctime), 0); - - /* this is the number of blocks in the file */ - if (inode->i_size == 0) { - inode->i_blocks = 0; - } else { - inode->i_blocks = ((inode->i_size - 1) >> EFS_BLOCKSIZE_BITS) + 1; - } - - rdev = be16_to_cpu(efs_inode->di_u.di_dev.odev); - if (rdev == 0xffff) { - rdev = be32_to_cpu(efs_inode->di_u.di_dev.ndev); - if (sysv_major(rdev) > 0xfff) - device = 0; - else - device = MKDEV(sysv_major(rdev), sysv_minor(rdev)); - } else - device = old_decode_dev(rdev); - - /* get the number of extents for this object */ - in->numextents = be16_to_cpu(efs_inode->di_numextents); - in->lastextent = 0; - - /* copy the extents contained within the inode to memory */ - for(i = 0; i < EFS_DIRECTEXTENTS; i++) { - extent_copy(&(efs_inode->di_u.di_extents[i]), &(in->extents[i])); - if (i < in->numextents && in->extents[i].cooked.ex_magic != 0) { - pr_warn("extent %d has bad magic number in inode %llu\n", - i, inode->i_ino); - brelse(bh); - goto read_inode_error; - } - } - - brelse(bh); - pr_debug("efs_iget(): inode %llu, extents %d, mode %o\n", - inode->i_ino, in->numextents, inode->i_mode); - switch (inode->i_mode & S_IFMT) { - case S_IFDIR: - inode->i_op = &efs_dir_inode_operations; - inode->i_fop = &efs_dir_operations; - break; - case S_IFREG: - inode->i_fop = &generic_ro_fops; - inode->i_data.a_ops = &efs_aops; - break; - case S_IFLNK: - inode->i_op = &page_symlink_inode_operations; - inode_nohighmem(inode); - inode->i_data.a_ops = &efs_symlink_aops; - break; - case S_IFCHR: - case S_IFBLK: - case S_IFIFO: - init_special_inode(inode, inode->i_mode, device); - break; - default: - pr_warn("unsupported inode mode %o\n", inode->i_mode); - goto read_inode_error; - break; - } - - unlock_new_inode(inode); - return inode; - -read_inode_error: - pr_warn("failed to read inode %llu\n", inode->i_ino); - iget_failed(inode); - return ERR_PTR(-EIO); -} - -static inline efs_block_t -efs_extent_check(efs_extent *ptr, efs_block_t block, struct efs_sb_info *sb) { - efs_block_t start; - efs_block_t length; - efs_block_t offset; - - /* - * given an extent and a logical block within a file, - * can this block be found within this extent ? - */ - start = ptr->cooked.ex_bn; - length = ptr->cooked.ex_length; - offset = ptr->cooked.ex_offset; - - if ((block >= offset) && (block < offset+length)) { - return(sb->fs_start + start + block - offset); - } else { - return 0; - } -} - -efs_block_t efs_map_block(struct inode *inode, efs_block_t block) { - struct efs_sb_info *sb = SUPER_INFO(inode->i_sb); - struct efs_inode_info *in = INODE_INFO(inode); - struct buffer_head *bh = NULL; - - int cur, last, first = 1; - int ibase, ioffset, dirext, direxts, indext, indexts; - efs_block_t iblock, result = 0, lastblock = 0; - efs_extent ext, *exts; - - last = in->lastextent; - - if (in->numextents <= EFS_DIRECTEXTENTS) { - /* first check the last extent we returned */ - if ((result = efs_extent_check(&in->extents[last], block, sb))) - return result; - - /* if we only have one extent then nothing can be found */ - if (in->numextents == 1) { - pr_err("%s() failed to map (1 extent)\n", __func__); - return 0; - } - - direxts = in->numextents; - - /* - * check the stored extents in the inode - * start with next extent and check forwards - */ - for(dirext = 1; dirext < direxts; dirext++) { - cur = (last + dirext) % in->numextents; - if ((result = efs_extent_check(&in->extents[cur], block, sb))) { - in->lastextent = cur; - return result; - } - } - - pr_err("%s() failed to map block %u (dir)\n", __func__, block); - return 0; - } - - pr_debug("%s(): indirect search for logical block %u\n", - __func__, block); - direxts = in->extents[0].cooked.ex_offset; - indexts = in->numextents; - - for(indext = 0; indext < indexts; indext++) { - cur = (last + indext) % indexts; - - /* - * work out which direct extent contains `cur'. - * - * also compute ibase: i.e. the number of the first - * indirect extent contained within direct extent `cur'. - * - */ - ibase = 0; - for(dirext = 0; cur < ibase && dirext < direxts; dirext++) { - ibase += in->extents[dirext].cooked.ex_length * - (EFS_BLOCKSIZE / sizeof(efs_extent)); - } - - if (dirext == direxts) { - /* should never happen */ - pr_err("couldn't find direct extent for indirect extent %d (block %u)\n", - cur, block); - if (bh) brelse(bh); - return 0; - } - - /* work out block number and offset of this indirect extent */ - iblock = sb->fs_start + in->extents[dirext].cooked.ex_bn + - (cur - ibase) / - (EFS_BLOCKSIZE / sizeof(efs_extent)); - ioffset = (cur - ibase) % - (EFS_BLOCKSIZE / sizeof(efs_extent)); - - if (first || lastblock != iblock) { - if (bh) brelse(bh); - - bh = sb_bread(inode->i_sb, iblock); - if (!bh) { - pr_err("%s() failed at block %d\n", - __func__, iblock); - return 0; - } - pr_debug("%s(): read indirect extent block %d\n", - __func__, iblock); - first = 0; - lastblock = iblock; - } - - exts = (efs_extent *) bh->b_data; - - extent_copy(&(exts[ioffset]), &ext); - - if (ext.cooked.ex_magic != 0) { - pr_err("extent %d has bad magic number in block %d\n", - cur, iblock); - if (bh) brelse(bh); - return 0; - } - - if ((result = efs_extent_check(&ext, block, sb))) { - if (bh) brelse(bh); - in->lastextent = cur; - return result; - } - } - if (bh) brelse(bh); - pr_err("%s() failed to map block %u (indir)\n", __func__, block); - return 0; -} - -MODULE_DESCRIPTION("Extent File System (efs)"); -MODULE_LICENSE("GPL"); diff --git a/fs/efs/namei.c b/fs/efs/namei.c deleted file mode 100644 index 38961ee1d1af..000000000000 --- a/fs/efs/namei.c +++ /dev/null @@ -1,120 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * namei.c - * - * Copyright (c) 1999 Al Smith - * - * Portions derived from work (c) 1995,1996 Christian Vogelgsang. - */ - -#include -#include -#include -#include "efs.h" - - -static efs_ino_t efs_find_entry(struct inode *inode, const char *name, int len) -{ - struct buffer_head *bh; - - int slot, namelen; - char *nameptr; - struct efs_dir *dirblock; - struct efs_dentry *dirslot; - efs_ino_t inodenum; - efs_block_t block; - - if (inode->i_size & (EFS_DIRBSIZE-1)) - pr_warn("%s(): directory size not a multiple of EFS_DIRBSIZE\n", - __func__); - - for(block = 0; block < inode->i_blocks; block++) { - - bh = sb_bread(inode->i_sb, efs_bmap(inode, block)); - if (!bh) { - pr_err("%s(): failed to read dir block %d\n", - __func__, block); - return 0; - } - - dirblock = (struct efs_dir *) bh->b_data; - - if (be16_to_cpu(dirblock->magic) != EFS_DIRBLK_MAGIC) { - pr_err("%s(): invalid directory block\n", __func__); - brelse(bh); - return 0; - } - - for (slot = 0; slot < dirblock->slots; slot++) { - dirslot = (struct efs_dentry *) (((char *) bh->b_data) + EFS_SLOTAT(dirblock, slot)); - - namelen = dirslot->namelen; - nameptr = dirslot->name; - - if ((namelen == len) && (!memcmp(name, nameptr, len))) { - inodenum = be32_to_cpu(dirslot->inode); - brelse(bh); - return inodenum; - } - } - brelse(bh); - } - return 0; -} - -struct dentry *efs_lookup(struct inode *dir, struct dentry *dentry, unsigned int flags) -{ - efs_ino_t inodenum; - struct inode *inode = NULL; - - inodenum = efs_find_entry(dir, dentry->d_name.name, dentry->d_name.len); - if (inodenum) - inode = efs_iget(dir->i_sb, inodenum); - - return d_splice_alias(inode, dentry); -} - -static struct inode *efs_nfs_get_inode(struct super_block *sb, u64 ino, - u32 generation) -{ - struct inode *inode; - - if (ino == 0) - return ERR_PTR(-ESTALE); - inode = efs_iget(sb, ino); - if (IS_ERR(inode)) - return ERR_CAST(inode); - - if (generation && inode->i_generation != generation) { - iput(inode); - return ERR_PTR(-ESTALE); - } - - return inode; -} - -struct dentry *efs_fh_to_dentry(struct super_block *sb, struct fid *fid, - int fh_len, int fh_type) -{ - return generic_fh_to_dentry(sb, fid, fh_len, fh_type, - efs_nfs_get_inode); -} - -struct dentry *efs_fh_to_parent(struct super_block *sb, struct fid *fid, - int fh_len, int fh_type) -{ - return generic_fh_to_parent(sb, fid, fh_len, fh_type, - efs_nfs_get_inode); -} - -struct dentry *efs_get_parent(struct dentry *child) -{ - struct dentry *parent = ERR_PTR(-ENOENT); - efs_ino_t ino; - - ino = efs_find_entry(d_inode(child), "..", 2); - if (ino) - parent = d_obtain_alias(efs_iget(child->d_sb, ino)); - - return parent; -} diff --git a/fs/efs/super.c b/fs/efs/super.c deleted file mode 100644 index 11fea3bbce7c..000000000000 --- a/fs/efs/super.c +++ /dev/null @@ -1,368 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * super.c - * - * Copyright (c) 1999 Al Smith - * - * Portions derived from work (c) 1995,1996 Christian Vogelgsang. - */ - -#include -#include -#include -#include -#include -#include -#include -#include -#include "efs.h" -#include -#include - -static int efs_statfs(struct dentry *dentry, struct kstatfs *buf); -static int efs_init_fs_context(struct fs_context *fc); - -static void efs_kill_sb(struct super_block *s) -{ - struct efs_sb_info *sbi = SUPER_INFO(s); - kill_block_super(s); - kfree(sbi); -} - -static struct pt_types sgi_pt_types[] = { - {0x00, "SGI vh"}, - {0x01, "SGI trkrepl"}, - {0x02, "SGI secrepl"}, - {0x03, "SGI raw"}, - {0x04, "SGI bsd"}, - {SGI_SYSV, "SGI sysv"}, - {0x06, "SGI vol"}, - {SGI_EFS, "SGI efs"}, - {0x08, "SGI lv"}, - {0x09, "SGI rlv"}, - {0x0A, "SGI xfs"}, - {0x0B, "SGI xfslog"}, - {0x0C, "SGI xlv"}, - {0x82, "Linux swap"}, - {0x83, "Linux native"}, - {0, NULL} -}; - -/* - * File system definition and registration. - */ -static struct file_system_type efs_fs_type = { - .owner = THIS_MODULE, - .name = "efs", - .kill_sb = efs_kill_sb, - .fs_flags = FS_REQUIRES_DEV, - .init_fs_context = efs_init_fs_context, -}; -MODULE_ALIAS_FS("efs"); - -static struct kmem_cache * efs_inode_cachep; - -static struct inode *efs_alloc_inode(struct super_block *sb) -{ - struct efs_inode_info *ei; - ei = alloc_inode_sb(sb, efs_inode_cachep, GFP_KERNEL); - if (!ei) - return NULL; - return &ei->vfs_inode; -} - -static void efs_free_inode(struct inode *inode) -{ - kmem_cache_free(efs_inode_cachep, INODE_INFO(inode)); -} - -static void init_once(void *foo) -{ - struct efs_inode_info *ei = (struct efs_inode_info *) foo; - - inode_init_once(&ei->vfs_inode); -} - -static int __init init_inodecache(void) -{ - efs_inode_cachep = kmem_cache_create("efs_inode_cache", - sizeof(struct efs_inode_info), 0, - SLAB_RECLAIM_ACCOUNT|SLAB_ACCOUNT, - init_once); - if (efs_inode_cachep == NULL) - return -ENOMEM; - return 0; -} - -static void destroy_inodecache(void) -{ - /* - * Make sure all delayed rcu free inodes are flushed before we - * destroy cache. - */ - rcu_barrier(); - kmem_cache_destroy(efs_inode_cachep); -} - -static const struct super_operations efs_superblock_operations = { - .alloc_inode = efs_alloc_inode, - .free_inode = efs_free_inode, - .statfs = efs_statfs, -}; - -static const struct export_operations efs_export_ops = { - .encode_fh = generic_encode_ino32_fh, - .fh_to_dentry = efs_fh_to_dentry, - .fh_to_parent = efs_fh_to_parent, - .get_parent = efs_get_parent, -}; - -static int __init init_efs_fs(void) { - int err; - pr_info(EFS_VERSION" - http://aeschi.ch.eu.org/efs/\n"); - err = init_inodecache(); - if (err) - goto out1; - err = register_filesystem(&efs_fs_type); - if (err) - goto out; - return 0; -out: - destroy_inodecache(); -out1: - return err; -} - -static void __exit exit_efs_fs(void) { - unregister_filesystem(&efs_fs_type); - destroy_inodecache(); -} - -module_init(init_efs_fs) -module_exit(exit_efs_fs) - -static efs_block_t efs_validate_vh(struct volume_header *vh) { - int i; - __be32 cs, *ui; - int csum; - efs_block_t sblock = 0; /* shuts up gcc */ - struct pt_types *pt_entry; - int pt_type, slice = -1; - - if (be32_to_cpu(vh->vh_magic) != VHMAGIC) { - /* - * assume that we're dealing with a partition and allow - * read_super() to try and detect a valid superblock - * on the next block. - */ - return 0; - } - - ui = ((__be32 *) (vh + 1)) - 1; - for(csum = 0; ui >= ((__be32 *) vh);) { - cs = *ui--; - csum += be32_to_cpu(cs); - } - if (csum) { - pr_warn("SGI disklabel: checksum bad, label corrupted\n"); - return 0; - } - -#ifdef DEBUG - pr_debug("bf: \"%16s\"\n", vh->vh_bootfile); - - for(i = 0; i < NVDIR; i++) { - int j; - char name[VDNAMESIZE+1]; - - for(j = 0; j < VDNAMESIZE; j++) { - name[j] = vh->vh_vd[i].vd_name[j]; - } - name[j] = (char) 0; - - if (name[0]) { - pr_debug("vh: %8s block: 0x%08x size: 0x%08x\n", - name, (int) be32_to_cpu(vh->vh_vd[i].vd_lbn), - (int) be32_to_cpu(vh->vh_vd[i].vd_nbytes)); - } - } -#endif - - for(i = 0; i < NPARTAB; i++) { - pt_type = (int) be32_to_cpu(vh->vh_pt[i].pt_type); - for(pt_entry = sgi_pt_types; pt_entry->pt_name; pt_entry++) { - if (pt_type == pt_entry->pt_type) break; - } -#ifdef DEBUG - if (be32_to_cpu(vh->vh_pt[i].pt_nblks)) { - pr_debug("pt %2d: start: %08d size: %08d type: 0x%02x (%s)\n", - i, (int)be32_to_cpu(vh->vh_pt[i].pt_firstlbn), - (int)be32_to_cpu(vh->vh_pt[i].pt_nblks), - pt_type, (pt_entry->pt_name) ? - pt_entry->pt_name : "unknown"); - } -#endif - if (IS_EFS(pt_type)) { - sblock = be32_to_cpu(vh->vh_pt[i].pt_firstlbn); - slice = i; - } - } - - if (slice == -1) { - pr_notice("partition table contained no EFS partitions\n"); -#ifdef DEBUG - } else { - pr_info("using slice %d (type %s, offset 0x%x)\n", slice, - (pt_entry->pt_name) ? pt_entry->pt_name : "unknown", - sblock); -#endif - } - return sblock; -} - -static int efs_validate_super(struct efs_sb_info *sb, struct efs_super *super) { - - if (!IS_EFS_MAGIC(be32_to_cpu(super->fs_magic))) - return -1; - - sb->fs_magic = be32_to_cpu(super->fs_magic); - sb->total_blocks = be32_to_cpu(super->fs_size); - sb->first_block = be32_to_cpu(super->fs_firstcg); - sb->group_size = be32_to_cpu(super->fs_cgfsize); - sb->data_free = be32_to_cpu(super->fs_tfree); - sb->inode_free = be32_to_cpu(super->fs_tinode); - sb->inode_blocks = be16_to_cpu(super->fs_cgisize); - sb->total_groups = be16_to_cpu(super->fs_ncg); - - return 0; -} - -static int efs_fill_super(struct super_block *s, struct fs_context *fc) -{ - struct efs_sb_info *sb; - struct buffer_head *bh; - struct inode *root; - - sb = kzalloc_obj(struct efs_sb_info); - if (!sb) - return -ENOMEM; - s->s_fs_info = sb; - s->s_time_min = 0; - s->s_time_max = U32_MAX; - - s->s_magic = EFS_SUPER_MAGIC; - if (!sb_set_blocksize(s, EFS_BLOCKSIZE)) { - pr_err("device does not support %d byte blocks\n", - EFS_BLOCKSIZE); - return invalf(fc, "device does not support %d byte blocks\n", - EFS_BLOCKSIZE); - } - - /* read the vh (volume header) block */ - bh = sb_bread(s, 0); - - if (!bh) { - pr_err("cannot read volume header\n"); - return -EIO; - } - - /* - * if this returns zero then we didn't find any partition table. - * this isn't (yet) an error - just assume for the moment that - * the device is valid and go on to search for a superblock. - */ - sb->fs_start = efs_validate_vh((struct volume_header *) bh->b_data); - brelse(bh); - - if (sb->fs_start == -1) { - return -EINVAL; - } - - bh = sb_bread(s, sb->fs_start + EFS_SUPER); - if (!bh) { - pr_err("cannot read superblock\n"); - return -EIO; - } - - if (efs_validate_super(sb, (struct efs_super *) bh->b_data)) { -#ifdef DEBUG - pr_warn("invalid superblock at block %u\n", - sb->fs_start + EFS_SUPER); -#endif - brelse(bh); - return -EINVAL; - } - brelse(bh); - - if (!sb_rdonly(s)) { -#ifdef DEBUG - pr_info("forcing read-only mode\n"); -#endif - s->s_flags |= SB_RDONLY; - } - s->s_op = &efs_superblock_operations; - s->s_export_op = &efs_export_ops; - root = efs_iget(s, EFS_ROOTINODE); - if (IS_ERR(root)) { - pr_err("get root inode failed\n"); - return PTR_ERR(root); - } - - s->s_root = d_make_root(root); - if (!(s->s_root)) { - pr_err("get root dentry failed\n"); - return -ENOMEM; - } - - return 0; -} - -static int efs_get_tree(struct fs_context *fc) -{ - return get_tree_bdev(fc, efs_fill_super); -} - -static int efs_reconfigure(struct fs_context *fc) -{ - sync_filesystem(fc->root->d_sb); - fc->sb_flags |= SB_RDONLY; - - return 0; -} - -static const struct fs_context_operations efs_context_opts = { - .get_tree = efs_get_tree, - .reconfigure = efs_reconfigure, -}; - -/* - * Set up the filesystem mount context. - */ -static int efs_init_fs_context(struct fs_context *fc) -{ - fc->ops = &efs_context_opts; - - return 0; -} - -static int efs_statfs(struct dentry *dentry, struct kstatfs *buf) { - struct super_block *sb = dentry->d_sb; - struct efs_sb_info *sbi = SUPER_INFO(sb); - u64 id = huge_encode_dev(sb->s_bdev->bd_dev); - - buf->f_type = EFS_SUPER_MAGIC; /* efs magic number */ - buf->f_bsize = EFS_BLOCKSIZE; /* blocksize */ - buf->f_blocks = sbi->total_groups * /* total data blocks */ - (sbi->group_size - sbi->inode_blocks); - buf->f_bfree = sbi->data_free; /* free data blocks */ - buf->f_bavail = sbi->data_free; /* free blocks for non-root */ - buf->f_files = sbi->total_groups * /* total inodes */ - sbi->inode_blocks * - (EFS_BLOCKSIZE / sizeof(struct efs_dinode)); - buf->f_ffree = sbi->inode_free; /* free inodes */ - buf->f_fsid = u64_to_fsid(id); - buf->f_namelen = EFS_MAXNAMELEN; /* max filename length */ - - return 0; -} - diff --git a/fs/efs/symlink.c b/fs/efs/symlink.c deleted file mode 100644 index 7749feded722..000000000000 --- a/fs/efs/symlink.c +++ /dev/null @@ -1,50 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * symlink.c - * - * Copyright (c) 1999 Al Smith - * - * Portions derived from work (c) 1995,1996 Christian Vogelgsang. - */ - -#include -#include -#include -#include "efs.h" - -static int efs_symlink_read_folio(struct file *file, struct folio *folio) -{ - char *link = folio_address(folio); - struct buffer_head *bh; - struct inode *inode = folio->mapping->host; - efs_block_t size = inode->i_size; - int err; - - err = -ENAMETOOLONG; - if (size > 2 * EFS_BLOCKSIZE) - goto fail; - - /* read first 512 bytes of link target */ - err = -EIO; - bh = sb_bread(inode->i_sb, efs_bmap(inode, 0)); - if (!bh) - goto fail; - memcpy(link, bh->b_data, (size > EFS_BLOCKSIZE) ? EFS_BLOCKSIZE : size); - brelse(bh); - if (size > EFS_BLOCKSIZE) { - bh = sb_bread(inode->i_sb, efs_bmap(inode, 1)); - if (!bh) - goto fail; - memcpy(link + EFS_BLOCKSIZE, bh->b_data, size - EFS_BLOCKSIZE); - brelse(bh); - } - link[size] = '\0'; - err = 0; -fail: - folio_end_read(folio, err == 0); - return err; -} - -const struct address_space_operations efs_symlink_aops = { - .read_folio = efs_symlink_read_folio -}; diff --git a/fs/erofs/data.c b/fs/erofs/data.c index 9aa48c8d67d1..d2f01245ee79 100644 --- a/fs/erofs/data.c +++ b/fs/erofs/data.c @@ -380,9 +380,11 @@ static int erofs_iomap_end(struct inode *inode, loff_t pos, loff_t length, return written; } +static DEFINE_IOMAP_ITER_NEXT_END(erofs_iomap_next, erofs_iomap_begin, + erofs_iomap_end); + static const struct iomap_ops erofs_iomap_ops = { - .iomap_begin = erofs_iomap_begin, - .iomap_end = erofs_iomap_end, + .iomap_next = erofs_iomap_next, }; int erofs_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo, diff --git a/fs/erofs/super.c b/fs/erofs/super.c index bc55be84d945..8ead1646f329 100644 --- a/fs/erofs/super.c +++ b/fs/erofs/super.c @@ -147,8 +147,8 @@ static int erofs_init_device(struct erofs_buf *buf, struct super_block *sb, if (!sbi->devs->flatdev) { file = erofs_is_fileio_mode(sbi) ? filp_open(dif->path, O_RDONLY | O_LARGEFILE, 0) : - bdev_file_open_by_path(dif->path, - BLK_OPEN_READ, sb->s_type, NULL); + fs_bdev_file_open_by_path(dif->path, + BLK_OPEN_READ, sb->s_type, sb); if (IS_ERR(file)) { if (file == ERR_PTR(-ENOTBLK)) return -EINVAL; @@ -841,28 +841,34 @@ static int erofs_fc_reconfigure(struct fs_context *fc) static int erofs_release_device_info(int id, void *ptr, void *data) { + struct super_block *sb = data; struct erofs_device_info *dif = ptr; fs_put_dax(dif->dax_dev, NULL); - if (dif->file) - fput(dif->file); + if (dif->file) { + if (S_ISBLK(file_inode(dif->file)->i_mode)) + fs_bdev_file_release(dif->file, sb); + else + fput(dif->file); + } kfree(dif->path); kfree(dif); return 0; } -static void erofs_free_dev_context(struct erofs_dev_context *devs) +static void erofs_free_dev_context(struct erofs_dev_context *devs, + struct super_block *sb) { if (!devs) return; - idr_for_each(&devs->tree, &erofs_release_device_info, NULL); + idr_for_each(&devs->tree, &erofs_release_device_info, sb); idr_destroy(&devs->tree); kfree(devs); } -static void erofs_sb_free(struct erofs_sb_info *sbi) +static void erofs_sb_free(struct erofs_sb_info *sbi, struct super_block *sb) { - erofs_free_dev_context(sbi->devs); + erofs_free_dev_context(sbi->devs, sb); kfree_sensitive(sbi->domain_id); if (sbi->dif0.file) fput(sbi->dif0.file); @@ -874,8 +880,13 @@ static void erofs_fc_free(struct fs_context *fc) { struct erofs_sb_info *sbi = fc->s_fs_info; - if (sbi) /* free here if an error occurs before transferring to sb */ - erofs_sb_free(sbi); + /* + * Freed here only if an error occurs before the sb is set up; at that + * point no block-backed device has been claimed (that happens in + * fill_super), so the NULL sb never reaches fs_bdev_file_release(). + */ + if (sbi) + erofs_sb_free(sbi, NULL); } static const struct fs_context_operations erofs_context_ops = { @@ -929,7 +940,7 @@ static void erofs_kill_sb(struct super_block *sb) kill_block_super(sb); erofs_drop_internal_inodes(sbi); fs_put_dax(sbi->dif0.dax_dev, NULL); - erofs_sb_free(sbi); + erofs_sb_free(sbi, sb); sb->s_fs_info = NULL; } @@ -941,7 +952,7 @@ static void erofs_put_super(struct super_block *sb) erofs_shrinker_unregister(sb); erofs_xattr_prefixes_cleanup(sb); erofs_drop_internal_inodes(sbi); - erofs_free_dev_context(sbi->devs); + erofs_free_dev_context(sbi->devs, sb); sbi->devs = NULL; } diff --git a/fs/erofs/zdata.c b/fs/erofs/zdata.c index 74520e910259..d022d1dff5a1 100644 --- a/fs/erofs/zdata.c +++ b/fs/erofs/zdata.c @@ -725,7 +725,7 @@ static bool z_erofs_get_pcluster(struct z_erofs_pcluster *pcl) return true; spin_lock(&pcl->lockref.lock); - if (__lockref_is_dead(&pcl->lockref)) { + if (lockref_is_dead(&pcl->lockref)) { spin_unlock(&pcl->lockref.lock); return false; } @@ -945,7 +945,7 @@ static void z_erofs_put_pcluster(struct erofs_sb_info *sbi, if (lockref_put_or_lock(&pcl->lockref)) return; - DBG_BUGON(__lockref_is_dead(&pcl->lockref)); + DBG_BUGON(lockref_is_dead(&pcl->lockref)); if (!--pcl->lockref.count) { if (try_free && xa_trylock(&sbi->managed_pslots)) { free = __erofs_try_to_release_pcluster(sbi, pcl); diff --git a/fs/erofs/zmap.c b/fs/erofs/zmap.c index 5811556a7b71..5f33af3fdf97 100644 --- a/fs/erofs/zmap.c +++ b/fs/erofs/zmap.c @@ -822,6 +822,9 @@ static int z_erofs_iomap_begin_report(struct inode *inode, loff_t offset, return 0; } +static DEFINE_IOMAP_ITER_NEXT(z_erofs_iomap_next_report, + z_erofs_iomap_begin_report); + const struct iomap_ops z_erofs_iomap_report_ops = { - .iomap_begin = z_erofs_iomap_begin_report, + .iomap_next = z_erofs_iomap_next_report, }; diff --git a/fs/eventpoll.c b/fs/eventpoll.c index eed8cecd94e3..e0c4bf88a838 100644 --- a/fs/eventpoll.c +++ b/fs/eventpoll.c @@ -2264,7 +2264,6 @@ static int ep_poll(struct eventpoll *ep, struct epoll_event __user *events, lockdep_assert_irqs_enabled(); if (timeout && (timeout->tv_sec | timeout->tv_nsec)) { - slack = select_estimate_accuracy(timeout); to = &expires; *to = timespec64_to_ktime(*timeout); } else if (timeout) { @@ -2343,10 +2342,13 @@ static int ep_poll(struct eventpoll *ep, struct epoll_event __user *events, spin_unlock_irq(&ep->lock); - if (!eavail) + if (!eavail) { + if (to) + slack = select_estimate_accuracy(timeout); timed_out = !ep_schedule_timeout(to) || !schedule_hrtimeout_range(to, slack, HRTIMER_MODE_ABS); + } __set_current_state(TASK_RUNNING); /* diff --git a/fs/exec.c b/fs/exec.c index c7b8f2d6366c..a14f28b15607 100644 --- a/fs/exec.c +++ b/fs/exec.c @@ -1101,6 +1101,17 @@ void __set_task_comm(struct task_struct *tsk, const char *buf, bool exec) perf_event_comm(tsk, exec); } +/* + * The file the process presents as: its exe link and comm. A transparent + * dispatch presents as the binary, which is bprm->executable. + */ +static struct file *bprm_identity_file(const struct linux_binprm *bprm) +{ + if (bprm->interp_flags & BINPRM_FLAGS_TRANSPARENT_INTERP) + return bprm->executable; + return bprm->file; +} + /* * Calling this is the point of no return. None of the failures will be * seen by userspace since either the process is already taking a fatal @@ -1112,6 +1123,10 @@ int begin_new_exec(struct linux_binprm * bprm) struct task_struct *me = current; int retval; + /* A pending PT_INTERP substitution this format cannot consume. */ + if (bprm->loader) + return -ENOEXEC; + /* Once we are committed compute the creds */ retval = bprm_creds_from_file(bprm); if (retval) @@ -1151,7 +1166,7 @@ int begin_new_exec(struct linux_binprm * bprm) * not visible until then. Doing it here also ensures * we don't race against replace_mm_exe_file(). */ - retval = set_mm_exe_file(bprm->mm, bprm->file); + retval = set_mm_exe_file(bprm->mm, bprm_identity_file(bprm)); if (retval) goto out; @@ -1241,6 +1256,8 @@ int begin_new_exec(struct linux_binprm * bprm) * Let's fix it up to be something reasonable. */ if (bprm->comm_from_dentry) { + struct file *comm_file = bprm_identity_file(bprm); + /* * Hold RCU lock to keep the name from being freed behind our back. * Use acquire semantics to make sure the terminating NUL from @@ -1250,7 +1267,7 @@ int begin_new_exec(struct linux_binprm * bprm) * detecting a concurrent rename and just want a terminated name. */ rcu_read_lock(); - __set_task_comm(me, smp_load_acquire(&bprm->file->f_path.dentry->d_name.name), + __set_task_comm(me, smp_load_acquire(&comm_file->f_path.dentry->d_name.name), true); rcu_read_unlock(); } else { @@ -1291,10 +1308,17 @@ int begin_new_exec(struct linux_binprm * bprm) /* Pass the opened binary to the interpreter. */ if (bprm->have_execfd) { - retval = FD_ADD(0, bprm->executable); - if (retval < 0) - goto out_unlock; + struct file *executable = bprm->executable; + + /* mm->exe_file carries its own write denial now so drop it. */ + exe_file_allow_write_access(executable); bprm->executable = NULL; + retval = FD_ADD(0, executable); + if (retval < 0) { + /* The reference was not consumed. */ + fput(executable); + goto out_unlock; + } bprm->execfd = retval; } return 0; @@ -1394,6 +1418,39 @@ static void do_close_execat(struct file *file) fput(file); } +/** + * bprm_open_interpreter - open the interpreter the binary asks for + * @bprm: binary that is being executed + * @path: the interpreter path named in the binary's PT_INTERP + * + * A binfmt_misc loader entry substitutes for the interpreter the binary + * names. Hand out the stashed substitute if there is one and open @path + * if there is not. The caller owns the reference either way and releases + * it like any other open_exec() one. + * + * Return: the interpreter on success, an ERR_PTR on failure + */ +struct file *bprm_open_interpreter(struct linux_binprm *bprm, const char *path) +{ + if (bprm->loader) + return no_free_ptr(bprm->loader); + return open_exec(path); +} + +/** + * bprm_drop_loader - discard a PT_INTERP substitute that does not apply + * @bprm: binary that is being executed + * + * A binary without PT_INTERP has nothing to substitute for, so drop the + * override and let the binary load natively rather than have + * begin_new_exec() refuse it. A no-op once bprm_open_interpreter() took + * the substitute. + */ +void bprm_drop_loader(struct linux_binprm *bprm) +{ + do_close_execat(no_free_ptr(bprm->loader)); +} + static void free_bprm(struct linux_binprm *bprm) { if (bprm->mm) { @@ -1413,11 +1470,16 @@ static void free_bprm(struct linux_binprm *bprm) if (bprm->old_mm) exec_mm_put_old(bprm->old_mm); do_close_execat(bprm->file); - if (bprm->executable) - fput(bprm->executable); + /* An unconsumed PT_INTERP substitute from a binfmt_misc loader entry. */ + bprm_drop_loader(bprm); + do_close_execat(bprm->executable); /* If a binfmt changed the interp, free it. */ if (bprm->interp != bprm->filename) kfree(bprm->interp); + kfree(bprm->bpf_interp); + if (bprm->bpf_interp_file) + fput(bprm->bpf_interp_file); + kfree(bprm->bpf_interp_arg); kfree(bprm->fdpath); kfree(bprm); } @@ -1729,19 +1791,23 @@ static int exec_binprm(struct linux_binprm *bprm) if (!bprm->interpreter) break; + /* A stashed PT_INTERP substitute belonged to the replaced file. */ + bprm_drop_loader(bprm); + exec = bprm->file; bprm->file = bprm->interpreter; bprm->interpreter = NULL; - exe_file_allow_write_access(exec); if (unlikely(bprm->have_execfd)) { if (bprm->executable) { - fput(exec); + do_close_execat(exec); return -ENOEXEC; } + /* Kept for AT_EXECFD; the write denial rides along until hand-over. */ bprm->executable = exec; - } else - fput(exec); + } else { + do_close_execat(exec); + } } audit_bprm(bprm); diff --git a/fs/exfat/iomap.c b/fs/exfat/iomap.c index d4d3ed933a63..8911aa84a730 100644 --- a/fs/exfat/iomap.c +++ b/fs/exfat/iomap.c @@ -151,8 +151,10 @@ static int exfat_write_iomap_begin(struct inode *inode, loff_t offset, loff_t le return __exfat_iomap_begin(inode, offset, length, flags, iomap, true); } +static DEFINE_IOMAP_ITER_NEXT(exfat_iomap_next, exfat_iomap_begin); + const struct iomap_ops exfat_iomap_ops = { - .iomap_begin = exfat_iomap_begin, + .iomap_next = exfat_iomap_next, }; /* @@ -193,9 +195,11 @@ static int exfat_write_iomap_end(struct inode *inode, loff_t pos, loff_t length, return written; } +static DEFINE_IOMAP_ITER_NEXT_END(exfat_write_iomap_next, + exfat_write_iomap_begin, exfat_write_iomap_end); + const struct iomap_ops exfat_write_iomap_ops = { - .iomap_begin = exfat_write_iomap_begin, - .iomap_end = exfat_write_iomap_end, + .iomap_next = exfat_write_iomap_next, }; /* diff --git a/fs/exfat/namei.c b/fs/exfat/namei.c index f26f987a34cf..a4dc83b5949c 100644 --- a/fs/exfat/namei.c +++ b/fs/exfat/namei.c @@ -553,7 +553,7 @@ out: } static int exfat_create(struct mnt_idmap *idmap, struct inode *dir, - struct dentry *dentry, umode_t mode, bool excl) + struct dentry *dentry, umode_t mode) { struct super_block *sb = dir->i_sb; struct inode *inode; diff --git a/fs/ext2/dir.c b/fs/ext2/dir.c index 278d4be8ecbe..e17bbc7598c1 100644 --- a/fs/ext2/dir.c +++ b/fs/ext2/dir.c @@ -734,6 +734,6 @@ const struct file_operations ext2_dir_operations = { #ifdef CONFIG_COMPAT .compat_ioctl = ext2_compat_ioctl, #endif - .fsync = ext2_fsync, + .fsync = simple_fsync, .setlease = generic_setlease, }; diff --git a/fs/ext2/ext2.h b/fs/ext2/ext2.h index 79f7b395258c..5642451bf191 100644 --- a/fs/ext2/ext2.h +++ b/fs/ext2/ext2.h @@ -735,6 +735,7 @@ extern unsigned long ext2_count_free (struct buffer_head *, unsigned); /* inode.c */ extern struct inode *ext2_iget (struct super_block *, unsigned long); extern int ext2_write_inode (struct inode *, struct writeback_control *); +extern int ext2_sync_inode_metadata(struct inode *, struct writeback_control *); extern void ext2_evict_inode(struct inode *); void ext2_write_failed(struct address_space *mapping, loff_t to); extern int ext2_get_block(struct inode *, sector_t, struct buffer_head *, int); @@ -772,8 +773,6 @@ extern void ext2_sync_super(struct super_block *sb, struct ext2_super_block *es, extern const struct file_operations ext2_dir_operations; /* file.c */ -extern int ext2_fsync(struct file *file, loff_t start, loff_t end, - int datasync); extern const struct inode_operations ext2_file_inode_operations; extern const struct file_operations ext2_file_operations; diff --git a/fs/ext2/file.c b/fs/ext2/file.c index 8dca9ec4cacd..b9020df7d89e 100644 --- a/fs/ext2/file.c +++ b/fs/ext2/file.c @@ -47,21 +47,6 @@ static int ext2_release_file (struct inode * inode, struct file * filp) return 0; } -int ext2_fsync(struct file *file, loff_t start, loff_t end, int datasync) -{ - int ret; - struct inode *inode = file->f_mapping->host; - struct super_block *sb = inode->i_sb; - - ret = mmb_fsync(file, &EXT2_I(inode)->i_metadata_bhs, - start, end, datasync); - if (ret == -EIO) - /* We don't really know where the IO error happened... */ - ext2_error(sb, __func__, - "detected IO error when writing metadata buffers"); - return ret; -} - static ssize_t ext2_dio_read_iter(struct kiocb *iocb, struct iov_iter *to) { struct file *file = iocb->ki_filp; @@ -213,7 +198,7 @@ const struct file_operations ext2_file_operations = { .mmap_prepare = generic_file_mmap_prepare, .open = ext2_file_open, .release = ext2_release_file, - .fsync = ext2_fsync, + .fsync = simple_fsync, .get_unmapped_area = thp_get_unmapped_area, .splice_read = filemap_splice_read, .splice_write = iter_file_splice_write, diff --git a/fs/ext2/inode.c b/fs/ext2/inode.c index 29808629cce5..a9245f0cda4d 100644 --- a/fs/ext2/inode.c +++ b/fs/ext2/inode.c @@ -39,8 +39,6 @@ #include "acl.h" #include "xattr.h" -static int __ext2_write_inode(struct inode *inode, int do_sync); - /* * Test whether an inode is a fast symlink. */ @@ -87,7 +85,7 @@ void ext2_evict_inode(struct inode * inode) /* set dtime */ EXT2_I(inode)->i_dtime = ktime_get_real_seconds(); mark_inode_dirty(inode); - __ext2_write_inode(inode, inode_needs_sync(inode)); + sync_inode_metadata(inode, inode_needs_sync(inode)); /* truncate to 0 */ inode->i_size = 0; if (inode->i_blocks) @@ -860,9 +858,11 @@ ext2_iomap_end(struct inode *inode, loff_t offset, loff_t length, return 0; } +static DEFINE_IOMAP_ITER_NEXT_END(ext2_iomap_next, ext2_iomap_begin, + ext2_iomap_end); + const struct iomap_ops ext2_iomap_ops = { - .iomap_begin = ext2_iomap_begin, - .iomap_end = ext2_iomap_end, + .iomap_next = ext2_iomap_next, }; int ext2_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo, @@ -1258,12 +1258,9 @@ static int ext2_setsize(struct inode *inode, loff_t newsize) filemap_invalidate_unlock(inode->i_mapping); inode_set_mtime_to_ts(inode, inode_set_ctime_current(inode)); - if (inode_needs_sync(inode)) { - mmb_sync(&EXT2_I(inode)->i_metadata_bhs); + mark_inode_dirty(inode); + if (inode_needs_sync(inode)) sync_inode_metadata(inode, 1); - } else { - mark_inode_dirty(inode); - } return 0; } @@ -1469,7 +1466,7 @@ bad_inode: return ERR_PTR(ret); } -static int __ext2_write_inode(struct inode *inode, int do_sync) +int ext2_write_inode(struct inode *inode, struct writeback_control *wbc) { struct ext2_inode_info *ei = EXT2_I(inode); struct super_block *sb = inode->i_sb; @@ -1560,22 +1557,38 @@ static int __ext2_write_inode(struct inode *inode, int do_sync) } else for (n = 0; n < EXT2_N_BLOCKS; n++) raw_inode->i_block[n] = ei->i_data[n]; mark_buffer_dirty(bh); - if (do_sync) { - sync_dirty_buffer(bh); - if (buffer_req(bh) && !buffer_uptodate(bh)) { - printk ("IO error syncing ext2 inode [%s:%08lx]\n", - sb->s_id, (unsigned long) ino); - err = -EIO; - } - } ei->i_state &= ~EXT2_STATE_NEW; brelse (bh); + set_inode_metadata_writeback(inode); return err; } -int ext2_write_inode(struct inode *inode, struct writeback_control *wbc) +int ext2_sync_inode_metadata(struct inode *inode, struct writeback_control *wbc) { - return __ext2_write_inode(inode, wbc->sync_mode == WB_SYNC_ALL); + struct buffer_head *bh; + struct ext2_inode *raw_inode = ext2_get_inode(inode->i_sb, inode->i_ino, + &bh); + int err = 0; + + if (IS_ERR(raw_inode)) + return -EIO; + err = mmb_sync(&EXT2_I(inode)->i_metadata_bhs); + if (err) { + ext2_error(inode->i_sb, __func__, + "Error syncing inode metadata ino=%lu\n", + (unsigned long)inode->i_ino); + goto out; + } + sync_dirty_buffer(bh); + if (buffer_write_io_error(bh)) { + ext2_error(inode->i_sb, __func__, + "IO error syncing inode %lu\n", + (unsigned long)inode->i_ino); + err = -EIO; + } +out: + brelse(bh); + return err; } int ext2_getattr(struct mnt_idmap *idmap, const struct path *path, diff --git a/fs/ext2/namei.c b/fs/ext2/namei.c index 0d09d22fe708..8666233ec63b 100644 --- a/fs/ext2/namei.c +++ b/fs/ext2/namei.c @@ -99,7 +99,7 @@ struct dentry *ext2_get_parent(struct dentry *child) */ static int ext2_create (struct mnt_idmap * idmap, struct inode * dir, struct dentry * dentry, - umode_t mode, bool excl) + umode_t mode) { struct inode *inode; int err; @@ -236,7 +236,7 @@ static struct dentry *ext2_mkdir(struct mnt_idmap * idmap, inode_inc_link_count(dir); - inode = ext2_new_inode(dir, S_IFDIR | mode, &dentry->d_name); + inode = ext2_new_inode(dir, mode, &dentry->d_name); err = PTR_ERR(inode); if (IS_ERR(inode)) goto out_dir; diff --git a/fs/ext2/super.c b/fs/ext2/super.c index 3999f8f3b156..a40f530872a4 100644 --- a/fs/ext2/super.c +++ b/fs/ext2/super.c @@ -362,6 +362,7 @@ static const struct super_operations ext2_sops = { .alloc_inode = ext2_alloc_inode, .free_inode = ext2_free_in_core_inode, .write_inode = ext2_write_inode, + .sync_inode_metadata = ext2_sync_inode_metadata, .evict_inode = ext2_evict_inode, .put_super = ext2_put_super, .sync_fs = ext2_sync_fs, diff --git a/fs/ext2/xattr.c b/fs/ext2/xattr.c index 5f49ec4afc36..9b68c490ab26 100644 --- a/fs/ext2/xattr.c +++ b/fs/ext2/xattr.c @@ -777,6 +777,7 @@ ext2_xattr_set2(struct inode *inode, struct buffer_head *old_bh, /* Update the inode. */ EXT2_I(inode)->i_file_acl = new_bh ? new_bh->b_blocknr : 0; inode_set_ctime_current(inode); + mark_inode_dirty(inode); if (IS_SYNC(inode)) { error = sync_inode_metadata(inode, 1); /* @@ -785,8 +786,7 @@ ext2_xattr_set2(struct inode *inode, struct buffer_head *old_bh, */ if (error) goto cleanup; - } else - mark_inode_dirty(inode); + } error = 0; if (old_bh && old_bh != new_bh) { diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h index 4e3b3165ee8f..b58e0705cd5d 100644 --- a/fs/ext4/ext4.h +++ b/fs/ext4/ext4.h @@ -1157,7 +1157,7 @@ struct ext4_inode_info { struct rw_semaphore i_data_sem; struct inode vfs_inode; struct jbd2_inode *jinode; - struct mapping_metadata_bhs i_metadata_bhs; + struct mapping_metadata_bhs *i_metadata_bhs; /* * File creation time. Its function is same as that of @@ -2137,6 +2137,17 @@ static inline bool ext4_inode_orphan_tracked(struct inode *inode) !list_empty(&EXT4_I(inode)->i_orphan); } +static inline struct mapping_metadata_bhs *ext4_i_metadata_bhs( + struct inode *inode) +{ + /* + * i_metadata_bhs is set in ext4_inode_attach_mmb() using cmpxchg(). + * We use READ_ONCE when accessing i_metadata_bhs to make sure we get + * consistent view for all accesses. + */ + return READ_ONCE(EXT4_I(inode)->i_metadata_bhs); +} + /* * Codes for operating systems */ @@ -3167,6 +3178,7 @@ extern struct inode *__ext4_iget(struct super_block *sb, unsigned long ino, __ext4_iget((sb), (ino), (flags), __func__, __LINE__) extern int ext4_write_inode(struct inode *, struct writeback_control *); +extern int ext4_sync_inode_metadata(struct inode *, struct writeback_control *); extern int ext4_setattr(struct mnt_idmap *, struct dentry *, struct iattr *); extern u32 ext4_dio_alignment(struct inode *inode); @@ -4028,6 +4040,9 @@ static inline void ext4_clear_io_unwritten_flag(ext4_io_end_t *io_end) extern const struct iomap_ops ext4_iomap_ops; extern const struct iomap_ops ext4_iomap_report_ops; +int ext4_iomap_begin(struct inode *inode, loff_t offset, loff_t length, + unsigned flags, struct iomap *iomap, struct iomap *srcmap); + static inline int ext4_buffer_uptodate(struct buffer_head *bh) { /* diff --git a/fs/ext4/ext4_jbd2.c b/fs/ext4/ext4_jbd2.c index b4dacd1a89e7..53ddedb52a6f 100644 --- a/fs/ext4/ext4_jbd2.c +++ b/fs/ext4/ext4_jbd2.c @@ -358,6 +358,21 @@ int __ext4_journal_get_create_access(const char *where, unsigned int line, return 0; } +static void ext4_inode_attach_mmb(struct inode *inode) +{ + struct mapping_metadata_bhs *mmb; + + /* + * It's difficult to handle failure when marking buffer dirty without + * leaving filesystem corrupted + */ + mmb = kmalloc_obj(*mmb, GFP_NOFS | __GFP_NOFAIL | __GFP_ACCOUNT); + mmb_init(mmb, &inode->i_data); + /* Someone swapped another mmb before us? */ + if (cmpxchg(&EXT4_I(inode)->i_metadata_bhs, NULL, mmb)) + kfree(mmb); +} + int __ext4_handle_dirty_metadata(const char *where, unsigned int line, handle_t *handle, struct inode *inode, struct buffer_head *bh) @@ -397,11 +412,13 @@ int __ext4_handle_dirty_metadata(const char *where, unsigned int line, err); } } else { - if (inode) - mmb_mark_buffer_dirty(bh, - &EXT4_I(inode)->i_metadata_bhs); - else + if (inode) { + if (!ext4_i_metadata_bhs(inode)) + ext4_inode_attach_mmb(inode); + mmb_mark_buffer_dirty(bh, ext4_i_metadata_bhs(inode)); + } else { mark_buffer_dirty(bh); + } if (inode && inode_needs_sync(inode)) { sync_dirty_buffer(bh); if (buffer_req(bh) && !buffer_uptodate(bh)) { diff --git a/fs/ext4/extents-test.c b/fs/ext4/extents-test.c index bd7795a82607..c3836ecb89f9 100644 --- a/fs/ext4/extents-test.c +++ b/fs/ext4/extents-test.c @@ -126,11 +126,6 @@ struct kunit_ext_test_param { struct kunit_ext_data_state exp_data_state[3]; }; -static void ext_kill_sb(struct super_block *sb) -{ - generic_shutdown_super(sb); -} - static int ext_init_fs_context(struct fs_context *fc) { return 0; @@ -138,13 +133,13 @@ static int ext_init_fs_context(struct fs_context *fc) static int ext_set(struct super_block *sb, struct fs_context *fc) { - return 0; + return set_anon_super_fc(sb, fc); } static struct file_system_type ext_fs_type = { .name = "extents test", .init_fs_context = ext_init_fs_context, - .kill_sb = ext_kill_sb, + .kill_sb = kill_anon_super, }; static void extents_kunit_exit(struct kunit *test) diff --git a/fs/ext4/extents.c b/fs/ext4/extents.c index d5f87a7f6c05..f258b2143d95 100644 --- a/fs/ext4/extents.c +++ b/fs/ext4/extents.c @@ -5196,8 +5196,10 @@ static int ext4_iomap_xattr_begin(struct inode *inode, loff_t offset, return error; } +static DEFINE_IOMAP_ITER_NEXT(ext4_iomap_xattr_next, ext4_iomap_xattr_begin); + static const struct iomap_ops ext4_iomap_xattr_ops = { - .iomap_begin = ext4_iomap_xattr_begin, + .iomap_next = ext4_iomap_xattr_next, }; static int ext4_fiemap_check_ranges(struct inode *inode, u64 start, u64 *len) diff --git a/fs/ext4/file.c b/fs/ext4/file.c index 9a16071b719d..2d6e20a5407a 100644 --- a/fs/ext4/file.c +++ b/fs/ext4/file.c @@ -91,7 +91,9 @@ static ssize_t ext4_dio_read_iter(struct kiocb *iocb, struct iov_iter *to) return generic_file_read_iter(iocb, to); } - ret = iomap_dio_rw(iocb, to, &ext4_iomap_ops, NULL, 0, NULL, 0); + ret = iomap_dio_read_simple(iocb, to, ext4_iomap_begin); + if (ret == -ENOTBLK) + ret = iomap_dio_rw(iocb, to, &ext4_iomap_ops, NULL, 0, NULL, 0); inode_unlock_shared(inode); file_accessed(iocb->ki_filp); diff --git a/fs/ext4/fsync.c b/fs/ext4/fsync.c index 924726dcc85f..2999c2cc8fcf 100644 --- a/fs/ext4/fsync.c +++ b/fs/ext4/fsync.c @@ -68,9 +68,6 @@ static int ext4_sync_parent(struct inode *inode) * through ext4_evict_inode()) and so we are safe to flush * metadata blocks and the inode. */ - ret = mmb_sync(&EXT4_I(inode)->i_metadata_bhs); - if (ret) - break; ret = sync_inode_metadata(inode, 1); if (ret) break; @@ -83,22 +80,11 @@ static int ext4_fsync_nojournal(struct file *file, loff_t start, loff_t end, int datasync, bool *needs_barrier) { struct inode *inode = file->f_inode; - struct writeback_control wbc = { - .sync_mode = WB_SYNC_ALL, - .nr_to_write = 0, - }; int ret; - ret = mmb_fsync_noflush(file, &EXT4_I(inode)->i_metadata_bhs, - start, end, datasync); + ret = sync_inode_metadata(inode, 1); if (ret) return ret; - - /* Force writeout of inode table buffer to disk */ - ret = ext4_write_inode(inode, &wbc); - if (ret) - return ret; - ret = ext4_sync_parent(inode); if (test_opt(inode->i_sb, BARRIER)) @@ -156,6 +142,10 @@ int ext4_sync_file(struct file *file, loff_t start, loff_t end, int datasync) if (sb_rdonly(inode->i_sb)) goto out; + ret = file_write_and_wait_range(file, start, end); + if (ret) + goto out; + if (!EXT4_SB(inode->i_sb)->s_journal) { ret = ext4_fsync_nojournal(file, start, end, datasync, &needs_barrier); @@ -164,10 +154,6 @@ int ext4_sync_file(struct file *file, loff_t start, loff_t end, int datasync) goto out; } - ret = file_write_and_wait_range(file, start, end); - if (ret) - goto out; - /* * The caller's filemap_fdatawrite()/wait will sync the data. * Metadata is in the journal, we wait for proper transaction to diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c index 853efadc944e..a120b0754be0 100644 --- a/fs/ext4/inode.c +++ b/fs/ext4/inode.c @@ -185,6 +185,8 @@ void ext4_evict_inode(struct inode *inode) if (EXT4_I(inode)->i_flags & EXT4_EA_INODE_FL) ext4_evict_ea_inode(inode); if (inode->i_nlink) { + struct mapping_metadata_bhs *mmb; + /* * If there's dirty page will lead to data loss, user * could see stale data. @@ -194,9 +196,9 @@ void ext4_evict_inode(struct inode *inode) ext4_warning_inode(inode, "data will be lost"); truncate_inode_pages_final(&inode->i_data); - /* Avoid mballoc special inode which has no proper iops */ - if (!EXT4_SB(inode->i_sb)->s_journal) - mmb_sync(&EXT4_I(inode)->i_metadata_bhs); + mmb = ext4_i_metadata_bhs(inode); + if (mmb) + mmb_sync(mmb); goto no_delete; } @@ -3439,6 +3441,7 @@ static bool ext4_release_folio(struct folio *folio, gfp_t wait) static bool ext4_inode_datasync_dirty(struct inode *inode) { journal_t *journal = EXT4_SB(inode->i_sb)->s_journal; + struct mapping_metadata_bhs *mmb; if (journal) { if (jbd2_transaction_committed(journal, @@ -3449,8 +3452,9 @@ static bool ext4_inode_datasync_dirty(struct inode *inode) return true; } + mmb = ext4_i_metadata_bhs(inode); /* Any metadata buffers to write? */ - if (mmb_has_buffers(&EXT4_I(inode)->i_metadata_bhs)) + if (mmb && mmb_has_buffers(mmb)) return true; return inode_state_read_once(inode) & I_DIRTY_DATASYNC; } @@ -3761,7 +3765,7 @@ retry: } -static int ext4_iomap_begin(struct inode *inode, loff_t offset, loff_t length, +int ext4_iomap_begin(struct inode *inode, loff_t offset, loff_t length, unsigned flags, struct iomap *iomap, struct iomap *srcmap) { int ret; @@ -3840,8 +3844,10 @@ out: return 0; } +static DEFINE_IOMAP_ITER_NEXT(ext4_iomap_next, ext4_iomap_begin); + const struct iomap_ops ext4_iomap_ops = { - .iomap_begin = ext4_iomap_begin, + .iomap_next = ext4_iomap_next, }; static int ext4_iomap_begin_report(struct inode *inode, loff_t offset, @@ -3895,8 +3901,10 @@ set_iomap: return 0; } +static DEFINE_IOMAP_ITER_NEXT(ext4_iomap_next_report, ext4_iomap_begin_report); + const struct iomap_ops ext4_iomap_report_ops = { - .iomap_begin = ext4_iomap_begin_report, + .iomap_next = ext4_iomap_next_report, }; /* @@ -5846,6 +5854,10 @@ out_brelse: * ext4_mark_inode_dirty(). This is a correctness thing for WB_SYNC_ALL * writeback. * + * For nojournal mode all the work is done in ext4_sync_inode_metadata() + * because inode content is already copied into raw inode buffer and inode + * is marked with I_METADATA_WRITEBACK. + * * Note that we are absolutely dependent upon all inode dirtiers doing the * right thing: they *must* call mark_inode_dirty() after dirtying info in * which we are interested. @@ -5871,42 +5883,54 @@ int ext4_write_inode(struct inode *inode, struct writeback_control *wbc) if (unlikely(err)) return err; - if (EXT4_SB(inode->i_sb)->s_journal) { - if (ext4_journal_current_handle()) { - ext4_debug("called recursively, non-PF_MEMALLOC!\n"); - dump_stack(); - return -EIO; - } + if (!EXT4_SB(inode->i_sb)->s_journal) + return 0; - /* - * No need to force transaction in WB_SYNC_NONE mode. Also - * ext4_sync_fs() will force the commit after everything is - * written. - */ - if (wbc->sync_mode != WB_SYNC_ALL || wbc->for_sync) - return 0; - - err = ext4_fc_commit(EXT4_SB(inode->i_sb)->s_journal, - EXT4_I(inode)->i_sync_tid); - } else { - struct ext4_iloc iloc; - - err = __ext4_get_inode_loc_noinmem(inode, &iloc); - if (err) - return err; - /* - * sync(2) will flush the whole buffer cache. No need to do - * it here separately for each inode. - */ - if (wbc->sync_mode == WB_SYNC_ALL && !wbc->for_sync) - sync_dirty_buffer(iloc.bh); - if (buffer_req(iloc.bh) && !buffer_uptodate(iloc.bh)) { - ext4_error_inode_block(inode, iloc.bh->b_blocknr, EIO, - "IO error syncing inode"); - err = -EIO; - } - brelse(iloc.bh); + if (ext4_journal_current_handle()) { + ext4_debug("called recursively, non-PF_MEMALLOC!\n"); + dump_stack(); + return -EIO; } + + /* + * No need to force transaction in WB_SYNC_NONE mode. Also + * ext4_sync_fs() will force the commit after everything is + * written. + */ + if (wbc->sync_mode != WB_SYNC_ALL || wbc->for_sync) + return 0; + + return ext4_fc_commit(EXT4_SB(inode->i_sb)->s_journal, + EXT4_I(inode)->i_sync_tid); +} + +int ext4_sync_inode_metadata(struct inode *inode, struct writeback_control *wbc) +{ + struct ext4_iloc iloc; + struct mapping_metadata_bhs *mmb; + int err; + + /* We should only get here in nojournal mode */ + if (WARN_ON_ONCE(EXT4_SB(inode->i_sb)->s_journal)) + return -EFSCORRUPTED; + + err = __ext4_get_inode_loc_noinmem(inode, &iloc); + if (err) + return err; + mmb = READ_ONCE(EXT4_I(inode)->i_metadata_bhs); + if (mmb) { + err = mmb_sync(mmb); + if (err) + goto out; + } + sync_dirty_buffer(iloc.bh); + if (buffer_write_io_error(iloc.bh)) { + ext4_error_inode_block(inode, iloc.bh->b_blocknr, EIO, + "IO error syncing inode"); + err = -EIO; + } +out: + brelse(iloc.bh); return err; } @@ -6447,6 +6471,20 @@ int ext4_mark_iloc_dirty(handle_t *handle, /* ext4_do_update_inode() does jbd2_journal_dirty_metadata */ err = ext4_do_update_inode(handle, inode, iloc); put_bh(iloc->bh); + /* + * Mark that there's metadata writeout pending for the inode so that it + * gets properly flushed on fsync(2) and similar. + */ + if (!EXT4_SB(inode->i_sb)->s_journal) { + /* + * Inode didn't need to go through dirtying, make sure it is + * attached to wb so that writeback can handle it. + */ + spin_lock(&inode->i_lock); + inode_attach_wb(inode, NULL); + spin_unlock(&inode->i_lock); + set_inode_metadata_writeback(inode); + } return err; } diff --git a/fs/ext4/mballoc-test.c b/fs/ext4/mballoc-test.c index 0424b8b0b4c3..d31780075c21 100644 --- a/fs/ext4/mballoc-test.c +++ b/fs/ext4/mballoc-test.c @@ -59,11 +59,6 @@ static const struct super_operations mbt_sops = { .free_inode = mbt_free_inode, }; -static void mbt_kill_sb(struct super_block *sb) -{ - generic_shutdown_super(sb); -} - static int mbt_init_fs_context(struct fs_context *fc) { return 0; @@ -72,7 +67,7 @@ static int mbt_init_fs_context(struct fs_context *fc) static struct file_system_type mbt_fs_type = { .name = "mballoc test", .init_fs_context = mbt_init_fs_context, - .kill_sb = mbt_kill_sb, + .kill_sb = kill_anon_super, }; static int mbt_mb_init(struct super_block *sb) @@ -136,7 +131,7 @@ static void mbt_mb_release(struct super_block *sb) static int mbt_set(struct super_block *sb, struct fs_context *fc) { - return 0; + return set_anon_super_fc(sb, fc); } static struct super_block *mbt_ext4_alloc_super_block(void) diff --git a/fs/ext4/namei.c b/fs/ext4/namei.c index 3b9740c1c16d..a6386c1d237f 100644 --- a/fs/ext4/namei.c +++ b/fs/ext4/namei.c @@ -2813,7 +2813,7 @@ static int ext4_add_nondir(handle_t *handle, * with d_instantiate(). */ static int ext4_create(struct mnt_idmap *idmap, struct inode *dir, - struct dentry *dentry, umode_t mode, bool excl) + struct dentry *dentry, umode_t mode) { handle_t *handle; struct inode *inode; @@ -3011,7 +3011,7 @@ static struct dentry *ext4_mkdir(struct mnt_idmap *idmap, struct inode *dir, credits = (EXT4_DATA_TRANS_BLOCKS(dir->i_sb) + EXT4_INDEX_EXTRA_TRANS_BLOCKS + 3); retry: - inode = ext4_new_inode_start_handle(idmap, dir, S_IFDIR | mode, + inode = ext4_new_inode_start_handle(idmap, dir, mode, &dentry->d_name, 0, NULL, EXT4_HT_DIR, credits); handle = ext4_journal_current_handle(); diff --git a/fs/ext4/super.c b/fs/ext4/super.c index 32c8e33b5036..5ec7ddd9d64a 100644 --- a/fs/ext4/super.c +++ b/fs/ext4/super.c @@ -1439,7 +1439,7 @@ static struct inode *ext4_alloc_inode(struct super_block *sb) INIT_WORK(&ei->i_rsv_conversion_work, ext4_end_io_rsv_work); ext4_fc_init_inode(&ei->vfs_inode); spin_lock_init(&ei->i_fc_lock); - mmb_init(&ei->i_metadata_bhs, &ei->vfs_inode.i_data); + ei->i_metadata_bhs = NULL; #ifdef CONFIG_LOCKDEP lockdep_set_subclass(&ei->i_data_sem, I_DATA_SEM_NORMAL); #endif @@ -1460,6 +1460,7 @@ static int ext4_drop_inode(struct inode *inode) static void ext4_free_in_core_inode(struct inode *inode) { fscrypt_free_inode(inode); + kfree(ext4_i_metadata_bhs(inode)); if (!list_empty(&(EXT4_I(inode)->i_fc_list))) { pr_warn("%s: inode %llu still in fc list", __func__, inode->i_ino); @@ -1537,9 +1538,11 @@ static void destroy_inodecache(void) void ext4_clear_inode(struct inode *inode) { + struct mapping_metadata_bhs *mmb = ext4_i_metadata_bhs(inode); + ext4_fc_del(inode); - if (!EXT4_SB(inode->i_sb)->s_journal) - mmb_invalidate(&EXT4_I(inode)->i_metadata_bhs); + if (mmb) + mmb_invalidate(mmb); clear_inode(inode); ext4_discard_preallocations(inode); /* @@ -1613,9 +1616,13 @@ static int ext4_nfs_commit_metadata(struct inode *inode) struct writeback_control wbc = { .sync_mode = WB_SYNC_ALL }; + int ret; trace_ext4_nfs_commit_metadata(inode); - return ext4_write_inode(inode, &wbc); + ret = ext4_write_inode(inode, &wbc); + if (!ret && inode_state_read_once(inode) & I_METADATA_WRITEBACK) + ret = ext4_sync_inode_metadata(inode, &wbc); + return ret; } #ifdef CONFIG_QUOTA @@ -1672,6 +1679,7 @@ static const struct super_operations ext4_sops = { .free_inode = ext4_free_in_core_inode, .destroy_inode = ext4_destroy_inode, .write_inode = ext4_write_inode, + .sync_inode_metadata = ext4_sync_inode_metadata, .dirty_inode = ext4_dirty_inode, .drop_inode = ext4_drop_inode, .evict_inode = ext4_evict_inode, @@ -5816,7 +5824,7 @@ failed_mount: brelse(sbi->s_sbh); if (sbi->s_journal_bdev_file) { invalidate_bdev(file_bdev(sbi->s_journal_bdev_file)); - bdev_fput(sbi->s_journal_bdev_file); + fs_bdev_file_release(sbi->s_journal_bdev_file, sb); } out_fail: invalidate_bdev(sb->s_bdev); @@ -6000,9 +6008,9 @@ static struct file *ext4_get_journal_blkdev(struct super_block *sb, struct ext4_super_block *es; int errno; - bdev_file = bdev_file_open_by_dev(j_dev, + bdev_file = fs_bdev_file_open_by_dev(j_dev, BLK_OPEN_READ | BLK_OPEN_WRITE | BLK_OPEN_RESTRICT_WRITES, - sb, &fs_holder_ops); + sb, sb); if (IS_ERR(bdev_file)) { ext4_msg(sb, KERN_ERR, "failed to open journal device unknown-block(%u,%u) %pe", @@ -6062,7 +6070,7 @@ static struct file *ext4_get_journal_blkdev(struct super_block *sb, out_bh: brelse(bh); out_bdev: - bdev_fput(bdev_file); + fs_bdev_file_release(bdev_file, sb); return ERR_PTR(errno); } @@ -6101,7 +6109,7 @@ static journal_t *ext4_open_dev_journal(struct super_block *sb, out_journal: ext4_journal_destroy(EXT4_SB(sb), journal); out_bdev: - bdev_fput(bdev_file); + fs_bdev_file_release(bdev_file, sb); return ERR_PTR(errno); } @@ -7519,7 +7527,7 @@ static void ext4_kill_sb(struct super_block *sb) kill_block_super(sb); if (bdev_file) - bdev_fput(bdev_file); + fs_bdev_file_release(bdev_file, sb); } static struct file_system_type ext4_fs_type = { diff --git a/fs/f2fs/data.c b/fs/f2fs/data.c index 1b2d9fe992ea..4a6e1b6b97f0 100644 --- a/fs/f2fs/data.c +++ b/fs/f2fs/data.c @@ -4579,6 +4579,8 @@ static int f2fs_iomap_begin(struct inode *inode, loff_t offset, loff_t length, return 0; } +static DEFINE_IOMAP_ITER_NEXT(f2fs_iomap_next, f2fs_iomap_begin); + const struct iomap_ops f2fs_iomap_ops = { - .iomap_begin = f2fs_iomap_begin, + .iomap_next = f2fs_iomap_next, }; diff --git a/fs/f2fs/namei.c b/fs/f2fs/namei.c index cac03b8e91a1..5ae647a352aa 100644 --- a/fs/f2fs/namei.c +++ b/fs/f2fs/namei.c @@ -366,7 +366,7 @@ fail_drop: } static int f2fs_create(struct mnt_idmap *idmap, struct inode *dir, - struct dentry *dentry, umode_t mode, bool excl) + struct dentry *dentry, umode_t mode) { struct f2fs_sb_info *sbi = F2FS_I_SB(dir); struct f2fs_lock_context lc; @@ -742,7 +742,7 @@ static struct dentry *f2fs_mkdir(struct mnt_idmap *idmap, struct inode *dir, if (err) return ERR_PTR(err); - inode = f2fs_new_inode(idmap, dir, S_IFDIR | mode, NULL); + inode = f2fs_new_inode(idmap, dir, mode, NULL); if (IS_ERR(inode)) return ERR_CAST(inode); diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c index 769d16d54997..de99eead6f8d 100644 --- a/fs/f2fs/super.c +++ b/fs/f2fs/super.c @@ -1971,7 +1971,7 @@ static void destroy_device_list(struct f2fs_sb_info *sbi) for (i = 0; i < sbi->s_ndevs; i++) { if (i > 0) - bdev_fput(FDEV(i).bdev_file); + fs_bdev_file_release(FDEV(i).bdev_file, sbi->sb); #ifdef CONFIG_BLK_DEV_ZONED kvfree(FDEV(i).blkz_seq); #endif @@ -4901,8 +4901,8 @@ static int f2fs_scan_devices(struct f2fs_sb_info *sbi) FDEV(i).end_blk = FDEV(i).start_blk + SEGS_TO_BLKS(sbi, FDEV(i).total_segments) - 1; - FDEV(i).bdev_file = bdev_file_open_by_path( - FDEV(i).path, mode, sbi->sb, NULL); + FDEV(i).bdev_file = fs_bdev_file_open_by_path( + FDEV(i).path, mode, sbi->sb, sbi->sb); } } if (IS_ERR(FDEV(i).bdev_file)) diff --git a/fs/failfs.c b/fs/failfs.c new file mode 100644 index 000000000000..66a36da3d236 --- /dev/null +++ b/fs/failfs.c @@ -0,0 +1,166 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* Copyright (c) 2026 Christian Brauner */ +#include +#include +#include +#include +#include +#include + +#include "internal.h" + +static struct path failfs_root_path = {}; + +void failfs_get_root(struct path *path) +{ + *path = failfs_root_path; + path_get(path); +} + +bool failfs_mnt(const struct vfsmount *mnt) +{ + return mnt->mnt_sb == failfs_root_path.mnt->mnt_sb; +} + +static int failfs_permission(struct mnt_idmap *idmap, struct inode *inode, + int mask) +{ + return -EOPNOTSUPP; +} + +static struct dentry *failfs_lookup(struct inode *dir, struct dentry *dentry, + unsigned int flags) +{ + /* Unreachable: ->permission() already failed the walk. */ + return ERR_PTR(-EOPNOTSUPP); +} + +static int failfs_getattr(struct mnt_idmap *idmap, const struct path *path, + struct kstat *stat, u32 request_mask, + unsigned int query_flags) +{ + return -EOPNOTSUPP; +} + +static const struct inode_operations failfs_dir_inode_operations = { + .permission = failfs_permission, + .lookup = failfs_lookup, + .getattr = failfs_getattr, +}; + +static const struct file_operations failfs_dir_operations = {}; + +static int failfs_d_weak_revalidate(struct dentry *dentry, unsigned int flags) +{ + /* + * The root is only ever reached as a path-walk terminal by jumping + * to it: as "/" when it is the caller's root, or through a + * /proc//{root,cwd} magic link. ->permission() already fails + * every walk of a component, but a jump lands on the root without + * one. Refuse here too so the root cannot be pinned by an O_PATH + * open or encoded into a file handle. + */ + return -EOPNOTSUPP; +} + +static char *failfs_dname(struct dentry *dentry, char *buffer, int buflen) +{ + return dynamic_dname(buffer, buflen, "failfs:/"); +} + +static const struct dentry_operations failfs_dentry_operations = { + .d_dname = failfs_dname, + .d_weak_revalidate = failfs_d_weak_revalidate, +}; + +static int failfs_statfs(struct dentry *dentry, struct kstatfs *buf) +{ + return -EOPNOTSUPP; +} + +static const struct super_operations failfs_super_operations = { + .statfs = failfs_statfs, +}; + +static int failfs_fill_super(struct super_block *s, struct fs_context *fc) +{ + struct inode *inode; + + s->s_maxbytes = MAX_LFS_FILESIZE; + s->s_blocksize = PAGE_SIZE; + s->s_blocksize_bits = PAGE_SHIFT; + s->s_magic = FAIL_FS_MAGIC; + s->s_op = &failfs_super_operations; + s->s_export_op = NULL; + s->s_xattr = NULL; + s->s_time_gran = 1; + s->s_d_flags = 0; + + inode = new_inode(s); + if (!inode) + return -ENOMEM; + + /* failfs supports no operations... */ + inode->i_mode = S_IFDIR; + set_nlink(inode, 2); + inode->i_op = &failfs_dir_inode_operations; + inode->i_fop = &failfs_dir_operations; + simple_inode_init_ts(inode); + inode->i_ino = 1; + /* ... and is immutable. */ + inode->i_flags |= S_IMMUTABLE; + + set_default_d_op(s, &failfs_dentry_operations); + s->s_root = d_make_root(inode); + if (!s->s_root) + return -ENOMEM; + + return 0; +} + +static int failfs_get_tree(struct fs_context *fc) +{ + return get_tree_single(fc, failfs_fill_super); +} + +static const struct fs_context_operations failfs_context_ops = { + .get_tree = failfs_get_tree, +}; + +static int failfs_init_fs_context(struct fs_context *fc) +{ + fc->ops = &failfs_context_ops; + fc->global = true; + fc->sb_flags |= SB_NOUSER; + fc->s_iflags |= SB_I_NOEXEC | SB_I_NODEV; + return 0; +} + +int failfs_current_chdir(void) +{ + struct path path; + + failfs_get_root(&path); + set_fs_pwd(current->fs, &path); + path_put(&path); + return 0; +} + +static struct file_system_type failfs_fs_type = { + .name = "failfs", + .init_fs_context = failfs_init_fs_context, + .kill_sb = kill_anon_super, +}; + +void __init failfs_init(void) +{ + struct vfsmount *mnt; + + /* A single instance that is member of no mount namespace. */ + mnt = kern_mount(&failfs_fs_type); + if (IS_ERR(mnt)) + panic("VFS: Failed to create failfs"); + + failfs_root_path.mnt = mnt; + failfs_root_path.dentry = mnt->mnt_root; +} diff --git a/fs/fat/dir.c b/fs/fat/dir.c index c6cca5d00ffd..35bdb62944a2 100644 --- a/fs/fat/dir.c +++ b/fs/fat/dir.c @@ -1109,10 +1109,10 @@ int fat_remove_entries(struct inode *dir, struct fat_slot_info *sinfo) } fat_truncate_time(dir, NULL, FAT_UPDATE_ATIME | FAT_UPDATE_CMTIME); + err = 0; + mark_inode_dirty(dir); if (IS_DIRSYNC(dir)) - (void)fat_sync_inode(dir); - else - mark_inode_dirty(dir); + (void)sync_inode_metadata(dir, 1); return 0; } diff --git a/fs/fat/fat.h b/fs/fat/fat.h index 2772675bd35a..61338413d9f3 100644 --- a/fs/fat/fat.h +++ b/fs/fat/fat.h @@ -421,7 +421,6 @@ extern void fat_detach(struct inode *inode); extern struct inode *fat_iget(struct super_block *sb, loff_t i_pos); extern struct inode *fat_build_inode(struct super_block *sb, struct msdos_dir_entry *de, loff_t i_pos); -extern int fat_sync_inode(struct inode *inode); extern int fat_fill_super(struct super_block *sb, struct fs_context *fc, void (*setup)(struct super_block *)); extern int fat_fill_inode(struct inode *inode, struct msdos_dir_entry *de); diff --git a/fs/fat/file.c b/fs/fat/file.c index 37e7049b4c8c..1c835ca5f21a 100644 --- a/fs/fat/file.c +++ b/fs/fat/file.c @@ -190,8 +190,7 @@ int fat_file_fsync(struct file *filp, loff_t start, loff_t end, int datasync) struct inode *fat_inode = MSDOS_SB(inode->i_sb)->fat_inode; int err; - err = mmb_fsync_noflush(filp, &MSDOS_I(inode)->i_metadata_bhs, - start, end, datasync); + err = simple_fsync_noflush(filp, start, end, datasync); if (err) return err; @@ -332,15 +331,15 @@ static int fat_free(struct inode *inode, int skip) } MSDOS_I(inode)->i_attrs |= ATTR_ARCH; fat_truncate_time(inode, NULL, FAT_UPDATE_CMTIME); + mark_inode_dirty(inode); if (wait) { - err = fat_sync_inode(inode); + err = sync_inode_metadata(inode, 1); if (err) { MSDOS_I(inode)->i_start = i_start; MSDOS_I(inode)->i_logstart = i_logstart; return err; } - } else - mark_inode_dirty(inode); + } /* Write a new EOF, and get the remaining cluster chain for freeing. */ if (skip) { diff --git a/fs/fat/inode.c b/fs/fat/inode.c index 3aa52481ad5c..5ea6f74a2a3f 100644 --- a/fs/fat/inode.c +++ b/fs/fat/inode.c @@ -623,7 +623,41 @@ out: EXPORT_SYMBOL_GPL(fat_build_inode); -static int __fat_write_inode(struct inode *inode, int wait); +static int __fat_write_inode(struct inode *inode); + +static int fat_sync_inode_metadata(struct inode *inode, + struct writeback_control *wbc) +{ + struct msdos_sb_info *sbi = MSDOS_SB(inode->i_sb); + struct buffer_head *bh; + loff_t i_pos; + sector_t blocknr; + int offset; + + /* The root directory has no directory entry of its own. */ + if (inode->i_ino == MSDOS_ROOT_INO) + goto sync_bhs; + i_pos = fat_i_pos_read(sbi, inode); + if (!i_pos) + goto sync_bhs; + + fat_get_blknr_offset(sbi, i_pos, &blocknr, &offset); + bh = sb_find_get_block_nonatomic(inode->i_sb, blocknr); + /* + * Buffer present? We leave buffer_dirty check for sync_dirty_buffer() + * for proper synchronization with ongoing IO. + */ + if (bh && buffer_uptodate(bh)) { + sync_dirty_buffer(bh); + if (buffer_write_io_error(bh)) { + brelse(bh); + return -EIO; + } + } + brelse(bh); +sync_bhs: + return mmb_sync(&MSDOS_I(inode)->i_metadata_bhs); +} static void fat_free_eofblocks(struct inode *inode) { @@ -640,7 +674,7 @@ static void fat_free_eofblocks(struct inode *inode) * any corruption on the next access to the cluster * chain for the file. */ - err = __fat_write_inode(inode, inode_needs_sync(inode)); + err = sync_inode_metadata(inode, inode_needs_sync(inode)); if (err) { fat_msg(inode->i_sb, KERN_WARNING, "Failed to " "update on disk inode for unused " @@ -854,7 +888,7 @@ static int fat_statfs(struct dentry *dentry, struct kstatfs *buf) return 0; } -static int __fat_write_inode(struct inode *inode, int wait) +static int __fat_write_inode(struct inode *inode) { struct super_block *sb = inode->i_sb; struct msdos_sb_info *sbi = MSDOS_SB(sb); @@ -863,10 +897,13 @@ static int __fat_write_inode(struct inode *inode, int wait) struct timespec64 mtime; loff_t i_pos; sector_t blocknr; - int err, offset; + int offset; - if (inode->i_ino == MSDOS_ROOT_INO) + if (inode->i_ino == MSDOS_ROOT_INO) { + /* No entry to update but the metadata bh list may need syncing. */ + set_inode_metadata_writeback(inode); return 0; + } retry: i_pos = fat_i_pos_read(sbi, inode); @@ -907,11 +944,9 @@ retry: } spin_unlock(&sbi->inode_hash_lock); mark_buffer_dirty(bh); - err = 0; - if (wait) - err = sync_dirty_buffer(bh); brelse(bh); - return err; + set_inode_metadata_writeback(inode); + return 0; } static int fat_write_inode(struct inode *inode, struct writeback_control *wbc) @@ -925,23 +960,17 @@ static int fat_write_inode(struct inode *inode, struct writeback_control *wbc) err = fat_clusters_flush(sb); mutex_unlock(&MSDOS_SB(sb)->s_lock); } else - err = __fat_write_inode(inode, wbc->sync_mode == WB_SYNC_ALL); + err = __fat_write_inode(inode); return err; } -int fat_sync_inode(struct inode *inode) -{ - return __fat_write_inode(inode, 1); -} - -EXPORT_SYMBOL_GPL(fat_sync_inode); - static int fat_show_options(struct seq_file *m, struct dentry *root); static const struct super_operations fat_sops = { .alloc_inode = fat_alloc_inode, .free_inode = fat_free_inode, .write_inode = fat_write_inode, + .sync_inode_metadata = fat_sync_inode_metadata, .evict_inode = fat_evict_inode, .put_super = fat_put_super, .statfs = fat_statfs, diff --git a/fs/fat/misc.c b/fs/fat/misc.c index 3027ef53af21..be18f6b5819b 100644 --- a/fs/fat/misc.c +++ b/fs/fat/misc.c @@ -146,16 +146,17 @@ int fat_chain_add(struct inode *inode, int new_dclus, int nr_cluster) } else { MSDOS_I(inode)->i_start = new_dclus; MSDOS_I(inode)->i_logstart = new_dclus; + mark_inode_dirty(inode); /* * Since generic_write_sync() synchronizes regular files later, * we sync here only directories. */ if (S_ISDIR(inode->i_mode) && IS_DIRSYNC(inode)) { - ret = fat_sync_inode(inode); + ret = sync_inode_metadata(inode, 1); if (ret) return ret; - } else - mark_inode_dirty(inode); + } + } if (new_fclus != (inode->i_blocks >> (sbi->cluster_bits - 9))) { fat_fs_error_ratelimit( diff --git a/fs/fat/namei_msdos.c b/fs/fat/namei_msdos.c index 0fd2971ad4b1..d46d1a3851f2 100644 --- a/fs/fat/namei_msdos.c +++ b/fs/fat/namei_msdos.c @@ -29,6 +29,9 @@ static int msdos_format_name(const unsigned char *name, int len, unsigned char c; int space; + if (len > NAME_MAX) + return -ENAMETOOLONG; + if (name[0] == '.') { /* dotfile because . and .. already done */ if (opts->dotsOK) { /* Get rid of dot - test for it elsewhere */ @@ -252,17 +255,16 @@ static int msdos_add_entry(struct inode *dir, const unsigned char *name, return err; fat_truncate_time(dir, ts, FAT_UPDATE_CMTIME); + mark_inode_dirty(dir); if (IS_DIRSYNC(dir)) - (void)fat_sync_inode(dir); - else - mark_inode_dirty(dir); + (void)sync_inode_metadata(dir, 1); return 0; } /***** Create a file */ static int msdos_create(struct mnt_idmap *idmap, struct inode *dir, - struct dentry *dentry, umode_t mode, bool excl) + struct dentry *dentry, umode_t mode) { struct super_block *sb = dir->i_sb; struct inode *inode = NULL; @@ -473,21 +475,20 @@ static int do_msdos_rename(struct inode *old_dir, unsigned char *old_name, MSDOS_I(old_inode)->i_attrs |= ATTR_HIDDEN; else MSDOS_I(old_inode)->i_attrs &= ~ATTR_HIDDEN; + mark_inode_dirty(old_inode); if (IS_DIRSYNC(old_dir)) { - err = fat_sync_inode(old_inode); + err = sync_inode_metadata(old_inode, 1); if (err) { MSDOS_I(old_inode)->i_attrs = old_attrs; goto out; } - } else - mark_inode_dirty(old_inode); + } inode_inc_iversion(old_dir); fat_truncate_time(old_dir, NULL, FAT_UPDATE_CMTIME); + mark_inode_dirty(old_dir); if (IS_DIRSYNC(old_dir)) - (void)fat_sync_inode(old_dir); - else - mark_inode_dirty(old_dir); + (void)sync_inode_metadata(old_dir, 1); goto out; } } @@ -518,12 +519,12 @@ static int do_msdos_rename(struct inode *old_dir, unsigned char *old_name, MSDOS_I(old_inode)->i_attrs |= ATTR_HIDDEN; else MSDOS_I(old_inode)->i_attrs &= ~ATTR_HIDDEN; + mark_inode_dirty(old_inode); if (IS_DIRSYNC(new_dir)) { - err = fat_sync_inode(old_inode); + err = sync_inode_metadata(old_inode, 1); if (err) goto error_inode; - } else - mark_inode_dirty(old_inode); + } if (update_dotdot) { fat_set_start(dotdot_de, MSDOS_I(new_dir)->i_logstart); @@ -545,10 +546,9 @@ static int do_msdos_rename(struct inode *old_dir, unsigned char *old_name, goto error_dotdot; inode_inc_iversion(old_dir); fat_truncate_time(old_dir, &ts, FAT_UPDATE_CMTIME); + mark_inode_dirty(old_dir); if (IS_DIRSYNC(old_dir)) - (void)fat_sync_inode(old_dir); - else - mark_inode_dirty(old_dir); + (void)sync_inode_metadata(old_dir, 1); if (new_inode) { drop_nlink(new_inode); @@ -577,8 +577,10 @@ error_inode: MSDOS_I(old_inode)->i_attrs = old_attrs; if (new_inode) { fat_attach(new_inode, new_i_pos); - if (corrupt) - corrupt |= fat_sync_inode(new_inode); + if (corrupt) { + mark_inode_dirty(new_inode); + corrupt |= sync_inode_metadata(new_inode, 1); + } } else { /* * If new entry was not sharing the data cluster, it diff --git a/fs/fat/namei_vfat.c b/fs/fat/namei_vfat.c index e909447873e3..da3e89c0b16a 100644 --- a/fs/fat/namei_vfat.c +++ b/fs/fat/namei_vfat.c @@ -678,10 +678,9 @@ static int vfat_add_entry(struct inode *dir, const struct qstr *qname, /* update timestamp */ fat_truncate_time(dir, ts, FAT_UPDATE_CMTIME); + mark_inode_dirty(dir); if (IS_DIRSYNC(dir)) - (void)fat_sync_inode(dir); - else - mark_inode_dirty(dir); + (void)sync_inode_metadata(dir, 1); cleanup: kfree(slots); return err; @@ -755,7 +754,7 @@ error: } static int vfat_create(struct mnt_idmap *idmap, struct inode *dir, - struct dentry *dentry, umode_t mode, bool excl) + struct dentry *dentry, umode_t mode) { struct super_block *sb = dir->i_sb; struct inode *inode; @@ -904,9 +903,9 @@ static int vfat_get_dotdot_de(struct inode *inode, struct buffer_head **bh, static int vfat_sync_ipos(struct inode *dir, struct inode *inode) { - if (IS_DIRSYNC(dir)) - return fat_sync_inode(inode); mark_inode_dirty(inode); + if (IS_DIRSYNC(dir)) + return sync_inode_metadata(inode, 1); return 0; } @@ -925,10 +924,9 @@ static void vfat_update_dir_metadata(struct inode *dir, struct timespec64 *ts) { inode_inc_iversion(dir); fat_truncate_time(dir, ts, FAT_UPDATE_CMTIME); + mark_inode_dirty(dir); if (IS_DIRSYNC(dir)) - (void)fat_sync_inode(dir); - else - mark_inode_dirty(dir); + (void)sync_inode_metadata(dir, 1); } static int vfat_rename(struct inode *old_dir, struct dentry *old_dentry, @@ -1024,8 +1022,10 @@ error_inode: fat_attach(old_inode, old_sinfo.i_pos); if (new_inode) { fat_attach(new_inode, new_i_pos); - if (corrupt) - corrupt |= fat_sync_inode(new_inode); + if (corrupt) { + mark_inode_dirty(new_inode); + corrupt |= sync_inode_metadata(new_inode, 1); + } } else { /* * If new entry was not sharing the data cluster, it diff --git a/fs/fs-writeback.c b/fs/fs-writeback.c index fdb8766d275a..71dd618db075 100644 --- a/fs/fs-writeback.c +++ b/fs/fs-writeback.c @@ -299,6 +299,7 @@ void __inode_attach_wb(struct inode *inode, struct folio *folio) if (unlikely(cmpxchg(&inode->i_wb, NULL, wb))) wb_put(wb); } +EXPORT_SYMBOL_GPL(__inode_attach_wb); /** * inode_cgwb_move_to_attached - put the inode onto wb->b_attached list @@ -1851,6 +1852,22 @@ __writeback_single_inode(struct inode *inode, struct writeback_control *wbc) if (ret == 0) ret = err; } + + /* + * Do we need to wait for inode metadata IO possibly submitted + * by previous WB_SYNC_NONE writeback? + */ + if (wbc->sync_mode == WB_SYNC_ALL && !wbc->for_sync && + inode_state_read_once(inode) & I_METADATA_WRITEBACK) { + int err; + + spin_lock(&inode->i_lock); + inode_state_clear(inode, I_METADATA_WRITEBACK); + spin_unlock(&inode->i_lock); + err = inode->i_sb->s_op->sync_inode_metadata(inode, wbc); + if (ret == 0) + ret = err; + } wbc->unpinned_netfs_wb = false; trace_writeback_single_inode(inode, wbc, nr_to_write); return ret; @@ -1892,14 +1909,17 @@ static int writeback_single_inode(struct inode *inode, /* * If the inode is already fully clean, then there's nothing to do. * - * For data-integrity syncs we also need to check whether any pages are - * still under writeback, e.g. due to prior WB_SYNC_NONE writeback. If - * there are any such pages, we'll need to wait for them. + * For data-integrity syncs we also need to check whether any folios or + * metadata are still under writeback, e.g. due to prior WB_SYNC_NONE + * writeback. If there, we'll need to wait for them. */ - if (!(inode_state_read(inode) & I_DIRTY_ALL) && - (wbc->sync_mode != WB_SYNC_ALL || - !mapping_tagged(inode->i_mapping, PAGECACHE_TAG_WRITEBACK))) - goto out; + if (!(inode_state_read(inode) & I_DIRTY_ALL)) { + if (wbc->sync_mode != WB_SYNC_ALL) + goto out; + if (!mapping_tagged(inode->i_mapping, PAGECACHE_TAG_WRITEBACK) && + !(inode_state_read(inode) & I_METADATA_WRITEBACK)) + goto out; + } inode_state_set(inode, I_SYNC); wbc_attach_and_unlock_inode(wbc, inode); diff --git a/fs/fs_struct.c b/fs/fs_struct.c index 394875d06fd6..34699f3b6f88 100644 --- a/fs/fs_struct.c +++ b/fs/fs_struct.c @@ -8,6 +8,7 @@ #include #include #include "internal.h" +#include "mount.h" /* * Replace the fs->{rootmnt,root} with {mnt,dentry}. Put the old values. @@ -60,8 +61,11 @@ void chroot_fs_refs(const struct path *old_root, const struct path *new_root) read_lock(&tasklist_lock); for_each_process_thread(g, p) { + if (p->flags & (PF_KTHREAD | PF_EXITING | PF_DUMPCORE)) + continue; + task_lock(p); - fs = p->fs; + fs = p->real_fs; if (fs) { int hits = 0; write_seqlock(&fs->seq); @@ -89,12 +93,13 @@ void free_fs_struct(struct fs_struct *fs) void exit_fs(struct task_struct *tsk) { - struct fs_struct *fs = tsk->fs; + struct fs_struct *fs = tsk->real_fs; if (fs) { int kill; task_lock(tsk); read_seqlock_excl(&fs->seq); + tsk->real_fs = NULL; tsk->fs = NULL; kill = !--fs->users; read_sequnlock_excl(&fs->seq); @@ -126,7 +131,7 @@ struct fs_struct *copy_fs_struct(struct fs_struct *old) int unshare_fs_struct(void) { - struct fs_struct *fs = current->fs; + struct fs_struct *fs = current->real_fs; struct fs_struct *new_fs = copy_fs_struct(fs); int kill; @@ -135,8 +140,10 @@ int unshare_fs_struct(void) task_lock(current); read_seqlock_excl(&fs->seq); + VFS_WARN_ON_ONCE(fs != current->fs); kill = !--fs->users; current->fs = new_fs; + current->real_fs = new_fs; read_sequnlock_excl(&fs->seq); task_unlock(current); @@ -147,9 +154,99 @@ int unshare_fs_struct(void) } EXPORT_SYMBOL_GPL(unshare_fs_struct); +/* + * PID 1 may choose to stop sharing fs_struct state with us. + * Either via unshare(CLONE_FS) or unshare(CLONE_NEWNS). Of + * course, PID 1 could have chosen to create arbitrary process + * trees that all share fs_struct state via CLONE_FS. This is a + * strong statement: We only care about PID 1 aka the thread-group + * leader so subthread's fs_struct state doesn't matter. + * + * PID 1 unsharing fs_struct state is a bug. PID 1 relies on + * various kthreads to be able to perform work based on its + * fs_struct state. Breaking that contract sucks for both sides. + * So just don't bother with extra work for this. No sane init + * system should ever do this. + * + * On older kernels if PID 1 unshared its filesystem state with us the + * kernel simply used the stale fs_struct state implicitly pinning + * anything that PID 1 had last used. Even if PID 1 might've moved on to + * some completely different fs_struct state and might've even unmounted + * the old root. + * + * This has hilarious consequences: Think continuing to dump coredump + * state into an implicitly pinned directory somewhere. Calling random + * binaries in the old rootfs via usermodehelpers. + * + * Be aggressive about this: We simply reject operating on stale + * fs_struct state by reverting to nullfs. Every kworker that does + * lookups after this point will fail. Every usermodehelper call will + * fail. Tough luck but let's be kind and emit a warning to userspace. + */ +static inline void validate_fs_switch(struct fs_struct *old_fs) +{ + might_sleep(); + + if (likely(current->pid != 1)) + return; + /* @old_fs may be dangling but for comparison it's fine */ + if (old_fs != userspace_init_fs) + return; + pr_warn("VFS: Pid 1 stopped sharing filesystem state\n"); + set_fs_root(userspace_init_fs, &init_fs.root); + set_fs_pwd(userspace_init_fs, &init_fs.root); +} + +struct fs_struct *switch_fs_struct(struct fs_struct *new_fs) +{ + struct fs_struct *fs; + + scoped_guard(task_lock, current) { + fs = current->fs; + VFS_WARN_ON_ONCE(fs != current->real_fs); + read_seqlock_excl(&fs->seq); + current->fs = new_fs; + current->real_fs = new_fs; + if (--fs->users) + new_fs = NULL; + else + new_fs = fs; + read_sequnlock_excl(&fs->seq); + } + + validate_fs_switch(fs); + return new_fs; +} + /* to be mentioned only in INIT_TASK */ struct fs_struct init_fs = { .users = 1, .seq = __SEQLOCK_UNLOCKED(init_fs.seq), .umask = 0022, }; + +struct fs_struct *userspace_init_fs __ro_after_init; +EXPORT_SYMBOL_GPL(userspace_init_fs); + +void __init init_userspace_fs(void) +{ + struct mount *m; + struct path root; + + /* Move PID 1 from nullfs into the initramfs. */ + m = topmost_overmount(current->nsproxy->mnt_ns->root); + root.mnt = &m->mnt; + root.dentry = root.mnt->mnt_root; + + VFS_WARN_ON_ONCE(current->pid != 1); + + set_fs_root(current->fs, &root); + set_fs_pwd(current->fs, &root); + + /* Hold a reference for the global pointer. */ + read_seqlock_excl(¤t->fs->seq); + current->fs->users++; + read_sequnlock_excl(¤t->fs->seq); + + userspace_init_fs = current->fs; +} diff --git a/fs/fuse/dax.c b/fs/fuse/dax.c index 8b53625ac7ab..85cdf0199bc0 100644 --- a/fs/fuse/dax.c +++ b/fs/fuse/dax.c @@ -653,9 +653,11 @@ static int fuse_iomap_end(struct inode *inode, loff_t pos, loff_t length, return 0; } +static DEFINE_IOMAP_ITER_NEXT_END(fuse_iomap_next, fuse_iomap_begin, + fuse_iomap_end); + static const struct iomap_ops fuse_iomap_ops = { - .iomap_begin = fuse_iomap_begin, - .iomap_end = fuse_iomap_end, + .iomap_next = fuse_iomap_next, }; static void fuse_wait_dax_page(struct inode *inode) diff --git a/fs/fuse/dir.c b/fs/fuse/dir.c index 0e2a1039fa43..d4e0029810c0 100644 --- a/fs/fuse/dir.c +++ b/fs/fuse/dir.c @@ -1084,7 +1084,7 @@ static int fuse_mknod(struct mnt_idmap *idmap, struct inode *dir, } static int fuse_create(struct mnt_idmap *idmap, struct inode *dir, - struct dentry *entry, umode_t mode, bool excl) + struct dentry *entry, umode_t mode) { return fuse_mknod(idmap, dir, entry, mode, 0); } @@ -1117,6 +1117,14 @@ static struct dentry *fuse_mkdir(struct mnt_idmap *idmap, struct inode *dir, if (!fm->fc->dont_mask) mode &= ~current_umask(); + /* + * vfs_mkdir() now passes S_IFDIR in @mode, but @mode is forwarded + * verbatim to the userspace server which has only ever been given the + * permission bits. Strip the type bit until the protocol is known to + * cope with it. + */ + mode &= ~S_IFDIR; + memset(&inarg, 0, sizeof(inarg)); inarg.mode = mode; inarg.umask = current_umask(); diff --git a/fs/fuse/file.c b/fs/fuse/file.c index ceada75310b8..f2c081f09791 100644 --- a/fs/fuse/file.c +++ b/fs/fuse/file.c @@ -890,8 +890,10 @@ static int fuse_iomap_begin(struct inode *inode, loff_t offset, loff_t length, return 0; } +static DEFINE_IOMAP_ITER_NEXT(fuse_iomap_next, fuse_iomap_begin); + static const struct iomap_ops fuse_iomap_ops = { - .iomap_begin = fuse_iomap_begin, + .iomap_next = fuse_iomap_next, }; struct fuse_fill_read_data { @@ -1219,8 +1221,7 @@ static ssize_t fuse_send_write_pages(struct fuse_io_args *ia, struct file *file = iocb->ki_filp; struct fuse_file *ff = file->private_data; struct fuse_mount *fm = ff->fm; - unsigned int offset, i; - bool short_write; + unsigned int i; int err; for (i = 0; i < ap->num_folios; i++) @@ -1235,24 +1236,9 @@ static ssize_t fuse_send_write_pages(struct fuse_io_args *ia, if (!err && ia->write.out.size > count) err = -EIO; - short_write = ia->write.out.size < count; - offset = ap->descs[0].offset; - count = ia->write.out.size; for (i = 0; i < ap->num_folios; i++) { struct folio *folio = ap->folios[i]; - if (err) { - folio_clear_uptodate(folio); - } else { - if (count >= folio_size(folio) - offset) - count -= folio_size(folio) - offset; - else { - if (short_write) - folio_clear_uptodate(folio); - count = 0; - } - offset = 0; - } if (ia->write.folio_locked && (i == ap->num_folios - 1)) folio_unlock(folio); folio_put(folio); @@ -1327,7 +1313,7 @@ static ssize_t fuse_fill_write_pages(struct fuse_io_args *ia, /* If we copied full folio, mark it uptodate */ if (tmp == folio_size(folio)) - folio_mark_uptodate(folio); + iomap_folio_mark_uptodate(folio); if (folio_test_uptodate(folio)) { folio_unlock(folio); diff --git a/fs/fuse/notify.c b/fs/fuse/notify.c index 29578104ae6c..1ba763705d91 100644 --- a/fs/fuse/notify.c +++ b/fs/fuse/notify.c @@ -2,6 +2,8 @@ #include "dev.h" #include "fuse_i.h" + +#include #include static int fuse_notify_poll(struct fuse_conn *fc, unsigned int size, @@ -192,7 +194,7 @@ static int fuse_notify_store(struct fuse_conn *fc, unsigned int size, if (!folio_test_uptodate(folio) && !err && folio_offset == 0 && (nr_bytes == folio_size(folio) || file_size == end)) { folio_zero_segment(folio, nr_bytes, folio_size(folio)); - folio_mark_uptodate(folio); + iomap_folio_mark_uptodate(folio); } folio_unlock(folio); folio_put(folio); diff --git a/fs/fuse/virtio_fs.c b/fs/fuse/virtio_fs.c index df25d4faca41..f15e516ebcb5 100644 --- a/fs/fuse/virtio_fs.c +++ b/fs/fuse/virtio_fs.c @@ -1024,8 +1024,7 @@ static void virtio_fs_cleanup_vqs(struct virtio_device *vdev) } /* Map a window offset to a page frame number. The window offset will have - * been produced by .iomap_begin(), which maps a file offset to a window - * offset. + * been produced by .iomap_next(), which maps a file offset to a window offset. */ static long virtio_fs_direct_access(struct dax_device *dax_dev, pgoff_t pgoff, long nr_pages, enum dax_access_mode mode, diff --git a/fs/gfs2/bmap.c b/fs/gfs2/bmap.c index 51ac1fd44f78..73c626971163 100644 --- a/fs/gfs2/bmap.c +++ b/fs/gfs2/bmap.c @@ -1200,9 +1200,11 @@ static int gfs2_iomap_end(struct inode *inode, loff_t pos, loff_t length, return 0; } +static DEFINE_IOMAP_ITER_NEXT_END(gfs2_iomap_next, gfs2_iomap_begin, + gfs2_iomap_end); + const struct iomap_ops gfs2_iomap_ops = { - .iomap_begin = gfs2_iomap_begin, - .iomap_end = gfs2_iomap_end, + .iomap_next = gfs2_iomap_next, }; /** diff --git a/fs/gfs2/glock.c b/fs/gfs2/glock.c index 558c8c660a3d..334fee90c27e 100644 --- a/fs/gfs2/glock.c +++ b/fs/gfs2/glock.c @@ -1951,7 +1951,7 @@ static void dump_glock_func(struct gfs2_glock *gl) static void withdraw_glock(struct gfs2_glock *gl) { spin_lock(&gl->gl_lockref.lock); - if (!__lockref_is_dead(&gl->gl_lockref)) { + if (!lockref_is_dead(&gl->gl_lockref)) { /* * We don't want to write back any more dirty data. Unlock the * remaining inode and resource group glocks; this will cause @@ -2300,7 +2300,7 @@ static void gfs2_glock_iter_next(struct gfs2_glock_iter *gi, loff_t n) continue; break; } else { - if (__lockref_is_dead(&gl->gl_lockref)) + if (lockref_is_dead(&gl->gl_lockref)) continue; n--; } diff --git a/fs/gfs2/inode.c b/fs/gfs2/inode.c index 8a77794bbd4a..f361876c5583 100644 --- a/fs/gfs2/inode.c +++ b/fs/gfs2/inode.c @@ -963,15 +963,14 @@ fail: * @dir: The directory in which to create the file * @dentry: The dentry of the new file * @mode: The mode of the new file - * @excl: Force fail if inode exists * * Returns: errno */ static int gfs2_create(struct mnt_idmap *idmap, struct inode *dir, - struct dentry *dentry, umode_t mode, bool excl) + struct dentry *dentry, umode_t mode) { - return gfs2_create_inode(dir, dentry, NULL, S_IFREG | mode, 0, NULL, 0, excl); + return gfs2_create_inode(dir, dentry, NULL, S_IFREG | mode, 0, NULL, 0, 1); } /** @@ -1351,7 +1350,7 @@ static struct dentry *gfs2_mkdir(struct mnt_idmap *idmap, struct inode *dir, { unsigned dsize = gfs2_max_stuffed_size(GFS2_I(dir)); - return ERR_PTR(gfs2_create_inode(dir, dentry, NULL, S_IFDIR | mode, 0, NULL, dsize, 0)); + return ERR_PTR(gfs2_create_inode(dir, dentry, NULL, mode, 0, NULL, dsize, 0)); } /** diff --git a/fs/gfs2/lock_dlm.c b/fs/gfs2/lock_dlm.c index ab7ac8e634bf..5a5ff3b5978e 100644 --- a/fs/gfs2/lock_dlm.c +++ b/fs/gfs2/lock_dlm.c @@ -126,7 +126,7 @@ static void gdlm_ast(void *arg) clear_bit(GLF_BLOCKING, &gl->gl_flags); /* If the glock is dead, we only react to a dlm_unlock() reply. */ - if (__lockref_is_dead(&gl->gl_lockref) && + if (lockref_is_dead(&gl->gl_lockref) && gl->gl_lksb.sb_status != -DLM_EUNLOCK) return; @@ -182,7 +182,7 @@ static void gdlm_bast(void *arg, int mode) { struct gfs2_glock *gl = arg; - if (__lockref_is_dead(&gl->gl_lockref)) + if (lockref_is_dead(&gl->gl_lockref)) return; switch (mode) { @@ -329,7 +329,7 @@ static void gdlm_put_lock(struct gfs2_glock *gl) uint32_t flags = 0; int error; - BUG_ON(!__lockref_is_dead(&gl->gl_lockref)); + BUG_ON(!lockref_is_dead(&gl->gl_lockref)); if (test_bit(GLF_INITIAL, &gl->gl_flags)) { gfs2_glock_free(gl); diff --git a/fs/gfs2/quota.c b/fs/gfs2/quota.c index 91e9975d25e8..001c8b39ca55 100644 --- a/fs/gfs2/quota.c +++ b/fs/gfs2/quota.c @@ -342,7 +342,7 @@ static void qd_put(struct gfs2_quota_data *qd) if (lockref_put_or_lock(&qd->qd_lockref)) return; - BUG_ON(__lockref_is_dead(&qd->qd_lockref)); + BUG_ON(lockref_is_dead(&qd->qd_lockref)); sdp = qd->qd_sbd; if (unlikely(!test_bit(SDF_JOURNAL_LIVE, &sdp->sd_flags))) { lockref_mark_dead(&qd->qd_lockref); @@ -486,7 +486,7 @@ static bool qd_grab_sync(struct gfs2_sbd *sdp, struct gfs2_quota_data *qd, qd->qd_sync_gen >= sync_gen) goto out; - if (__lockref_is_dead(&qd->qd_lockref)) + if (lockref_is_dead(&qd->qd_lockref)) goto out; qd->qd_lockref.count++; diff --git a/fs/hfs/dir.c b/fs/hfs/dir.c index e13450bb933e..e1f1fb351464 100644 --- a/fs/hfs/dir.c +++ b/fs/hfs/dir.c @@ -184,7 +184,7 @@ static int hfs_dir_release(struct inode *inode, struct file *file) * the directory and the name (and its length) of the new file. */ static int hfs_create(struct mnt_idmap *idmap, struct inode *dir, - struct dentry *dentry, umode_t mode, bool excl) + struct dentry *dentry, umode_t mode) { struct inode *inode; int res; @@ -219,7 +219,7 @@ static struct dentry *hfs_mkdir(struct mnt_idmap *idmap, struct inode *dir, struct inode *inode; int res; - inode = hfs_new_inode(dir, &dentry->d_name, S_IFDIR | mode); + inode = hfs_new_inode(dir, &dentry->d_name, mode); if (IS_ERR(inode)) return ERR_CAST(inode); diff --git a/fs/hfs/super.c b/fs/hfs/super.c index a466c401f6bb..ecdafc658928 100644 --- a/fs/hfs/super.c +++ b/fs/hfs/super.c @@ -82,7 +82,7 @@ void hfs_mark_mdb_dirty(struct super_block *sb) spin_lock(&sbi->work_lock); if (!sbi->work_queued) { delay = msecs_to_jiffies(dirty_writeback_interval * 10); - queue_delayed_work(system_long_wq, &sbi->mdb_work, delay); + queue_delayed_work(system_dfl_long_wq, &sbi->mdb_work, delay); sbi->work_queued = 1; } spin_unlock(&sbi->work_lock); diff --git a/fs/hfsplus/dir.c b/fs/hfsplus/dir.c index 8bf6c7cdd9a8..51fcba2e6d40 100644 --- a/fs/hfsplus/dir.c +++ b/fs/hfsplus/dir.c @@ -562,7 +562,7 @@ out: } static int hfsplus_create(struct mnt_idmap *idmap, struct inode *dir, - struct dentry *dentry, umode_t mode, bool excl) + struct dentry *dentry, umode_t mode) { return hfsplus_mknod(&nop_mnt_idmap, dir, dentry, mode, 0); } @@ -570,7 +570,7 @@ static int hfsplus_create(struct mnt_idmap *idmap, struct inode *dir, static struct dentry *hfsplus_mkdir(struct mnt_idmap *idmap, struct inode *dir, struct dentry *dentry, umode_t mode) { - return ERR_PTR(hfsplus_mknod(&nop_mnt_idmap, dir, dentry, mode | S_IFDIR, 0)); + return ERR_PTR(hfsplus_mknod(&nop_mnt_idmap, dir, dentry, mode, 0)); } static int hfsplus_rename(struct mnt_idmap *idmap, diff --git a/fs/hfsplus/super.c b/fs/hfsplus/super.c index 5777e31de45a..ff7d6b3336a6 100644 --- a/fs/hfsplus/super.c +++ b/fs/hfsplus/super.c @@ -312,7 +312,7 @@ void hfsplus_mark_mdb_dirty(struct super_block *sb) spin_lock(&sbi->work_lock); if (!sbi->work_queued) { delay = msecs_to_jiffies(dirty_writeback_interval * 10); - queue_delayed_work(system_long_wq, &sbi->sync_work, delay); + queue_delayed_work(system_dfl_long_wq, &sbi->sync_work, delay); sbi->work_queued = 1; } spin_unlock(&sbi->work_lock); diff --git a/fs/hostfs/hostfs_kern.c b/fs/hostfs/hostfs_kern.c index abe86d72d9ef..7add056d47d8 100644 --- a/fs/hostfs/hostfs_kern.c +++ b/fs/hostfs/hostfs_kern.c @@ -593,7 +593,7 @@ static struct inode *hostfs_iget(struct super_block *sb, char *name) } static int hostfs_create(struct mnt_idmap *idmap, struct inode *dir, - struct dentry *dentry, umode_t mode, bool excl) + struct dentry *dentry, umode_t mode) { struct inode *inode; char *name; diff --git a/fs/hpfs/file.c b/fs/hpfs/file.c index 29e876705369..6a629ab956fc 100644 --- a/fs/hpfs/file.c +++ b/fs/hpfs/file.c @@ -156,8 +156,10 @@ static int hpfs_iomap_begin(struct inode *inode, loff_t offset, loff_t length, return 0; } +static DEFINE_IOMAP_ITER_NEXT(hpfs_iomap_next, hpfs_iomap_begin); + static const struct iomap_ops hpfs_iomap_ops = { - .iomap_begin = hpfs_iomap_begin, + .iomap_next = hpfs_iomap_next, }; static int hpfs_read_folio(struct file *file, struct folio *folio) diff --git a/fs/hpfs/namei.c b/fs/hpfs/namei.c index 353e13a615f5..9446f4038874 100644 --- a/fs/hpfs/namei.c +++ b/fs/hpfs/namei.c @@ -105,10 +105,10 @@ static struct dentry *hpfs_mkdir(struct mnt_idmap *idmap, struct inode *dir, if (!uid_eq(result->i_uid, current_fsuid()) || !gid_eq(result->i_gid, current_fsgid()) || - result->i_mode != (mode | S_IFDIR)) { + result->i_mode != mode) { result->i_uid = current_fsuid(); result->i_gid = current_fsgid(); - result->i_mode = mode | S_IFDIR; + result->i_mode = mode; hpfs_write_inode_nolock(result); } hpfs_update_directory_times(dir); @@ -129,7 +129,7 @@ bail: } static int hpfs_create(struct mnt_idmap *idmap, struct inode *dir, - struct dentry *dentry, umode_t mode, bool excl) + struct dentry *dentry, umode_t mode) { const unsigned char *name = dentry->d_name.name; unsigned len = dentry->d_name.len; diff --git a/fs/hugetlbfs/inode.c b/fs/hugetlbfs/inode.c index 216e1a0dd0b2..38e9e59f64d8 100644 --- a/fs/hugetlbfs/inode.c +++ b/fs/hugetlbfs/inode.c @@ -971,7 +971,7 @@ static struct dentry *hugetlbfs_mkdir(struct mnt_idmap *idmap, struct inode *dir struct dentry *dentry, umode_t mode) { int retval = hugetlbfs_mknod(idmap, dir, dentry, - mode | S_IFDIR, 0); + mode, 0); if (!retval) inc_nlink(dir); return ERR_PTR(retval); @@ -979,7 +979,7 @@ static struct dentry *hugetlbfs_mkdir(struct mnt_idmap *idmap, struct inode *dir static int hugetlbfs_create(struct mnt_idmap *idmap, struct inode *dir, struct dentry *dentry, - umode_t mode, bool excl) + umode_t mode) { return hugetlbfs_mknod(idmap, dir, dentry, mode | S_IFREG, 0); } diff --git a/fs/inode.c b/fs/inode.c index 31c5b9ee3a81..238fcd1cad6e 100644 --- a/fs/inode.c +++ b/fs/inode.c @@ -763,21 +763,18 @@ void clear_inode(struct inode *inode) fsverity_cleanup_inode(inode); /* - * We have to cycle the i_pages lock here because reclaim can be in the - * process of removing the last page (in __filemap_remove_folio()) - * and we must not free the mapping under it. + * We have to cycle the i_pages lock here because reclaim + * can be in the process of removing the last page (in + * __filemap_remove_folio()) and we must not free the mapping + * under it. We also remove nodes which are empty; these + * can occur in two different ways. The first is that radix + * tree expansion can fail partway and the second is that THP + * collapse_file() can allocate some temporary nodes and not + * clean them up. */ - xa_lock_irq(&inode->i_data.i_pages); + xa_destroy(&inode->i_data.i_pages); + BUG_ON(inode->i_data.nrpages); - /* - * Almost always, mapping_empty(&inode->i_data) here; but there are - * two known and long-standing ways in which nodes may get left behind - * (when deep radix-tree node allocation failed partway; or when THP - * collapse_file() failed). Until those two known cases are cleaned up, - * or a cleanup function is called here, do not BUG_ON(!mapping_empty), - * nor even WARN_ON(!mapping_empty). - */ - xa_unlock_irq(&inode->i_data.i_pages); BUG_ON(!(inode_state_read_once(inode) & I_FREEING)); BUG_ON(inode_state_read_once(inode) & I_CLEAR); BUG_ON(!list_empty(&inode->i_wb_list)); @@ -2833,8 +2830,8 @@ struct timespec64 inode_set_ctime_to_ts(struct inode *inode, struct timespec64 t { trace_inode_set_ctime_to_ts(inode, &ts); set_normalized_timespec64(&ts, ts.tv_sec, ts.tv_nsec); - inode->i_ctime_sec = ts.tv_sec; - inode->i_ctime_nsec = ts.tv_nsec; + WRITE_ONCE(inode->i_ctime_sec, ts.tv_sec); + WRITE_ONCE(inode->i_ctime_nsec, ts.tv_nsec); return ts; } EXPORT_SYMBOL(inode_set_ctime_to_ts); @@ -2908,7 +2905,7 @@ struct timespec64 inode_set_ctime_current(struct inode *inode) */ cns = smp_load_acquire(&inode->i_ctime_nsec); if (cns & I_CTIME_QUERIED) { - struct timespec64 ctime = { .tv_sec = inode->i_ctime_sec, + struct timespec64 ctime = { .tv_sec = inode_get_ctime_sec(inode), .tv_nsec = cns & ~I_CTIME_QUERIED }; if (timespec64_compare(&now, &ctime) <= 0) { @@ -2920,7 +2917,7 @@ struct timespec64 inode_set_ctime_current(struct inode *inode) mgtime_counter_inc(mg_ctime_updates); /* No need to cmpxchg if it's exactly the same */ - if (cns == now.tv_nsec && inode->i_ctime_sec == now.tv_sec) { + if (cns == now.tv_nsec && inode_get_ctime_sec(inode) == now.tv_sec) { trace_ctime_xchg_skip(inode, &now); goto out; } @@ -2929,7 +2926,7 @@ retry: /* Try to swap the nsec value into place. */ if (try_cmpxchg(&inode->i_ctime_nsec, &cur, now.tv_nsec)) { /* If swap occurred, then we're (mostly) done */ - inode->i_ctime_sec = now.tv_sec; + WRITE_ONCE(inode->i_ctime_sec, now.tv_sec); trace_ctime_ns_xchg(inode, cns, now.tv_nsec, cur); mgtime_counter_inc(mg_ctime_swaps); } else { @@ -2944,7 +2941,7 @@ retry: goto retry; } /* Otherwise, keep the existing ctime */ - now.tv_sec = inode->i_ctime_sec; + now.tv_sec = inode_get_ctime_sec(inode); now.tv_nsec = cur & ~I_CTIME_QUERIED; } out: @@ -2977,7 +2974,7 @@ struct timespec64 inode_set_ctime_deleg(struct inode *inode, struct timespec64 u /* pairs with try_cmpxchg below */ cur = smp_load_acquire(&inode->i_ctime_nsec); cur_ts.tv_nsec = cur & ~I_CTIME_QUERIED; - cur_ts.tv_sec = inode->i_ctime_sec; + cur_ts.tv_sec = inode_get_ctime_sec(inode); /* If the update is older than the existing value, skip it. */ if (timespec64_compare(&update, &cur_ts) <= 0) @@ -3003,7 +3000,7 @@ struct timespec64 inode_set_ctime_deleg(struct inode *inode, struct timespec64 u retry: old = cur; if (try_cmpxchg(&inode->i_ctime_nsec, &cur, update.tv_nsec)) { - inode->i_ctime_sec = update.tv_sec; + WRITE_ONCE(inode->i_ctime_sec, update.tv_sec); mgtime_counter_inc(mg_ctime_swaps); return update; } @@ -3019,7 +3016,7 @@ retry: goto retry; /* Otherwise, it was a new timestamp. */ - cur_ts.tv_sec = inode->i_ctime_sec; + cur_ts.tv_sec = inode_get_ctime_sec(inode); cur_ts.tv_nsec = cur & ~I_CTIME_QUERIED; return cur_ts; } diff --git a/fs/internal.h b/fs/internal.h index 355d93f92208..c658c8a5ebd5 100644 --- a/fs/internal.h +++ b/fs/internal.h @@ -137,6 +137,7 @@ extern int reconfigure_super(struct fs_context *); extern bool super_trylock_shared(struct super_block *sb); struct super_block *user_get_super(dev_t, bool excl); void put_super(struct super_block *sb); +void __init super_dev_init(void); extern bool mount_capable(struct fs_context *); /* @@ -362,3 +363,7 @@ int anon_inode_setattr(struct mnt_idmap *idmap, struct dentry *dentry, struct iattr *attr); void pidfs_get_root(struct path *path); void nsfs_get_root(struct path *path); +void failfs_get_root(struct path *path); +void __init failfs_init(void); +bool failfs_mnt(const struct vfsmount *mnt); +int failfs_current_chdir(void); diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c index 6d9a2efd4bee..0a5ebfda90f1 100644 --- a/fs/iomap/buffered-io.c +++ b/fs/iomap/buffered-io.c @@ -107,6 +107,12 @@ static void iomap_set_range_uptodate(struct folio *folio, size_t off, folio_mark_uptodate(folio); } +void iomap_folio_mark_uptodate(struct folio *folio) +{ + iomap_set_range_uptodate(folio, 0, folio_size(folio)); +} +EXPORT_SYMBOL_GPL(iomap_folio_mark_uptodate); + /* * Find the next dirty block in the folio. end_blk is inclusive. * If no dirty block is found, this will return end_blk + 1. @@ -792,7 +798,7 @@ EXPORT_SYMBOL_GPL(iomap_is_partially_uptodate); */ struct folio *iomap_get_folio(struct iomap_iter *iter, loff_t pos, size_t len) { - fgf_t fgp = FGP_WRITEBEGIN | FGP_NOFS; + fgf_t fgp = FGP_WRITEBEGIN; if (iter->flags & IOMAP_NOWAIT) fgp |= FGP_NOWAIT; @@ -1182,7 +1188,6 @@ static bool iomap_write_end(struct iomap_iter *iter, size_t len, size_t copied, static int iomap_write_iter(struct iomap_iter *iter, struct iov_iter *i, const struct iomap_write_ops *write_ops) { - ssize_t total_written = 0; int status = 0; struct address_space *mapping = iter->inode->i_mapping; size_t chunk = mapping_max_folio_size(mapping); @@ -1278,12 +1283,11 @@ retry: goto retry; } } else { - total_written += written; iomap_iter_advance(iter, written); } } while (iov_iter_count(i) && iomap_length(iter)); - return total_written ? 0 : status; + return status; } ssize_t diff --git a/fs/iomap/direct-io.c b/fs/iomap/direct-io.c index e2cd5f92babe..b3368d64e81b 100644 --- a/fs/iomap/direct-io.c +++ b/fs/iomap/direct-io.c @@ -10,6 +10,7 @@ #include #include #include +#include #include "internal.h" #include "trace.h" @@ -88,9 +89,9 @@ static inline enum fserror_type iomap_dio_err_type(const struct iomap_dio *dio) return FSERR_DIRECTIO_READ; } -static inline bool should_report_dio_fserror(const struct iomap_dio *dio) +static inline bool should_report_dio_fserror(int error) { - switch (dio->error) { + switch (error) { case 0: case -EAGAIN: case -ENOTBLK: @@ -110,7 +111,7 @@ ssize_t iomap_dio_complete(struct iomap_dio *dio) if (dops && dops->end_io) ret = dops->end_io(iocb, dio->size, ret, dio->flags); - if (should_report_dio_fserror(dio)) + if (should_report_dio_fserror(dio->error)) fserror_report_io(file_inode(iocb->ki_filp), iomap_dio_err_type(dio), offset, dio->size, dio->error, GFP_NOFS); @@ -403,6 +404,14 @@ out_put_bio: return ret; } +static inline unsigned int iomap_dio_alignment(struct inode *inode, + struct block_device *bdev, unsigned int dio_flags) +{ + if (dio_flags & IOMAP_DIO_FSBLOCK_ALIGNED) + return i_blocksize(inode); + return bdev_logical_block_size(bdev); +} + static int iomap_dio_bio_iter(struct iomap_iter *iter, struct iomap_dio *dio) { const struct iomap *iomap = &iter->iomap; @@ -421,10 +430,7 @@ static int iomap_dio_bio_iter(struct iomap_iter *iter, struct iomap_dio *dio) * File systems that write out of place and always allocate new blocks * need each bio to be block aligned as that's the unit of allocation. */ - if (dio->flags & IOMAP_DIO_FSBLOCK_ALIGNED) - alignment = fs_block_size; - else - alignment = bdev_logical_block_size(iomap->bdev); + alignment = iomap_dio_alignment(inode, iomap->bdev, dio->flags); if ((pos | length) & (alignment - 1)) return -EINVAL; @@ -907,3 +913,173 @@ iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter, return iomap_dio_complete(dio); } EXPORT_SYMBOL_GPL(iomap_dio_rw); + +struct iomap_dio_simple { + struct kiocb *iocb; + size_t size; + unsigned int dio_flags; + struct work_struct work; + /* + * Align @bio to a cacheline boundary so that, combined with the + * front_pad passed to bioset_init(), the bio sits at the start of + * a cacheline in memory returned by the (HWCACHE-aligned) bio + * slab. This keeps the hot fields block layer touches on submit + * and completion (bi_iter, bi_status, ...) within a single line. + */ + struct bio bio ____cacheline_aligned_in_smp; +}; + +static struct bio_set iomap_dio_simple_pool; + +static ssize_t iomap_dio_simple_complete(struct iomap_dio_simple *sr) +{ + struct bio *bio = &sr->bio; + struct kiocb *iocb = sr->iocb; + struct inode *inode = file_inode(iocb->ki_filp); + ssize_t ret; + + if (unlikely(bio->bi_status)) { + ret = blk_status_to_errno(bio->bi_status); + if (should_report_dio_fserror(ret)) + fserror_report_io(inode, FSERR_DIRECTIO_READ, + iocb->ki_pos, sr->size, ret, + GFP_NOFS); + } else { + ret = sr->size; + iocb->ki_pos += ret; + } + + if (sr->dio_flags & IOMAP_DIO_USER_BACKED) { + bio_check_pages_dirty(bio); + } else { + bio_release_pages(bio, false); + bio_put(bio); + } + inode_dio_end(inode); + trace_iomap_dio_complete(iocb, ret < 0 ? ret : 0, ret); + return ret; +} + +static void iomap_dio_simple_complete_work(struct work_struct *work) +{ + struct iomap_dio_simple *sr = + container_of(work, struct iomap_dio_simple, work); + struct kiocb *iocb = sr->iocb; + + WRITE_ONCE(iocb->private, NULL); + iocb->ki_complete(iocb, iomap_dio_simple_complete(sr)); +} + +static void iomap_dio_simple_end_io(struct bio *bio) +{ + struct iomap_dio_simple *sr = + container_of(bio, struct iomap_dio_simple, bio); + struct kiocb *iocb = sr->iocb; + + if (unlikely(sr->bio.bi_status)) { + struct inode *inode = file_inode(iocb->ki_filp); + + INIT_WORK(&sr->work, iomap_dio_simple_complete_work); + queue_work(inode->i_sb->s_dio_done_wq, &sr->work); + return; + } + + WRITE_ONCE(iocb->private, NULL); + iocb->ki_complete(iocb, iomap_dio_simple_complete(sr)); +} + +ssize_t __iomap_dio_read_simple(struct kiocb *iocb, struct iov_iter *iter, + struct iomap_iter *iomi) +{ + gfp_t gfp = (iomi->flags & IOMAP_NOWAIT) ? GFP_NOWAIT : GFP_KERNEL; + struct iomap_dio_simple *sr; + unsigned int alignment; + struct bio *bio; + ssize_t ret; + + if (iomi->iomap.type != IOMAP_MAPPED || + iomi->iomap.offset + iomi->iomap.length < iomi->pos + iomi->len || + (iomi->iomap.flags & IOMAP_F_INTEGRITY)) { + ret = -ENOTBLK; + goto out_dio_end; + } + + alignment = iomap_dio_alignment(iomi->inode, iomi->iomap.bdev, 0); + if ((iomi->pos | iomi->len) & (alignment - 1)) { + ret = -EINVAL; + goto out_dio_end; + } + + if (unlikely(!iomi->inode->i_sb->s_dio_done_wq && + !is_sync_kiocb(iocb))) { + ret = sb_init_dio_done_wq(iomi->inode->i_sb); + if (ret < 0) + goto out_dio_end; + } + + trace_iomap_dio_rw_begin(iocb, iter, 0, 0); + + bio = bio_alloc_bioset(iomi->iomap.bdev, + bio_iov_vecs_to_alloc(iter, BIO_MAX_VECS), + REQ_OP_READ, gfp, &iomap_dio_simple_pool); + if (!bio) { + ret = -EAGAIN; + goto out_dio_end; + } + sr = container_of(bio, struct iomap_dio_simple, bio); + sr->iocb = iocb; + sr->dio_flags = 0; + + bio->bi_iter.bi_sector = iomap_sector(&iomi->iomap, iomi->pos); + bio->bi_ioprio = iocb->ki_ioprio; + + ret = bio_iov_iter_get_pages(bio, iter, alignment - 1); + if (unlikely(ret)) + goto out_bio_put; + + if (bio->bi_iter.bi_size != iomi->len) { + iov_iter_revert(iter, bio->bi_iter.bi_size); + ret = -ENOTBLK; + goto out_bio_release_pages; + } + + sr->size = bio->bi_iter.bi_size; + if (user_backed_iter(iter)) { + bio_set_pages_dirty(bio); + sr->dio_flags |= IOMAP_DIO_USER_BACKED; + } + + if (iocb->ki_flags & IOCB_NOWAIT) + bio->bi_opf |= REQ_NOWAIT; + + if (is_sync_kiocb(iocb)) { + submit_bio_wait(bio); + return iomap_dio_simple_complete(sr); + } + + if ((iocb->ki_flags & IOCB_HIPRI)) { + bio->bi_opf |= REQ_POLLED; + WRITE_ONCE(iocb->private, bio); + } + bio->bi_end_io = iomap_dio_simple_end_io; + submit_bio(bio); + trace_iomap_dio_rw_queued(iomi->inode, iocb->ki_pos, iomi->len); + return -EIOCBQUEUED; + +out_bio_release_pages: + bio_release_pages(bio, false); +out_bio_put: + bio_put(bio); +out_dio_end: + inode_dio_end(iomi->inode); + return ret; +} +EXPORT_SYMBOL_GPL(__iomap_dio_read_simple); + +static int __init iomap_dio_init(void) +{ + return bioset_init(&iomap_dio_simple_pool, 4, + offsetof(struct iomap_dio_simple, bio), + BIOSET_NEED_BVECS | BIOSET_PERCPU_CACHE); +} +fs_initcall(iomap_dio_init); diff --git a/fs/iomap/iter.c b/fs/iomap/iter.c index e4a29829591a..c445a38b6285 100644 --- a/fs/iomap/iter.c +++ b/fs/iomap/iter.c @@ -6,12 +6,19 @@ #include #include "trace.h" -static inline void iomap_iter_clean_fbatch(struct iomap_iter *iter) +/* + * Release the iter folio batch. Note that the iomap flag is meant to control + * the I/O path for the mapping and may not be set in error situations. + */ +static inline void iomap_iter_clean_fbatch(const struct iomap_iter *iter, + struct iomap *iomap) { - if (iter->iomap.flags & IOMAP_F_FOLIO_BATCH) { + if (!iter->fbatch) + return; + iomap->flags &= ~IOMAP_F_FOLIO_BATCH; + if (folio_batch_count(iter->fbatch)) { folio_batch_release(iter->fbatch); folio_batch_reinit(iter->fbatch); - iter->iomap.flags &= ~IOMAP_F_FOLIO_BATCH; } } @@ -40,9 +47,60 @@ static inline void iomap_iter_done(struct iomap_iter *iter) } /** - * iomap_iter - iterate over a ranges in a file - * @iter: iteration structue - * @ops: iomap ops provided by the file system + * iomap_iter_continue - decide whether iteration should continue + * @iter: iteration structure + * @iomap: the mapping that was just processed + * @srcmap: the source mapping that was just processed + * + * Helper normally called via iomap_iter_next(). Called after the previous + * mapping has been finished to determine whether there is more of the file + * range left to process. + * + * Returns 1 if there is more work to do, in which case @iomap and @srcmap are + * cleared so the caller can produce the next mapping; zero if the range is + * fully consumed; or a negative errno on error. + */ +int iomap_iter_continue(const struct iomap_iter *iter, struct iomap *iomap, + struct iomap *srcmap, int ret) +{ + const bool stale = iomap->flags & IOMAP_F_STALE; + const ssize_t advanced = iter->pos - iter->iter_start_pos; + + if (ret < 0 && !advanced) + return ret; + + /* + * Use iter->len to determine whether to continue onto the next mapping. + * Explicitly terminate on error status or if the current iter has not + * advanced at all (i.e. no work was done for some reason) unless the + * mapping has been marked stale and needs to be reprocessed. + */ + if (WARN_ON_ONCE(iter->status > 0)) + /* detect old return semantics where this would advance */ + ret = -EIO; + else if (iter->status < 0) + ret = iter->status; + else if (iter->len == 0 || (!advanced && !stale)) + ret = 0; + else + ret = 1; + + iomap_iter_clean_fbatch(iter, iomap); + + if (ret <= 0) + return ret; + + memset(iomap, 0, sizeof(*iomap)); + memset(srcmap, 0, sizeof(*srcmap)); + + return ret; +} +EXPORT_SYMBOL_GPL(iomap_iter_continue); + +/** + * iomap_iter - iterate over ranges in a file + * @iter: iteration structure + * @ops: iomap ops provided by the filesystem * * Iterate over filesystem-provided space mappings for the provided file range. * @@ -56,61 +114,21 @@ static inline void iomap_iter_done(struct iomap_iter *iter) */ int iomap_iter(struct iomap_iter *iter, const struct iomap_ops *ops) { - bool stale = iter->iomap.flags & IOMAP_F_STALE; - ssize_t advanced; - u64 olen; int ret; trace_iomap_iter(iter, ops, _RET_IP_); - if (!iter->iomap.length) - goto begin; - - /* - * Calculate how far the iter was advanced and the original length bytes - * for ->iomap_end(). - */ - advanced = iter->pos - iter->iter_start_pos; - olen = iter->len + advanced; - - if (ops->iomap_end) { - ret = ops->iomap_end(iter->inode, iter->iter_start_pos, - iomap_length_trim(iter, iter->iter_start_pos, - olen), - advanced, iter->flags, &iter->iomap); - if (ret < 0 && !advanced) - return ret; - } - - /* detect old return semantics where this would advance */ - if (WARN_ON_ONCE(iter->status > 0)) - iter->status = -EIO; - - /* - * Use iter->len to determine whether to continue onto the next mapping. - * Explicitly terminate on error status or if the current iter has not - * advanced at all (i.e. no work was done for some reason) unless the - * mapping has been marked stale and needs to be reprocessed. - */ - if (iter->status < 0) - ret = iter->status; - else if (iter->len == 0 || (!advanced && !stale)) - ret = 0; + if (ops->iomap_next) + ret = ops->iomap_next(iter, &iter->iomap, &iter->srcmap); else - ret = 1; - iomap_iter_clean_fbatch(iter); + ret = iomap_iter_next(iter, &iter->iomap, &iter->srcmap, + ops->iomap_begin, ops->iomap_end); + iter->status = 0; - if (ret <= 0) - return ret; + if (ret > 0) + iomap_iter_done(iter); + else if (ret < 0) + iomap_iter_clean_fbatch(iter, &iter->iomap); - memset(&iter->iomap, 0, sizeof(iter->iomap)); - memset(&iter->srcmap, 0, sizeof(iter->srcmap)); - -begin: - ret = ops->iomap_begin(iter->inode, iter->pos, iter->len, iter->flags, - &iter->iomap, &iter->srcmap); - if (ret < 0) - return ret; - iomap_iter_done(iter); - return 1; + return ret; } diff --git a/fs/jffs2/dir.c b/fs/jffs2/dir.c index c4088c3b4ac0..656c920864c5 100644 --- a/fs/jffs2/dir.c +++ b/fs/jffs2/dir.c @@ -26,7 +26,7 @@ static int jffs2_readdir (struct file *, struct dir_context *); static int jffs2_create (struct mnt_idmap *, struct inode *, - struct dentry *, umode_t, bool); + struct dentry *, umode_t); static struct dentry *jffs2_lookup (struct inode *,struct dentry *, unsigned int); static int jffs2_link (struct dentry *,struct inode *,struct dentry *); @@ -163,7 +163,7 @@ static int jffs2_readdir(struct file *file, struct dir_context *ctx) static int jffs2_create(struct mnt_idmap *idmap, struct inode *dir_i, - struct dentry *dentry, umode_t mode, bool excl) + struct dentry *dentry, umode_t mode) { struct jffs2_raw_inode *ri; struct jffs2_inode_info *f, *dir_f; @@ -462,8 +462,6 @@ static struct dentry *jffs2_mkdir (struct mnt_idmap *idmap, struct inode *dir_i, uint32_t alloclen; int ret; - mode |= S_IFDIR; - ri = jffs2_alloc_raw_inode(); if (!ri) return ERR_PTR(-ENOMEM); diff --git a/fs/jffs2/wbuf.c b/fs/jffs2/wbuf.c index 8ff7a0b6add2..3b7803c75d58 100644 --- a/fs/jffs2/wbuf.c +++ b/fs/jffs2/wbuf.c @@ -1177,7 +1177,7 @@ void jffs2_dirty_trigger(struct jffs2_sb_info *c) return; delay = msecs_to_jiffies(dirty_writeback_interval * 10); - if (queue_delayed_work(system_long_wq, &c->wbuf_dwork, delay)) + if (queue_delayed_work(system_dfl_long_wq, &c->wbuf_dwork, delay)) jffs2_dbg(1, "%s()\n", __func__); } diff --git a/fs/jfs/namei.c b/fs/jfs/namei.c index 442d62679262..8a36c218f0f7 100644 --- a/fs/jfs/namei.c +++ b/fs/jfs/namei.c @@ -61,7 +61,7 @@ static inline void free_ea_wmap(struct inode *inode) * */ static int jfs_create(struct mnt_idmap *idmap, struct inode *dip, - struct dentry *dentry, umode_t mode, bool excl) + struct dentry *dentry, umode_t mode) { int rc = 0; tid_t tid; /* transaction id */ @@ -223,7 +223,7 @@ static struct dentry *jfs_mkdir(struct mnt_idmap *idmap, struct inode *dip, * block there while holding dtree page, so we allocate the inode & * begin the transaction before we search the directory. */ - ip = ialloc(dip, S_IFDIR | mode); + ip = ialloc(dip, mode); if (IS_ERR(ip)) { rc = PTR_ERR(ip); goto out2; diff --git a/fs/kernel_read_file.c b/fs/kernel_read_file.c index de32c95d823d..9c2ba9240083 100644 --- a/fs/kernel_read_file.c +++ b/fs/kernel_read_file.c @@ -150,18 +150,13 @@ ssize_t kernel_read_file_from_path_initns(const char *path, loff_t offset, enum kernel_read_file_id id) { struct file *file; - struct path root; ssize_t ret; if (!path || !*path) return -EINVAL; - task_lock(&init_task); - get_fs_root(init_task.fs, &root); - task_unlock(&init_task); - - file = file_open_root(&root, path, O_RDONLY, 0); - path_put(&root); + scoped_with_init_fs() + file = filp_open(path, O_RDONLY, 0); if (IS_ERR(file)) return PTR_ERR(file); diff --git a/fs/libfs.c b/fs/libfs.c index 5a0d276379d1..27d7dc16fcb0 100644 --- a/fs/libfs.c +++ b/fs/libfs.c @@ -1559,9 +1559,12 @@ int simple_fsync_noflush(struct file *file, loff_t start, loff_t end, if (err) return err; - if (!(inode_state_read_once(inode) & I_DIRTY_ALL)) + if (!(inode_state_read_once(inode) & + (I_DIRTY_ALL | I_SYNC | I_METADATA_WRITEBACK))) goto out; - if (datasync && !(inode_state_read_once(inode) & I_DIRTY_DATASYNC)) + if (datasync && + !(inode_state_read_once(inode) & + (I_DIRTY_DATASYNC | I_SYNC | I_METADATA_WRITEBACK))) goto out; ret = sync_inode_metadata(inode, 1); diff --git a/fs/minix/dir.c b/fs/minix/dir.c index 361d26d87d2e..2ca16f849d5a 100644 --- a/fs/minix/dir.c +++ b/fs/minix/dir.c @@ -23,7 +23,7 @@ const struct file_operations minix_dir_operations = { .llseek = generic_file_llseek, .read = generic_read_dir, .iterate_shared = minix_readdir, - .fsync = minix_fsync, + .fsync = simple_fsync, }; /* diff --git a/fs/minix/file.c b/fs/minix/file.c index 86e5943cd2ff..02aabbdb5dea 100644 --- a/fs/minix/file.c +++ b/fs/minix/file.c @@ -10,13 +10,6 @@ #include #include "minix.h" -int minix_fsync(struct file *file, loff_t start, loff_t end, int datasync) -{ - return mmb_fsync(file, - &minix_i(file->f_mapping->host)->i_metadata_bhs, - start, end, datasync); -} - /* * We have mostly NULLs here: the current defaults are OK for * the minix filesystem. @@ -26,7 +19,7 @@ const struct file_operations minix_file_operations = { .read_iter = generic_file_read_iter, .write_iter = generic_file_write_iter, .mmap_prepare = generic_file_mmap_prepare, - .fsync = minix_fsync, + .fsync = simple_fsync, .splice_read = filemap_splice_read, }; diff --git a/fs/minix/inode.c b/fs/minix/inode.c index c30cc590698d..daf83e4ff25c 100644 --- a/fs/minix/inode.c +++ b/fs/minix/inode.c @@ -24,6 +24,8 @@ static int minix_write_inode(struct inode *inode, struct writeback_control *wbc); +static int minix_sync_inode_metadata(struct inode *inode, + struct writeback_control *wbc); static int minix_statfs(struct dentry *dentry, struct kstatfs *buf); void __minix_error_inode(struct inode *inode, const char *function, @@ -128,6 +130,7 @@ static const struct super_operations minix_sops = { .alloc_inode = minix_alloc_inode, .free_inode = minix_free_in_core_inode, .write_inode = minix_write_inode, + .sync_inode_metadata = minix_sync_inode_metadata, .evict_inode = minix_evict_inode, .put_super = minix_put_super, .statfs = minix_statfs, @@ -630,7 +633,7 @@ struct inode *minix_iget(struct super_block *sb, unsigned long ino) /* * The minix V1 function to synchronize an inode. */ -static struct buffer_head * V1_minix_update_inode(struct inode * inode) +static int V1_minix_update_inode(struct inode * inode) { struct buffer_head * bh; struct minix_inode * raw_inode; @@ -639,7 +642,7 @@ static struct buffer_head * V1_minix_update_inode(struct inode * inode) raw_inode = minix_V1_raw_inode(inode->i_sb, inode->i_ino, &bh); if (!raw_inode) - return NULL; + return -EIO; raw_inode->i_mode = inode->i_mode; raw_inode->i_uid = fs_high2lowuid(i_uid_read(inode)); raw_inode->i_gid = fs_high2lowgid(i_gid_read(inode)); @@ -651,13 +654,15 @@ static struct buffer_head * V1_minix_update_inode(struct inode * inode) else for (i = 0; i < 9; i++) raw_inode->i_zone[i] = minix_inode->u.i1_data[i]; mark_buffer_dirty(bh); - return bh; + brelse(bh); + set_inode_metadata_writeback(inode); + return 0; } /* * The minix V2 function to synchronize an inode. */ -static struct buffer_head * V2_minix_update_inode(struct inode * inode) +static int V2_minix_update_inode(struct inode * inode) { struct buffer_head * bh; struct minix2_inode * raw_inode; @@ -666,7 +671,7 @@ static struct buffer_head * V2_minix_update_inode(struct inode * inode) raw_inode = minix_V2_raw_inode(inode->i_sb, inode->i_ino, &bh); if (!raw_inode) - return NULL; + return -EIO; raw_inode->i_mode = inode->i_mode; raw_inode->i_uid = fs_high2lowuid(i_uid_read(inode)); raw_inode->i_gid = fs_high2lowgid(i_gid_read(inode)); @@ -680,29 +685,42 @@ static struct buffer_head * V2_minix_update_inode(struct inode * inode) else for (i = 0; i < 10; i++) raw_inode->i_zone[i] = minix_inode->u.i2_data[i]; mark_buffer_dirty(bh); - return bh; + brelse(bh); + set_inode_metadata_writeback(inode); + return 0; } static int minix_write_inode(struct inode *inode, struct writeback_control *wbc) +{ + if (INODE_VERSION(inode) == MINIX_V1) + return V1_minix_update_inode(inode); + return V2_minix_update_inode(inode); +} + +static int minix_sync_inode_metadata(struct inode *inode, + struct writeback_control *wbc) { int err = 0; struct buffer_head *bh; + void *raw_inode; if (INODE_VERSION(inode) == MINIX_V1) - bh = V1_minix_update_inode(inode); + raw_inode = minix_V1_raw_inode(inode->i_sb, inode->i_ino, &bh); else - bh = V2_minix_update_inode(inode); - if (!bh) + raw_inode = minix_V2_raw_inode(inode->i_sb, inode->i_ino, &bh); + if (!raw_inode) return -EIO; - if (wbc->sync_mode == WB_SYNC_ALL && buffer_dirty(bh)) { - sync_dirty_buffer(bh); - if (buffer_req(bh) && !buffer_uptodate(bh)) { - printk("IO error syncing minix inode [%s:%08llx]\n", - inode->i_sb->s_id, inode->i_ino); - err = -EIO; - } + err = mmb_sync(&minix_i(inode)->i_metadata_bhs); + if (err) + goto out; + sync_dirty_buffer(bh); + if (buffer_write_io_error(bh)) { + printk("IO error syncing minix inode [%s:%08llx]\n", + inode->i_sb->s_id, inode->i_ino); + err = -EIO; } - brelse (bh); +out: + brelse(bh); return err; } diff --git a/fs/minix/minix.h b/fs/minix/minix.h index 9e52d4302f0d..78722ce22e1e 100644 --- a/fs/minix/minix.h +++ b/fs/minix/minix.h @@ -59,7 +59,6 @@ int minix_getattr(struct mnt_idmap *, const struct path *, struct kstat *, u32, unsigned int); int minix_prepare_chunk(struct folio *folio, loff_t pos, unsigned len); struct mapping_metadata_bhs *minix_get_metadata_bhs(struct inode *inode); -int minix_fsync(struct file *file, loff_t start, loff_t end, int datasync); extern void V1_minix_truncate(struct inode *); extern void V2_minix_truncate(struct inode *); diff --git a/fs/minix/namei.c b/fs/minix/namei.c index 263e4ba8b1c8..5525ba367ed7 100644 --- a/fs/minix/namei.c +++ b/fs/minix/namei.c @@ -64,7 +64,7 @@ static int minix_tmpfile(struct mnt_idmap *idmap, struct inode *dir, } static int minix_create(struct mnt_idmap *idmap, struct inode *dir, - struct dentry *dentry, umode_t mode, bool excl) + struct dentry *dentry, umode_t mode) { return minix_mknod(&nop_mnt_idmap, dir, dentry, mode, 0); } @@ -110,7 +110,7 @@ static struct dentry *minix_mkdir(struct mnt_idmap *idmap, struct inode *dir, struct inode * inode; int err; - inode = minix_new_inode(dir, S_IFDIR | mode); + inode = minix_new_inode(dir, mode); if (IS_ERR(inode)) return ERR_CAST(inode); diff --git a/fs/namei.c b/fs/namei.c index 19ce43c9a6e6..3f9bf103ba12 100644 --- a/fs/namei.c +++ b/fs/namei.c @@ -4140,11 +4140,6 @@ EXPORT_SYMBOL(end_renaming); * after setgid stripping allows the same ordering for both non-POSIX ACL and * POSIX ACL supporting filesystems. * - * Note that it's currently valid for @type to be 0 if a directory is created. - * Filesystems raise that flag individually and we need to check whether each - * filesystem can deal with receiving S_IFDIR from the vfs before we enforce a - * non-zero type. - * * Returns: mode to be passed to the filesystem */ static inline umode_t vfs_prepare_mode(struct mnt_idmap *idmap, @@ -4190,7 +4185,7 @@ int vfs_create(struct mnt_idmap *idmap, struct dentry *dentry, umode_t mode, return error; if (!dir->i_op->create) - return -EACCES; /* shouldn't it be ENOSYS? */ + return -EOPNOTSUPP; mode = vfs_prepare_mode(idmap, dir, mode, S_IALLUGO, S_IFREG); error = security_inode_create(dir, dentry, mode); @@ -4199,7 +4194,7 @@ int vfs_create(struct mnt_idmap *idmap, struct dentry *dentry, umode_t mode, error = try_break_deleg(dir, LEASE_BREAK_DIR_CREATE, di); if (error) return error; - error = dir->i_op->create(idmap, dir, dentry, mode, true); + error = dir->i_op->create(idmap, dir, dentry, mode); if (!error) fsnotify_create(dir, dentry); return error; @@ -4336,50 +4331,83 @@ static int may_o_create(struct mnt_idmap *idmap, return security_inode_create(dir->dentry->d_inode, dentry, mode); } -/* - * Attempt to atomically look up, create and open a file from a negative - * dentry. +/** + * atomic_open() - atomically look up, create and open a file + * @path: parent directory path + * @dentry: child to ->atomic_open() + * @file: file to attach child to + * @open_flag: open flags + * @mode: create mode + * @create_error: return value from may_o_create() * - * Returns 0 if successful. The file will have been created and attached to - * @file by the filesystem calling finish_open(). + * Attempt to look up, create and open @dentry, which must be negative, in a + * single call into the filesystem. * - * If the file was looked up only or didn't need creating, FMODE_OPENED won't - * be set. The caller will need to perform the open themselves. @path will - * have been updated to point to the new dentry. This may be negative. + * If a non-error dentry is returned then: when FMODE_OPENED is set, + * the file will have been attached to @file by the filesystem calling + * finish_open(). If FMODE_OPENED isn't set, the filesystem instead called + * finish_no_open() and the caller will need to perform the open themselves. * - * Returns an error code otherwise. + * FMODE_CREATED is set when the call to ->atomic_open() actually created + * the file. + * + * Returns: the opened or looked-up dentry, or ERR_PTR() on failure. The + * reference to @dentry is consumed in either case. */ static struct dentry *atomic_open(const struct path *path, struct dentry *dentry, struct file *file, - int open_flag, umode_t mode) + int open_flag, umode_t mode, int create_error) { struct dentry *const DENTRY_NOT_SET = (void *) -1UL; - struct inode *dir = path->dentry->d_inode; + struct inode *dir_inode = path->dentry->d_inode; int error; file->__f_path.dentry = DENTRY_NOT_SET; file->__f_path.mnt = path->mnt; - error = dir->i_op->atomic_open(dir, dentry, file, + error = dir_inode->i_op->atomic_open(dir_inode, dentry, file, open_to_namei_flags(open_flag), mode); d_lookup_done(dentry); + if (!error) { if (file->f_mode & FMODE_OPENED) { - if (unlikely(dentry != file->f_path.dentry)) { + /* finish_open() called */ + struct dentry *opened = file->f_path.dentry; + + if (unlikely(opened != dentry)) { dput(dentry); - dentry = dget(file->f_path.dentry); + dentry = dget(opened); } - } else if (WARN_ON(file->f_path.dentry == DENTRY_NOT_SET)) { - error = -EIO; - } else { - if (file->f_path.dentry) { + } else if (likely(file->f_path.dentry != DENTRY_NOT_SET)) { + /* finish_no_open() called */ + struct dentry *replaced = file->f_path.dentry; + + if (replaced) { dput(dentry); - dentry = file->f_path.dentry; + dentry = replaced; } if (unlikely(d_is_negative(dentry))) error = -ENOENT; + } else { + const char *fsname = dentry->d_sb->s_type->name; + + WARN(1, "%s: ->atomic_open() left file->f_path.dentry unset!\n", + fsname); + error = -EIO; } } + if (error) { + if (unlikely(create_error) && error == -ENOENT) { + /* + * Should have done a create, but errored before. + * Some filesystems return -ENOENT directly instead of + * calling finish_no_open() with a negative dentry; + * either way it should only mean the child doesn't exist, + * so a refused create is safe to record here. + */ + audit_inode_child(dir_inode, dentry, AUDIT_TYPE_CHILD_CREATE); + error = create_error; + } dput(dentry); dentry = ERR_PTR(error); } @@ -4389,32 +4417,52 @@ static struct dentry *atomic_open(const struct path *path, struct dentry *dentry /* * Look up and maybe create and open the last component. * - * Must be called with parent locked (exclusive in O_CREAT case). + * Takes the parent inode lock itself, exclusive if O_CREAT was requested and + * shared otherwise, and drops it again before returning. The caller must not + * hold it. * - * Returns 0 on success, that is, if - * the file was successfully atomically created (if necessary) and opened, or - * the file was not completely opened at this time, though lookups and - * creations were performed. - * These case are distinguished by presence of FMODE_OPENED on file->f_mode. - * In the latter case dentry returned in @path might be negative if O_CREAT - * hadn't been specified. + * On success returns the dentry of the last component. If FMODE_OPENED is set + * on file->f_mode the file was also opened and attached to @file; otherwise + * only lookup and creation were performed and the caller has to open it. In + * the latter case the dentry may be negative if O_CREAT hadn't been specified. * - * An error code is returned on failure. + * Returns ERR_PTR() on failure. */ static struct dentry *lookup_open(struct nameidata *nd, struct file *file, - const struct open_flags *op, - bool got_write, struct delegated_inode *delegated_inode) + const struct open_flags *op) { + struct delegated_inode delegated_inode = { }; struct mnt_idmap *idmap; struct dentry *dir = nd->path.dentry; struct inode *dir_inode = dir->d_inode; - int open_flag = op->open_flag; + int open_flag; struct dentry *dentry; - int error, create_error = 0; - umode_t mode = op->mode; + int error, create_error; + umode_t mode; + bool got_write; - if (unlikely(IS_DEADDIR(dir_inode))) - return ERR_PTR(-ENOENT); +retry: + open_flag = op->open_flag; + got_write = false; + mode = op->mode; + create_error = 0; + + if (open_flag & (O_CREAT | O_TRUNC | O_WRONLY | O_RDWR)) { + got_write = !mnt_want_write(nd->path.mnt); + /* + * do _not_ fail yet - we might not need that or fail with + * a different error; we'll be dropping this one anyway. + */ + } + if (open_flag & O_CREAT) + inode_lock(dir_inode); + else + inode_lock_shared(dir_inode); + + if (unlikely(IS_DEADDIR(dir_inode))) { + dentry = ERR_PTR(-ENOENT); + goto out; + } file->f_mode &= ~FMODE_CREATED; dentry = d_lookup(dir, &nd->last); @@ -4422,7 +4470,7 @@ static struct dentry *lookup_open(struct nameidata *nd, struct file *file, if (!dentry) { dentry = d_alloc_parallel(dir, &nd->last); if (IS_ERR(dentry)) - return dentry; + goto out; } if (d_in_lookup(dentry)) break; @@ -4437,8 +4485,8 @@ static struct dentry *lookup_open(struct nameidata *nd, struct file *file, dentry = NULL; } if (dentry->d_inode) { - /* Cached positive dentry: will open in f_op->open */ - return dentry; + /* Cached positive dentry: will open in do_open(). */ + goto out; } if (open_flag & O_CREAT) @@ -4459,7 +4507,7 @@ static struct dentry *lookup_open(struct nameidata *nd, struct file *file, if (open_flag & O_CREAT) { if (open_flag & O_EXCL) open_flag &= ~O_TRUNC; - mode = vfs_prepare_mode(idmap, dir->d_inode, mode, mode, mode); + mode = vfs_prepare_mode(idmap, dir_inode, mode, mode, mode); if (likely(got_write)) create_error = may_o_create(idmap, &nd->path, dentry, mode); @@ -4471,10 +4519,9 @@ static struct dentry *lookup_open(struct nameidata *nd, struct file *file, if (dir_inode->i_op->atomic_open) { if (nd->flags & LOOKUP_DIRECTORY) open_flag |= O_DIRECTORY; - dentry = atomic_open(&nd->path, dentry, file, open_flag, mode); - if (unlikely(create_error) && dentry == ERR_PTR(-ENOENT)) - dentry = ERR_PTR(create_error); - return dentry; + dentry = atomic_open(&nd->path, dentry, file, open_flag, mode, + create_error); + goto out; } if (d_in_lookup(dentry)) { @@ -4490,37 +4537,164 @@ static struct dentry *lookup_open(struct nameidata *nd, struct file *file, dentry = res; } } - - /* Negative dentry, just create the file */ - if (!dentry->d_inode && (open_flag & O_CREAT)) { - /* but break the directory lease first! */ - error = try_break_deleg(dir_inode, LEASE_BREAK_DIR_CREATE, delegated_inode); - if (error) - goto out_dput; - - file->f_mode |= FMODE_CREATED; - audit_inode_child(dir_inode, dentry, AUDIT_TYPE_CHILD_CREATE); - if (!dir_inode->i_op->create) { - error = -EACCES; - goto out_dput; - } - - error = dir_inode->i_op->create(idmap, dir_inode, dentry, - mode, open_flag & O_EXCL); - if (error) - goto out_dput; + if (dentry->d_inode || !(op->open_flag & O_CREAT)) { + /* + * No need to create a file. If lookup returned a positive + * dentry, the file will be opened in do_open(). + */ + goto out; } - if (unlikely(create_error) && !dentry->d_inode) { + + /* Negative dentry with O_CREAT flag set */ + audit_inode_child(dir_inode, dentry, AUDIT_TYPE_CHILD_CREATE); + + if (unlikely(create_error)) { + /* should have done a create, but we already errored */ error = create_error; goto out_dput; } + + error = try_break_deleg(dir_inode, LEASE_BREAK_DIR_CREATE, &delegated_inode); + if (error) + goto out_dput; + + file->f_mode |= FMODE_CREATED; + if (!dir_inode->i_op->create) { + error = -EOPNOTSUPP; + goto out_dput; + } + + error = dir_inode->i_op->create(idmap, dir_inode, dentry, mode); + if (error) + goto out_dput; +out: + if (!IS_ERR(dentry)) { + if (file->f_mode & FMODE_CREATED) + fsnotify_create(dir_inode, dentry); + if (file->f_mode & FMODE_OPENED) + fsnotify_open(file); + } + if ((open_flag & O_CREAT) || create_error) + inode_unlock(dir_inode); + else + inode_unlock_shared(dir_inode); + + if (got_write) + mnt_drop_write(nd->path.mnt); + + if (is_delegated(&delegated_inode)) { + /* Must have come through out_dput: dentry is an ERR_PTR() */ + error = break_deleg_wait(&delegated_inode); + + if (!error) + goto retry; + dentry = ERR_PTR(error); + } + return dentry; out_dput: dput(dentry); - return ERR_PTR(error); + dentry = ERR_PTR(error); + goto out; } +/** + * vfs_lookup_open - open and possibly create a regular file + * @parent: directory to contain file + * @last: final component of file name + * @open_flag: O_flags + * @mode: initial permissions for file + * + * Open a file after lookup and/or create. This provides similar + * functionality to open_last_lookups() for non-VFS users, particularly + * nfsd. + * It uses ->atomic_open or ->lookup / ->create / ->open as appropriate. + * + * If the fs object found is not a regular file then an error is returned. + * In some cases, related errors are repurposed so that the caller can + * determine the type of file found from the error. + * -EISDIR : a directory was found + * -ELOOP : a symlink was found + * -ENODEV : a block or character device special file was found + * -EFTYPE : any other non-regular file was found, such as FIFO or SOCK. + * or ->atomic_open responded to __O_REGULAR. + * + * Returns: the opened struct file, or an error. + */ +struct file *vfs_lookup_open(struct path *parent, struct qstr *last, + int open_flag, umode_t mode) +{ + struct file *file __free(fput) = NULL; + struct nameidata nd = {}; + struct open_flags op = {}; + struct dentry *dentry; + int error = 0; + + WARN_ONCE(mode & ~S_IALLUGO, "mode must only have permission bits"); + WARN_ONCE(open_flag & ~(O_ACCMODE|O_CREAT|O_EXCL|O_TRUNC|__O_REGULAR), + "open_flag has unsupported flags"); + + mode |= S_IFREG; + open_flag |= __O_REGULAR; + + error = lookup_noperm_common(last, parent->dentry); + if (error) + return ERR_PTR(error); + + file = alloc_empty_file(open_flag, current_cred()); + if (IS_ERR(file)) + return file; + + nd.path = *parent; + nd.last = *last; + nd.flags = LOOKUP_OPEN; + if (open_flag & O_CREAT) { + nd.flags |= LOOKUP_CREATE; + if (open_flag & O_EXCL) + nd.flags |= LOOKUP_EXCL; + } + op.open_flag = open_flag; + op.mode = mode; + dentry = lookup_open(&nd, file, &op); + + if (IS_ERR(dentry)) + return ERR_CAST(dentry); + + if (d_really_is_negative(dentry)) { + error = -ENOENT; + } else if (!(file->f_mode & FMODE_CREATED) && (open_flag & O_EXCL)) { + error = -EEXIST; + } else if ((dentry->d_inode->i_mode & S_IFMT) != S_IFREG) { + switch (dentry->d_inode->i_mode & S_IFMT) { + case S_IFDIR: + error = -EISDIR; + break; + case S_IFLNK: + error = -ELOOP; + break; + case S_IFBLK: + case S_IFCHR: + error = -ENODEV; + break; + case S_IFIFO: + case S_IFSOCK: + default: + error = -EFTYPE; + break; + } + } else if (!(file->f_mode & FMODE_OPENED)) { + nd.path.dentry = dentry; + error = vfs_open(&nd.path, file); + } + dput(dentry); + + if (error) + return ERR_PTR(error); + return no_free_ptr(file); +} +EXPORT_SYMBOL_FOR_MODULES(vfs_lookup_open, "nfsd"); + static inline bool trailing_slashes(struct nameidata *nd) { return (bool)nd->last.name[nd->last.len]; @@ -4560,10 +4734,7 @@ static struct dentry *lookup_fast_for_open(struct nameidata *nd, int open_flag) static const char *open_last_lookups(struct nameidata *nd, struct file *file, const struct open_flags *op) { - struct delegated_inode delegated_inode = { }; - struct dentry *dir = nd->path.dentry; int open_flag = op->open_flag; - bool got_write = false; struct dentry *dentry; const char *res; @@ -4592,44 +4763,10 @@ static const char *open_last_lookups(struct nameidata *nd, return ERR_PTR(-ECHILD); } } -retry: - if (open_flag & (O_CREAT | O_TRUNC | O_WRONLY | O_RDWR)) { - got_write = !mnt_want_write(nd->path.mnt); - /* - * do _not_ fail yet - we might not need that or fail with - * a different error; let lookup_open() decide; we'll be - * dropping this one anyway. - */ - } - if (open_flag & O_CREAT) - inode_lock(dir->d_inode); - else - inode_lock_shared(dir->d_inode); - dentry = lookup_open(nd, file, op, got_write, &delegated_inode); - if (!IS_ERR(dentry)) { - if (file->f_mode & FMODE_CREATED) - fsnotify_create(dir->d_inode, dentry); - if (file->f_mode & FMODE_OPENED) - fsnotify_open(file); - } - if (open_flag & O_CREAT) - inode_unlock(dir->d_inode); - else - inode_unlock_shared(dir->d_inode); - if (got_write) - mnt_drop_write(nd->path.mnt); - - if (IS_ERR(dentry)) { - if (is_delegated(&delegated_inode)) { - int error = break_deleg_wait(&delegated_inode); - - if (!error) - goto retry; - return ERR_PTR(error); - } + dentry = lookup_open(nd, file, op); + if (IS_ERR(dentry)) return ERR_CAST(dentry); - } if (file->f_mode & (FMODE_OPENED | FMODE_CREATED)) { dput(nd->path.dentry); @@ -5051,7 +5188,7 @@ struct file *dentry_create(struct path *path, int flags, umode_t mode, /* atomic_open will dput(dentry) on error */ dget(orig_dentry); - dentry = atomic_open(path, dentry, file, flags, mode); + dentry = atomic_open(path, dentry, file, flags, mode, create_error); error = PTR_ERR_OR_ZERO(dentry); if (IS_ERR(dentry)) @@ -5061,9 +5198,6 @@ struct file *dentry_create(struct path *path, int flags, umode_t mode, /* Drop the extra reference */ dput(orig_dentry); - if (unlikely(create_error) && error == -ENOENT) - error = create_error; - if (!error) { if (file->f_mode & FMODE_CREATED) fsnotify_create(dir->d_inode, dentry); @@ -5117,7 +5251,7 @@ int vfs_mknod(struct mnt_idmap *idmap, struct inode *dir, return -EPERM; if (!dir->i_op->mknod) - return -EPERM; + return -EOPNOTSUPP; mode = vfs_prepare_mode(idmap, dir, mode, mode, mode); error = devcgroup_inode_mknod(mode, dev); @@ -5256,11 +5390,11 @@ struct dentry *vfs_mkdir(struct mnt_idmap *idmap, struct inode *dir, if (error) goto err; - error = -EPERM; + error = -EOPNOTSUPP; if (!dir->i_op->mkdir) goto err; - mode = vfs_prepare_mode(idmap, dir, mode, S_IRWXUGO | S_ISVTX, 0); + mode = vfs_prepare_mode(idmap, dir, mode, S_IRWXUGO | S_ISVTX, S_IFDIR); error = security_inode_mkdir(dir, dentry, mode); if (error) goto err; @@ -5360,7 +5494,7 @@ int vfs_rmdir(struct mnt_idmap *idmap, struct inode *dir, return error; if (!dir->i_op->rmdir) - return -EPERM; + return -EOPNOTSUPP; dget(dentry); inode_lock(dentry->d_inode); @@ -5496,7 +5630,7 @@ int vfs_unlink(struct mnt_idmap *idmap, struct inode *dir, return error; if (!dir->i_op->unlink) - return -EPERM; + return -EOPNOTSUPP; inode_lock(target); if (IS_SWAPFILE(target)) @@ -5647,7 +5781,7 @@ int vfs_symlink(struct mnt_idmap *idmap, struct inode *dir, return error; if (!dir->i_op->symlink) - return -EPERM; + return -EOPNOTSUPP; error = security_inode_symlink(dir, dentry, oldname); if (error) @@ -5769,7 +5903,7 @@ int vfs_link(struct dentry *old_dentry, struct mnt_idmap *idmap, if (HAS_UNMAPPED_ID(idmap, inode)) return -EPERM; if (!dir->i_op->link) - return -EPERM; + return -EOPNOTSUPP; if (S_ISDIR(inode->i_mode)) return -EPERM; @@ -5978,7 +6112,7 @@ int vfs_rename(struct renamedata *rd) return error; if (!old_dir->i_op->rename) - return -EPERM; + return -EOPNOTSUPP; /* * If we are going to change the parent - check write permissions, diff --git a/fs/namespace.c b/fs/namespace.c index 3d5cd5bf3b05..1ecd96c918b3 100644 --- a/fs/namespace.c +++ b/fs/namespace.c @@ -2908,6 +2908,9 @@ static int do_change_type(const struct path *path, int ms_flags) for (m = mnt; m; m = (recurse ? next_mnt(m, mnt) : NULL)) change_mnt_propagation(m, type); + guard(mount_locked_reader)(); + touch_mnt_namespace(mnt->mnt_ns); + return 0; } @@ -3481,6 +3484,10 @@ static int do_set_group(const struct path *from_path, const struct path *to_path list_add(&to->mnt_share, &from->mnt_share); set_mnt_shared(to); } + + guard(mount_locked_reader)(); + touch_mnt_namespace(to->mnt_ns); + return 0; } @@ -6184,12 +6191,14 @@ static void __init init_mount_tree(void) struct path root; /* - * We create two mounts: + * We create three mounts: * * (1) nullfs with mount id 1 * (2) mutable rootfs with mount id 2 + * (3) private nullfs for kthreads (SB_KERNMOUNT) * - * with (2) mounted on top of (1). + * with (2) mounted on top of (1). The init_task's root and pwd + * are pointed at (3) so all kthreads start isolated in nullfs. */ nullfs_mnt = vfs_kern_mount(&nullfs_fs_type, 0, "nullfs", NULL); if (IS_ERR(nullfs_mnt)) @@ -6229,12 +6238,14 @@ static void __init init_mount_tree(void) init_mnt_ns.nr_mounts++; } + nullfs_mnt = kern_mount(&nullfs_fs_type); + if (IS_ERR(nullfs_mnt)) + panic("VFS: Failed to create private nullfs instance"); + root.mnt = nullfs_mnt; + root.dentry = nullfs_mnt->mnt_root; + init_task.nsproxy->mnt_ns = &init_mnt_ns; get_mnt_ns(&init_mnt_ns); - - /* The root and pwd always point to the mutable rootfs. */ - root.mnt = mnt; - root.dentry = mnt->mnt_root; set_fs_pwd(current->fs, &root); set_fs_root(current->fs, &root); @@ -6259,8 +6270,7 @@ void __init mnt_init(void) HASH_ZERO, &mp_hash_shift, &mp_hash_mask, 0, 0); - if (!mount_hashtable || !mountpoint_hashtable) - panic("Failed to allocate mount hash table\n"); + super_dev_init(); kernfs_init(); @@ -6274,6 +6284,7 @@ void __init mnt_init(void) shmem_init(); init_rootfs(); init_mount_tree(); + failfs_init(); } void put_mnt_ns(struct mnt_namespace *ns) @@ -6283,7 +6294,7 @@ void put_mnt_ns(struct mnt_namespace *ns) guard(namespace_excl)(); emptied_ns = ns; guard(mount_writer)(); - umount_tree(ns->root, 0); + umount_tree(ns->root, UMOUNT_CONNECTED); } struct vfsmount *kern_mount(struct file_system_type *type) diff --git a/fs/nfs/blocklayout/dev.c b/fs/nfs/blocklayout/dev.c index bb35f88501ce..368d20daf67b 100644 --- a/fs/nfs/blocklayout/dev.c +++ b/fs/nfs/blocklayout/dev.c @@ -4,6 +4,7 @@ */ #include #include +#include #include #include #include @@ -363,15 +364,22 @@ static struct file * bl_open_path(struct pnfs_block_volume *v, const char *prefix) { struct file *bdev_file; - const char *devname; + const char *devname __free(kfree) = NULL; devname = kasprintf(GFP_KERNEL, "/dev/disk/by-id/%s%*phN", prefix, v->scsi.designator_len, v->scsi.designator); if (!devname) return ERR_PTR(-ENOMEM); - bdev_file = bdev_file_open_by_path(devname, - BLK_OPEN_READ | BLK_OPEN_WRITE, NULL, NULL); + if (tsk_is_kthread(current)) { + scoped_with_init_fs() + bdev_file = bdev_file_open_by_path(devname, + BLK_OPEN_READ | BLK_OPEN_WRITE, + NULL, NULL); + } else { + bdev_file = bdev_file_open_by_path(devname, + BLK_OPEN_READ | BLK_OPEN_WRITE, NULL, NULL); + } if (IS_ERR(bdev_file)) { dprintk("failed to open device %s (%ld)\n", devname, PTR_ERR(bdev_file)); @@ -380,7 +388,6 @@ bl_open_path(struct pnfs_block_volume *v, const char *prefix) file_bdev(bdev_file)->bd_disk->disk_name); } - kfree(devname); return bdev_file; } diff --git a/fs/nfs/dir.c b/fs/nfs/dir.c index c7caffb31935..36f2e8588922 100644 --- a/fs/nfs/dir.c +++ b/fs/nfs/dir.c @@ -2427,9 +2427,9 @@ out_err: } int nfs_create(struct mnt_idmap *idmap, struct inode *dir, - struct dentry *dentry, umode_t mode, bool excl) + struct dentry *dentry, umode_t mode) { - return nfs_do_create(dir, dentry, mode, excl ? O_EXCL : 0); + return nfs_do_create(dir, dentry, mode, O_EXCL); } EXPORT_SYMBOL_GPL(nfs_create); @@ -2474,7 +2474,7 @@ struct dentry *nfs_mkdir(struct mnt_idmap *idmap, struct inode *dir, dir->i_sb->s_id, dir->i_ino, dentry); attr.ia_valid = ATTR_MODE; - attr.ia_mode = mode | S_IFDIR; + attr.ia_mode = mode; trace_nfs_mkdir_enter(dir, dentry); ret = NFS_PROTO(dir)->mkdir(dir, dentry, &attr); diff --git a/fs/nfs/internal.h b/fs/nfs/internal.h index e4533f583632..7f96a258af76 100644 --- a/fs/nfs/internal.h +++ b/fs/nfs/internal.h @@ -395,7 +395,7 @@ extern unsigned long nfs_access_cache_scan(struct shrinker *shrink, struct dentry *nfs_lookup(struct inode *, struct dentry *, unsigned int); void nfs_d_prune_case_insensitive_aliases(struct inode *inode); int nfs_create(struct mnt_idmap *, struct inode *, struct dentry *, - umode_t, bool); + umode_t); struct dentry *nfs_mkdir(struct mnt_idmap *, struct inode *, struct dentry *, umode_t); int nfs_rmdir(struct inode *, struct dentry *); diff --git a/fs/nilfs2/Kconfig b/fs/nilfs2/Kconfig index 7dae168e346e..0a5ace60e6ab 100644 --- a/fs/nilfs2/Kconfig +++ b/fs/nilfs2/Kconfig @@ -3,7 +3,7 @@ config NILFS2_FS tristate "NILFS2 file system support" select BUFFER_HEAD select CRC32 - select LEGACY_DIRECT_IO + select FS_IOMAP help NILFS2 is a log-structured file system (LFS) supporting continuous snapshotting. In addition to versioning capability of the entire diff --git a/fs/nilfs2/Makefile b/fs/nilfs2/Makefile index 43b60b8a4d07..516e6b85a03c 100644 --- a/fs/nilfs2/Makefile +++ b/fs/nilfs2/Makefile @@ -3,4 +3,4 @@ obj-$(CONFIG_NILFS2_FS) += nilfs2.o nilfs2-y := inode.o file.o dir.o super.o namei.o page.o mdt.o \ btnode.o bmap.o btree.o direct.o dat.o recovery.o \ the_nilfs.o segbuf.o segment.o cpfile.o sufile.o \ - ifile.o alloc.o gcinode.o ioctl.o sysfs.o + ifile.o alloc.o gcinode.o ioctl.o sysfs.o iomap.o diff --git a/fs/nilfs2/file.c b/fs/nilfs2/file.c index f93b68c4877c..ad2e87c049c9 100644 --- a/fs/nilfs2/file.c +++ b/fs/nilfs2/file.c @@ -10,9 +10,12 @@ #include #include #include +#include +#include #include #include "nilfs.h" #include "segment.h" +#include "iomap.h" int nilfs_sync_file(struct file *file, loff_t start, loff_t end, int datasync) { @@ -133,20 +136,51 @@ static int nilfs_file_mmap_prepare(struct vm_area_desc *desc) return 0; } +static int nilfs_file_open(struct inode *inode, struct file *file) +{ + file->f_mode |= FMODE_CAN_ODIRECT; + return generic_file_open(inode, file); +} + +static ssize_t nilfs_file_read_iter(struct kiocb *iocb, struct iov_iter *to) +{ + if (iocb->ki_flags & IOCB_DIRECT) { + return iomap_dio_rw(iocb, to, &nilfs_iomap_ops, + NULL, 0, NULL, 0); + } else + return generic_file_read_iter(iocb, to); +} + +static ssize_t nilfs_file_write_iter(struct kiocb *iocb, struct iov_iter *from) +{ + /* + * NILFS2 cannot perform true direct I/O writes: new blocks are + * delay-allocated and are only given a real disk address when + * the segment constructor writes them out as part of a log, + * which works directly on buffer_head lists rather than + * through iomap. Fall back to the ordinary buffered write path + * for O_DIRECT writes. + */ + if (iocb->ki_flags & IOCB_DIRECT) + iocb->ki_flags &= ~IOCB_DIRECT; + + return generic_file_write_iter(iocb, from); +} + /* * We have mostly NULL's here: the current defaults are ok for * the nilfs filesystem. */ const struct file_operations nilfs_file_operations = { .llseek = generic_file_llseek, - .read_iter = generic_file_read_iter, - .write_iter = generic_file_write_iter, + .read_iter = nilfs_file_read_iter, + .write_iter = nilfs_file_write_iter, .unlocked_ioctl = nilfs_ioctl, #ifdef CONFIG_COMPAT .compat_ioctl = nilfs_compat_ioctl, #endif /* CONFIG_COMPAT */ .mmap_prepare = nilfs_file_mmap_prepare, - .open = generic_file_open, + .open = nilfs_file_open, /* .release = nilfs_release_file, */ .fsync = nilfs_sync_file, .splice_read = filemap_splice_read, diff --git a/fs/nilfs2/inode.c b/fs/nilfs2/inode.c index 51f7e125a311..f4a9d9ea9c3f 100644 --- a/fs/nilfs2/inode.c +++ b/fs/nilfs2/inode.c @@ -257,18 +257,6 @@ static int nilfs_write_end(const struct kiocb *iocb, return err ? : copied; } -static ssize_t -nilfs_direct_IO(struct kiocb *iocb, struct iov_iter *iter) -{ - struct inode *inode = file_inode(iocb->ki_filp); - - if (iov_iter_rw(iter) == WRITE) - return 0; - - /* Needs synchronization with the cleaner */ - return blockdev_direct_IO(iocb, inode, iter, nilfs_get_block); -} - const struct address_space_operations nilfs_aops = { .read_folio = nilfs_read_folio, .writepages = nilfs_writepages, @@ -277,7 +265,6 @@ const struct address_space_operations nilfs_aops = { .write_begin = nilfs_write_begin, .write_end = nilfs_write_end, .invalidate_folio = block_invalidate_folio, - .direct_IO = nilfs_direct_IO, .migrate_folio = buffer_migrate_folio_norefs, .is_partially_uptodate = block_is_partially_uptodate, }; diff --git a/fs/nilfs2/iomap.c b/fs/nilfs2/iomap.c new file mode 100644 index 000000000000..3ae3bf6ed368 --- /dev/null +++ b/fs/nilfs2/iomap.c @@ -0,0 +1,71 @@ +// SPDX-License-Identifier: GPL-2.0+ +/* + * NILFS iomap support implementation. + * + * Written by Viacheslav Dubeyko. + */ + +#include +#include +#include "nilfs.h" +#include "mdt.h" +#include "iomap.h" + +static int nilfs_iomap_begin(struct inode *inode, loff_t offset, + loff_t length, unsigned int flags, + struct iomap *iomap, struct iomap *srcmap) +{ + struct the_nilfs *nilfs = inode->i_sb->s_fs_info; + struct nilfs_inode_info *ii = NILFS_I(inode); + sector_t blkoff = offset >> inode->i_blkbits; + unsigned int maxblocks; + __u64 blknum = 0; + int ret; + + /* Completely beyond EOF. Treat as hole */ + if (i_size_read(inode) <= offset) { + iomap->type = IOMAP_HOLE; + iomap->addr = IOMAP_NULL_ADDR; + iomap->offset = offset; + iomap->length = length; + return 0; + } + + /* Clamp length if the requested range goes beyond i_size */ + if (offset + length > i_size_read(inode)) { + loff_t i_size = i_size_read(inode); + unsigned int blocksize = i_blocksize(inode); + + length = round_up(i_size, blocksize) - offset; + } + + maxblocks = min_t(loff_t, length >> inode->i_blkbits, INT_MAX); + if (maxblocks == 0) + maxblocks = 1; + + down_read(&NILFS_MDT(nilfs->ns_dat)->mi_sem); + ret = nilfs_bmap_lookup_contig(ii->i_bmap, blkoff, &blknum, maxblocks); + up_read(&NILFS_MDT(nilfs->ns_dat)->mi_sem); + + if (ret == -ENOENT) { + iomap->type = IOMAP_HOLE; + iomap->addr = IOMAP_NULL_ADDR; + iomap->offset = offset; + iomap->length = min_t(loff_t, length, i_blocksize(inode)); + return 0; + } else if (ret < 0) + return ret; + + iomap->bdev = inode->i_sb->s_bdev; + iomap->offset = offset; + iomap->length = min_t(loff_t, length, (loff_t)ret << inode->i_blkbits); + iomap->addr = (loff_t)blknum << inode->i_blkbits; + iomap->type = IOMAP_MAPPED; + iomap->flags = IOMAP_F_MERGED; + + return 0; +} + +const struct iomap_ops nilfs_iomap_ops = { + .iomap_begin = nilfs_iomap_begin, +}; diff --git a/fs/nilfs2/iomap.h b/fs/nilfs2/iomap.h new file mode 100644 index 000000000000..adef3e22346d --- /dev/null +++ b/fs/nilfs2/iomap.h @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ +/* + * NILFS iomap support declarations. + * + * Written by Viacheslav Dubeyko. + */ + +#ifndef _NILFS_IOMAP_H +#define _NILFS_IOMAP_H + +extern const struct iomap_ops nilfs_iomap_ops; + +#endif /* _NILFS_IOMAP_H */ diff --git a/fs/nilfs2/namei.c b/fs/nilfs2/namei.c index e2fe95de3d71..77c5f7f74fbf 100644 --- a/fs/nilfs2/namei.c +++ b/fs/nilfs2/namei.c @@ -86,7 +86,7 @@ nilfs_lookup(struct inode *dir, struct dentry *dentry, unsigned int flags) * with d_instantiate(). */ static int nilfs_create(struct mnt_idmap *idmap, struct inode *dir, - struct dentry *dentry, umode_t mode, bool excl) + struct dentry *dentry, umode_t mode) { struct inode *inode; struct nilfs_transaction_info ti; @@ -231,7 +231,7 @@ static struct dentry *nilfs_mkdir(struct mnt_idmap *idmap, struct inode *dir, inc_nlink(dir); - inode = nilfs_new_inode(dir, S_IFDIR | mode); + inode = nilfs_new_inode(dir, mode); err = PTR_ERR(inode); if (IS_ERR(inode)) goto out_dir; diff --git a/fs/ntfs/iomap.c b/fs/ntfs/iomap.c index 26a1831a2c18..50b1a701f04d 100644 --- a/fs/ntfs/iomap.c +++ b/fs/ntfs/iomap.c @@ -277,8 +277,10 @@ static int ntfs_read_iomap_begin(struct inode *inode, loff_t offset, loff_t leng srcmap, true); } +static DEFINE_IOMAP_ITER_NEXT(ntfs_read_iomap_next, ntfs_read_iomap_begin); + const struct iomap_ops ntfs_read_iomap_ops = { - .iomap_begin = ntfs_read_iomap_begin, + .iomap_next = ntfs_read_iomap_next, }; /* @@ -329,13 +331,17 @@ static int ntfs_zero_read_iomap_end(struct inode *inode, loff_t pos, loff_t leng return written; } +static DEFINE_IOMAP_ITER_NEXT_END(ntfs_zero_read_iomap_next, + ntfs_seek_iomap_begin, ntfs_zero_read_iomap_end); + static const struct iomap_ops ntfs_zero_read_iomap_ops = { - .iomap_begin = ntfs_seek_iomap_begin, - .iomap_end = ntfs_zero_read_iomap_end, + .iomap_next = ntfs_zero_read_iomap_next, }; +static DEFINE_IOMAP_ITER_NEXT(ntfs_seek_iomap_next, ntfs_seek_iomap_begin); + const struct iomap_ops ntfs_seek_iomap_ops = { - .iomap_begin = ntfs_seek_iomap_begin, + .iomap_next = ntfs_seek_iomap_next, }; int ntfs_dio_zero_range(struct inode *inode, loff_t offset, loff_t length) @@ -732,9 +738,11 @@ static int ntfs_write_iomap_end(struct inode *inode, loff_t pos, loff_t length, return written; } +static DEFINE_IOMAP_ITER_NEXT_END(ntfs_write_iomap_next, + ntfs_write_iomap_begin, ntfs_write_iomap_end); + const struct iomap_ops ntfs_write_iomap_ops = { - .iomap_begin = ntfs_write_iomap_begin, - .iomap_end = ntfs_write_iomap_end, + .iomap_next = ntfs_write_iomap_next, }; static int ntfs_page_mkwrite_iomap_begin(struct inode *inode, loff_t offset, @@ -745,9 +753,11 @@ static int ntfs_page_mkwrite_iomap_begin(struct inode *inode, loff_t offset, NTFS_IOMAP_FLAGS_MKWRITE); } +static DEFINE_IOMAP_ITER_NEXT_END(ntfs_page_mkwrite_iomap_next, + ntfs_page_mkwrite_iomap_begin, ntfs_write_iomap_end); + const struct iomap_ops ntfs_page_mkwrite_iomap_ops = { - .iomap_begin = ntfs_page_mkwrite_iomap_begin, - .iomap_end = ntfs_write_iomap_end, + .iomap_next = ntfs_page_mkwrite_iomap_next, }; static int ntfs_dio_iomap_begin(struct inode *inode, loff_t offset, @@ -758,9 +768,11 @@ static int ntfs_dio_iomap_begin(struct inode *inode, loff_t offset, NTFS_IOMAP_FLAGS_DIO); } +static DEFINE_IOMAP_ITER_NEXT_END(ntfs_dio_iomap_next, + ntfs_dio_iomap_begin, ntfs_write_iomap_end); + const struct iomap_ops ntfs_dio_iomap_ops = { - .iomap_begin = ntfs_dio_iomap_begin, - .iomap_end = ntfs_write_iomap_end, + .iomap_next = ntfs_dio_iomap_next, }; static ssize_t ntfs_writeback_range(struct iomap_writepage_ctx *wpc, diff --git a/fs/ntfs/namei.c b/fs/ntfs/namei.c index 96045face63f..7091b2496fac 100644 --- a/fs/ntfs/namei.c +++ b/fs/ntfs/namei.c @@ -733,7 +733,7 @@ err_out: } static int ntfs_create(struct mnt_idmap *idmap, struct inode *dir, - struct dentry *dentry, umode_t mode, bool excl) + struct dentry *dentry, umode_t mode) { struct ntfs_volume *vol = NTFS_SB(dir->i_sb); struct ntfs_inode *ni; @@ -1079,7 +1079,7 @@ static struct dentry *ntfs_mkdir(struct mnt_idmap *idmap, struct inode *dir, if (!(vol->vol_flags & VOLUME_IS_DIRTY)) ntfs_set_volume_flags(vol, VOLUME_IS_DIRTY); - ni = __ntfs_create(idmap, dir, uname, uname_len, S_IFDIR | mode, 0, NULL, 0); + ni = __ntfs_create(idmap, dir, uname, uname_len, mode, 0, NULL, 0); kmem_cache_free(ntfs_name_cache, uname); if (IS_ERR(ni)) { err = PTR_ERR(ni); diff --git a/fs/ntfs3/inode.c b/fs/ntfs3/inode.c index 286678e23824..56b4f6469a28 100644 --- a/fs/ntfs3/inode.c +++ b/fs/ntfs3/inode.c @@ -2316,9 +2316,11 @@ const struct address_space_operations ntfs_aops_cmpr = { .invalidate_folio = iomap_invalidate_folio, }; +static DEFINE_IOMAP_ITER_NEXT_END(ntfs_iomap_next, ntfs_iomap_begin, + ntfs_iomap_end); + const struct iomap_ops ntfs_iomap_ops = { - .iomap_begin = ntfs_iomap_begin, - .iomap_end = ntfs_iomap_end, + .iomap_next = ntfs_iomap_next, }; const struct iomap_write_ops ntfs_iomap_folio_ops = { diff --git a/fs/ntfs3/namei.c b/fs/ntfs3/namei.c index 66eba128cc23..ec59bbabd3c5 100644 --- a/fs/ntfs3/namei.c +++ b/fs/ntfs3/namei.c @@ -112,7 +112,7 @@ static struct dentry *ntfs_lookup(struct inode *dir, struct dentry *dentry, * ntfs_create - inode_operations::create */ static int ntfs_create(struct mnt_idmap *idmap, struct inode *dir, - struct dentry *dentry, umode_t mode, bool excl) + struct dentry *dentry, umode_t mode) { return ntfs_create_inode(idmap, dir, dentry, NULL, S_IFREG | mode, 0, NULL, 0, NULL); @@ -232,7 +232,7 @@ static struct dentry *ntfs_mkdir(struct mnt_idmap *idmap, struct inode *dir, struct dentry *dentry, umode_t mode) { return ERR_PTR(ntfs_create_inode(idmap, dir, dentry, NULL, - S_IFDIR | mode, 0, NULL, 0, NULL)); + mode, 0, NULL, 0, NULL)); } /* diff --git a/fs/nullfs.c b/fs/nullfs.c index fdbd3e5d3d71..e06352c7b2cc 100644 --- a/fs/nullfs.c +++ b/fs/nullfs.c @@ -4,6 +4,8 @@ #include #include +#include "mount.h" + static const struct super_operations nullfs_super_operations = { .statfs = simple_statfs, }; @@ -40,14 +42,9 @@ static int nullfs_fs_fill_super(struct super_block *s, struct fs_context *fc) return 0; } -/* - * For now this is a single global instance. If needed we can make it - * mountable by userspace at which point we will need to make it - * multi-instance. - */ static int nullfs_fs_get_tree(struct fs_context *fc) { - return get_tree_single(fc, nullfs_fs_fill_super); + return get_tree_nodev(fc, nullfs_fs_fill_super); } static const struct fs_context_operations nullfs_fs_context_ops = { @@ -57,9 +54,8 @@ static const struct fs_context_operations nullfs_fs_context_ops = { static int nullfs_init_fs_context(struct fs_context *fc) { fc->ops = &nullfs_fs_context_ops; - fc->global = true; - fc->sb_flags = SB_NOUSER; - fc->s_iflags = SB_I_NOEXEC | SB_I_NODEV; + fc->sb_flags |= SB_NOUSER; + fc->s_iflags |= SB_I_NOEXEC | SB_I_NODEV; return 0; } diff --git a/fs/ocfs2/dlmfs/dlmfs.c b/fs/ocfs2/dlmfs/dlmfs.c index 5821e33df78f..53df5dd10ad0 100644 --- a/fs/ocfs2/dlmfs/dlmfs.c +++ b/fs/ocfs2/dlmfs/dlmfs.c @@ -422,7 +422,7 @@ static struct dentry *dlmfs_mkdir(struct mnt_idmap * idmap, goto bail; } - inode = dlmfs_get_inode(dir, dentry, mode | S_IFDIR); + inode = dlmfs_get_inode(dir, dentry, mode); if (!inode) { status = -ENOMEM; mlog_errno(status); @@ -453,8 +453,7 @@ bail: static int dlmfs_create(struct mnt_idmap *idmap, struct inode *dir, struct dentry *dentry, - umode_t mode, - bool excl) + umode_t mode) { int status = 0; struct inode *inode; diff --git a/fs/ocfs2/namei.c b/fs/ocfs2/namei.c index 1277666c77cd..b23dd678a7e0 100644 --- a/fs/ocfs2/namei.c +++ b/fs/ocfs2/namei.c @@ -657,7 +657,7 @@ static struct dentry *ocfs2_mkdir(struct mnt_idmap *idmap, trace_ocfs2_mkdir(dir, dentry, dentry->d_name.len, dentry->d_name.name, OCFS2_I(dir)->ip_blkno, mode); - ret = ocfs2_mknod(&nop_mnt_idmap, dir, dentry, mode | S_IFDIR, 0); + ret = ocfs2_mknod(&nop_mnt_idmap, dir, dentry, mode, 0); if (ret) mlog_errno(ret); @@ -667,8 +667,7 @@ static struct dentry *ocfs2_mkdir(struct mnt_idmap *idmap, static int ocfs2_create(struct mnt_idmap *idmap, struct inode *dir, struct dentry *dentry, - umode_t mode, - bool excl) + umode_t mode) { int ret; diff --git a/fs/ocfs2/super.c b/fs/ocfs2/super.c index bcac614ff02a..c62e389d4dd6 100644 --- a/fs/ocfs2/super.c +++ b/fs/ocfs2/super.c @@ -1882,7 +1882,6 @@ static void ocfs2_dismount_volume(struct super_block *sb, int mnt_err) ocfs2_delete_osb(osb); kfree(osb); - sb->s_dev = 0; sb->s_fs_info = NULL; } diff --git a/fs/omfs/dir.c b/fs/omfs/dir.c index 2ed541fccf33..692297cf84e7 100644 --- a/fs/omfs/dir.c +++ b/fs/omfs/dir.c @@ -282,11 +282,11 @@ out_free_inode: static struct dentry *omfs_mkdir(struct mnt_idmap *idmap, struct inode *dir, struct dentry *dentry, umode_t mode) { - return ERR_PTR(omfs_add_node(dir, dentry, mode | S_IFDIR)); + return ERR_PTR(omfs_add_node(dir, dentry, mode)); } static int omfs_create(struct mnt_idmap *idmap, struct inode *dir, - struct dentry *dentry, umode_t mode, bool excl) + struct dentry *dentry, umode_t mode) { return omfs_add_node(dir, dentry, mode | S_IFREG); } diff --git a/fs/open.c b/fs/open.c index 408925d7bd0b..6b1c14e684a9 100644 --- a/fs/open.c +++ b/fs/open.c @@ -570,9 +570,12 @@ retry: SYSCALL_DEFINE1(fchdir, unsigned int, fd) { - CLASS(fd_raw, f)(fd); int error; + if ((int)fd == FD_FAILFS_ROOT) + return failfs_current_chdir(); + + CLASS(fd_raw, f)(fd); if (fd_empty(f)) return -EBADF; @@ -615,6 +618,52 @@ dput_and_out: return error; } +SYSCALL_DEFINE2(fchroot, int, fd, unsigned int, flags) +{ + struct path path; + int error; + + if (flags) + return -EINVAL; + + if (fd == FD_FAILFS_ROOT) { + if (!ns_capable(current_user_ns(), CAP_SYS_CHROOT)) { + if (!task_no_new_privs(current)) + return -EPERM; + /* A shared fs_struct lets a sibling exec setuid past the check above. */ + if (current->fs->users != 1) + return -EINVAL; + /* Moving the root to failfs lifts the old root's ".." barrier. */ + if (current_chrooted()) + return -EPERM; + } + failfs_get_root(&path); + } else { + CLASS(fd_raw, f)(fd); + if (fd_empty(f)) + return -EBADF; + + if (!d_can_lookup(fd_file(f)->f_path.dentry)) + return -ENOTDIR; + + error = file_permission(fd_file(f), MAY_EXEC | MAY_CHDIR); + if (error) + return error; + + if (!ns_capable(current_user_ns(), CAP_SYS_CHROOT)) + return -EPERM; + + path = fd_file(f)->f_path; + path_get(&path); + } + + error = security_path_chroot(&path); + if (!error) + set_fs_root(current->fs, &path); + path_put(&path); + return error; +} + int chmod_common(const struct path *path, umode_t mode) { struct inode *inode = path->dentry->d_inode; diff --git a/fs/orangefs/namei.c b/fs/orangefs/namei.c index 75e65e72c2d6..8ebc34e112d5 100644 --- a/fs/orangefs/namei.c +++ b/fs/orangefs/namei.c @@ -18,8 +18,7 @@ static int orangefs_create(struct mnt_idmap *idmap, struct inode *dir, struct dentry *dentry, - umode_t mode, - bool exclusive) + umode_t mode) { struct orangefs_inode_s *parent = ORANGEFS_I(dir); struct orangefs_kernel_op_s *new_op; @@ -333,7 +332,7 @@ static struct dentry *orangefs_mkdir(struct mnt_idmap *idmap, struct inode *dir, ref = new_op->downcall.resp.mkdir.refn; - inode = orangefs_new_inode(dir->i_sb, dir, S_IFDIR | mode, 0, &ref); + inode = orangefs_new_inode(dir->i_sb, dir, mode, 0, &ref); if (IS_ERR(inode)) { gossip_err("*** Failed to allocate orangefs dir inode\n"); ret = PTR_ERR(inode); diff --git a/fs/overlayfs/dir.c b/fs/overlayfs/dir.c index a033743dbf51..610c0116252c 100644 --- a/fs/overlayfs/dir.c +++ b/fs/overlayfs/dir.c @@ -689,8 +689,8 @@ static int ovl_create_or_link(struct dentry *dentry, struct inode *inode, return err; } -static int ovl_create_object(struct dentry *dentry, int mode, dev_t rdev, - const char *link) +static int ovl_create_object(struct mnt_idmap *idmap, struct dentry *dentry, + int mode, dev_t rdev, const char *link) { int err; struct inode *inode; @@ -717,7 +717,7 @@ static int ovl_create_object(struct dentry *dentry, int mode, dev_t rdev, inode_state_set(inode, I_CREATING); spin_unlock(&inode->i_lock); - inode_init_owner(&nop_mnt_idmap, inode, dentry->d_parent->d_inode, mode); + inode_init_owner(idmap, inode, dentry->d_parent->d_inode, mode); attr.mode = inode->i_mode; err = ovl_create_or_link(dentry, inode, &attr, false); @@ -732,15 +732,15 @@ out: } static int ovl_create(struct mnt_idmap *idmap, struct inode *dir, - struct dentry *dentry, umode_t mode, bool excl) + struct dentry *dentry, umode_t mode) { - return ovl_create_object(dentry, (mode & 07777) | S_IFREG, 0, NULL); + return ovl_create_object(idmap, dentry, (mode & 07777) | S_IFREG, 0, NULL); } static struct dentry *ovl_mkdir(struct mnt_idmap *idmap, struct inode *dir, struct dentry *dentry, umode_t mode) { - return ERR_PTR(ovl_create_object(dentry, (mode & 07777) | S_IFDIR, 0, NULL)); + return ERR_PTR(ovl_create_object(idmap, dentry, (mode & 07777) | S_IFDIR, 0, NULL)); } static int ovl_mknod(struct mnt_idmap *idmap, struct inode *dir, @@ -750,13 +750,13 @@ static int ovl_mknod(struct mnt_idmap *idmap, struct inode *dir, if (S_ISCHR(mode) && rdev == WHITEOUT_DEV) return -EPERM; - return ovl_create_object(dentry, mode, rdev, NULL); + return ovl_create_object(idmap, dentry, mode, rdev, NULL); } static int ovl_symlink(struct mnt_idmap *idmap, struct inode *dir, struct dentry *dentry, const char *link) { - return ovl_create_object(dentry, S_IFLNK, 0, link); + return ovl_create_object(idmap, dentry, S_IFLNK, 0, link); } static int ovl_set_link_redirect(struct dentry *dentry) @@ -1444,7 +1444,7 @@ static int ovl_tmpfile(struct mnt_idmap *idmap, struct inode *dir, if (!inode) goto drop_write; - inode_init_owner(&nop_mnt_idmap, inode, dir, mode); + inode_init_owner(idmap, inode, dir, mode); err = ovl_create_tmpfile(file, dentry, inode, inode->i_mode); if (err) goto put_inode; diff --git a/fs/overlayfs/inode.c b/fs/overlayfs/inode.c index bc71231cad53..401cb8c75520 100644 --- a/fs/overlayfs/inode.c +++ b/fs/overlayfs/inode.c @@ -26,10 +26,18 @@ int ovl_setattr(struct mnt_idmap *idmap, struct dentry *dentry, bool full_copy_up = false; struct dentry *upperdentry; - err = setattr_prepare(&nop_mnt_idmap, dentry, attr); + err = setattr_prepare(idmap, dentry, attr); if (err) return err; + /* Rebase ownership from the mount idmap into overlay id space. */ + if (attr->ia_valid & ATTR_UID) + attr->ia_vfsuid = VFSUIDT_INIT(from_vfsuid(idmap, + i_user_ns(d_inode(dentry)), attr->ia_vfsuid)); + if (attr->ia_valid & ATTR_GID) + attr->ia_vfsgid = VFSGIDT_INIT(from_vfsgid(idmap, + i_user_ns(d_inode(dentry)), attr->ia_vfsgid)); + if (attr->ia_valid & ATTR_SIZE) { /* Truncate should trigger data copy up as well */ full_copy_up = true; @@ -172,6 +180,8 @@ int ovl_getattr(struct mnt_idmap *idmap, const struct path *path, int fsid = 0; int err; bool metacopy_blocks = false; + vfsuid_t vfsuid; + vfsgid_t vfsgid; metacopy_blocks = ovl_is_metacopy_dentry(dentry); @@ -284,6 +294,12 @@ int ovl_getattr(struct mnt_idmap *idmap, const struct path *path, if (!is_dir && ovl_test_flag(OVL_INDEX, d_inode(dentry))) stat->nlink = dentry->d_inode->i_nlink; + /* Map ownership of the real inode through the overlay mount idmap. */ + vfsuid = make_vfsuid(idmap, i_user_ns(inode), stat->uid); + vfsgid = make_vfsgid(idmap, i_user_ns(inode), stat->gid); + stat->uid = vfsuid_into_kuid(vfsuid); + stat->gid = vfsgid_into_kgid(vfsgid); + return err; } @@ -306,7 +322,7 @@ int ovl_permission(struct mnt_idmap *idmap, * Check overlay inode with the creds of task and underlying inode * with creds of mounter */ - err = generic_permission(&nop_mnt_idmap, inode, mask); + err = generic_permission(idmap, inode, mask); if (err) return err; @@ -534,7 +550,7 @@ int ovl_set_acl(struct mnt_idmap *idmap, struct dentry *dentry, return -EOPNOTSUPP; if (type == ACL_TYPE_DEFAULT && !S_ISDIR(inode->i_mode)) return acl ? -EACCES : 0; - if (!inode_owner_or_capable(&nop_mnt_idmap, inode)) + if (!inode_owner_or_capable(idmap, inode)) return -EPERM; /* @@ -542,8 +558,8 @@ int ovl_set_acl(struct mnt_idmap *idmap, struct dentry *dentry, * be done with mounter's capabilities and so that won't do it for us). */ if (unlikely(inode->i_mode & S_ISGID) && type == ACL_TYPE_ACCESS && - !in_group_p(inode->i_gid) && - !capable_wrt_inode_uidgid(&nop_mnt_idmap, inode, CAP_FSETID)) { + !in_group_or_capable(idmap, inode, + i_gid_into_vfsgid(idmap, inode))) { struct iattr iattr = { .ia_valid = ATTR_KILL_SGID }; err = ovl_setattr(&nop_mnt_idmap, dentry, &iattr); diff --git a/fs/overlayfs/overlayfs.h b/fs/overlayfs/overlayfs.h index b75df37f70ac..e0d8c6152e9f 100644 --- a/fs/overlayfs/overlayfs.h +++ b/fs/overlayfs/overlayfs.h @@ -320,6 +320,7 @@ static inline int ovl_do_setxattr(struct ovl_fs *ofs, struct dentry *dentry, const char *name, const void *value, size_t size, int flags) { + /* Use vfs_setxattr(), not __vfs_setxattr(): it idmaps the security.capability rootid. */ int err = vfs_setxattr(ovl_upper_mnt_idmap(ofs), dentry, name, value, size, flags); diff --git a/fs/overlayfs/super.c b/fs/overlayfs/super.c index 60f0b7ceef0a..f2889cf9bc07 100644 --- a/fs/overlayfs/super.c +++ b/fs/overlayfs/super.c @@ -1573,7 +1573,7 @@ struct file_system_type ovl_fs_type = { .name = "overlay", .init_fs_context = ovl_init_fs_context, .parameters = ovl_parameter_spec, - .fs_flags = FS_USERNS_MOUNT, + .fs_flags = FS_USERNS_MOUNT | FS_ALLOW_IDMAP, .kill_sb = kill_anon_super, }; MODULE_ALIAS_FS("overlay"); diff --git a/fs/overlayfs/xattrs.c b/fs/overlayfs/xattrs.c index 859e80ae6f40..5ae44b9c8790 100644 --- a/fs/overlayfs/xattrs.c +++ b/fs/overlayfs/xattrs.c @@ -84,6 +84,7 @@ static int ovl_xattr_get(struct dentry *dentry, struct inode *inode, const char struct path realpath; ovl_i_path_real(inode, &realpath); + /* Use vfs_getxattr(), not __vfs_getxattr(): it idmaps the security.capability rootid. */ with_ovl_creds(dentry->d_sb) return vfs_getxattr(mnt_idmap(realpath.mnt), realpath.dentry, name, value, size); } diff --git a/fs/pipe.c b/fs/pipe.c index 429b0714ec57..3c6061cefe79 100644 --- a/fs/pipe.c +++ b/fs/pipe.c @@ -111,47 +111,6 @@ void pipe_double_lock(struct pipe_inode_info *pipe1, pipe_lock(pipe2); } -#define PIPE_PREALLOC_MAX 8 - -struct anon_pipe_prealloc { - struct page *pages[PIPE_PREALLOC_MAX]; - unsigned int count; -}; - -/* - * Pre-allocate pages outside pipe->mutex for multi-page writes. - * alloc_page() with GFP_HIGHUSER can sleep in reclaim and runs memcg - * charging; doing it under the mutex stalls a concurrent reader. - * - * Loop alloc_page() instead of alloc_pages_bulk_*(): the bulk path refuses - * __GFP_ACCOUNT under memcg (see commit 8dcb3060d81d "memcg: page_alloc: - * skip bulk allocator for __GFP_ACCOUNT") and silently degrades to a single - * page. A per-page loop keeps memcg accounting and the task NUMA mempolicy - * honoured for every page; the per-call overhead is small compared to the - * pipe->mutex hold-time being shrunk. Any shortfall is covered by the - * in-lock alloc_page() fallback in anon_pipe_get_page(). - */ -static void anon_pipe_get_page_prealloc(struct anon_pipe_prealloc *prealloc, - size_t total_len) -{ - unsigned int want, i; - struct page *page; - - prealloc->count = 0; - if (total_len <= PAGE_SIZE) - return; - - want = min_t(unsigned int, DIV_ROUND_UP(total_len, PAGE_SIZE), - PIPE_PREALLOC_MAX); - - for (i = 0; i < want; i++) { - page = alloc_page(GFP_HIGHUSER | __GFP_ACCOUNT); - if (!page) - break; - prealloc->pages[prealloc->count++] = page; - } -} - static struct page *anon_pipe_prealloc_pop(struct anon_pipe_prealloc *prealloc) { if (!prealloc->count) @@ -162,24 +121,85 @@ static struct page *anon_pipe_prealloc_pop(struct anon_pipe_prealloc *prealloc) return prealloc->pages[prealloc->count]; } -static struct page *anon_pipe_get_page(struct pipe_inode_info *pipe, - struct anon_pipe_prealloc *prealloc) +/* Push a page to the prealloc pool. Returns true if added, false if full. */ +static bool anon_pipe_prealloc_push(struct anon_pipe_prealloc *prealloc, + struct page *page) +{ + if (prealloc->count >= PIPE_PREALLOC_MAX) + return false; + prealloc->pages[prealloc->count++] = page; + return true; +} + +/* + * Top up the pipe's own pool, then take pipe->mutex and return with it held. + * The shortfall is allocated outside the lock; the push and the caller's write + * then run under a single lock acquisition, avoiding a separate prefill + * lock/unlock cycle. anon_pipe_get_page() drains the pool instead of allocating + * under the lock. + */ +static void anon_pipe_prefill_and_lock(struct pipe_inode_info *pipe, size_t total_len) +{ + struct page *pages[PIPE_PREALLOC_MAX]; + unsigned int want, have, need, n = 0; + + want = min_t(unsigned int, DIV_ROUND_UP(total_len, PAGE_SIZE), + PIPE_PREALLOC_MAX); + /* Unlocked read; the pool is refilled under the lock below. */ + have = min_t(unsigned int, READ_ONCE(pipe->prealloc.count), want); + need = want - have; + + if (!need) { + mutex_lock(&pipe->mutex); + return; + } + + while (n < need) { + struct page *page = alloc_page(GFP_HIGHUSER | __GFP_ACCOUNT); + + if (!page) + break; + pages[n++] = page; + } + + mutex_lock(&pipe->mutex); + while (n && anon_pipe_prealloc_push(&pipe->prealloc, pages[n - 1])) + n--; + + /* + * Just flush any extra page that got affected by the TOCTOU + * effect + */ + while (n) + put_page(pages[--n]); +} + +/* + * Called with pipe->mutex held. Trim the pool down to PIPE_PREALLOC_KEEP under + * the lock, drop it, then free the excess outside the critical section. + */ +static void anon_pipe_trim_and_unlock(struct pipe_inode_info *pipe) +{ + struct page *excess[PIPE_PREALLOC_MAX]; + unsigned int nexcess = 0; + + while (pipe->prealloc.count > PIPE_PREALLOC_KEEP) + excess[nexcess++] = anon_pipe_prealloc_pop(&pipe->prealloc); + mutex_unlock(&pipe->mutex); + + while (nexcess) + put_page(excess[--nexcess]); +} + +static struct page *anon_pipe_get_page(struct pipe_inode_info *pipe) { struct page *page; - /* Drain prealloc first to keep tmp_page[] hot for later small writes. */ - page = anon_pipe_prealloc_pop(prealloc); + /* Drain the prealloc pool before allocating. Called with mutex held. */ + page = anon_pipe_prealloc_pop(&pipe->prealloc); if (page) return page; - for (int i = 0; i < ARRAY_SIZE(pipe->tmp_page); i++) { - if (pipe->tmp_page[i]) { - page = pipe->tmp_page[i]; - pipe->tmp_page[i] = NULL; - return page; - } - } - /* FWIW: This is called with pipe->mutex held */ return alloc_page(GFP_HIGHUSER | __GFP_ACCOUNT); } @@ -187,48 +207,11 @@ static struct page *anon_pipe_get_page(struct pipe_inode_info *pipe, static void anon_pipe_put_page(struct pipe_inode_info *pipe, struct page *page) { - if (page_count(page) == 1) { - for (int i = 0; i < ARRAY_SIZE(pipe->tmp_page); i++) { - if (!pipe->tmp_page[i]) { - pipe->tmp_page[i] = page; - return; - } - } - } - - put_page(page); -} - -/* - * Stash leftover prealloc pages in tmp_page[] so the next write to this - * pipe gets a hot page without entering the allocator. - */ -static void anon_pipe_refill_tmp_pages(struct pipe_inode_info *pipe, - struct anon_pipe_prealloc *prealloc) -{ - int i, idx; - - if (!prealloc->count) + if (page_count(page) == 1 && + anon_pipe_prealloc_push(&pipe->prealloc, page)) return; - for (i = 0; i < ARRAY_SIZE(pipe->tmp_page); i++) { - if (pipe->tmp_page[i]) - continue; - if (!prealloc->count) - return; - idx = --prealloc->count; - pipe->tmp_page[i] = prealloc->pages[idx]; - prealloc->pages[idx] = NULL; - } -} - -/* Runs after mutex_unlock() to keep put_page() out of the critical section. */ -static void anon_pipe_free_pages(struct anon_pipe_prealloc *prealloc) -{ - while (prealloc->count) { - prealloc->count--; - put_page(prealloc->pages[prealloc->count]); - } + put_page(page); } static void anon_pipe_buf_release(struct pipe_inode_info *pipe, @@ -485,7 +468,8 @@ anon_pipe_read(struct kiocb *iocb, struct iov_iter *to) } if (pipe_is_empty(pipe)) wake_next_reader = false; - mutex_unlock(&pipe->mutex); + /* Consumed buffers may have refilled the pool; trim it and unlock. */ + anon_pipe_trim_and_unlock(pipe); if (wake_writer) wake_up_interruptible_sync_poll(&pipe->wr_wait, EPOLLOUT | EPOLLWRNORM); @@ -524,7 +508,6 @@ anon_pipe_write(struct kiocb *iocb, struct iov_iter *from) { struct file *filp = iocb->ki_filp; struct pipe_inode_info *pipe = filp->private_data; - struct anon_pipe_prealloc prealloc; unsigned int head; ssize_t ret = 0; size_t total_len = iov_iter_count(from); @@ -548,9 +531,7 @@ anon_pipe_write(struct kiocb *iocb, struct iov_iter *from) if (unlikely(total_len == 0)) return 0; - anon_pipe_get_page_prealloc(&prealloc, total_len); - - mutex_lock(&pipe->mutex); + anon_pipe_prefill_and_lock(pipe, total_len); if (!pipe->readers) { if ((iocb->ki_flags & IOCB_NOSIGNAL) == 0) @@ -607,7 +588,7 @@ anon_pipe_write(struct kiocb *iocb, struct iov_iter *from) struct page *page; int copied; - page = anon_pipe_get_page(pipe, &prealloc); + page = anon_pipe_get_page(pipe); if (unlikely(!page)) { if (!ret) ret = -ENOMEM; @@ -671,11 +652,9 @@ anon_pipe_write(struct kiocb *iocb, struct iov_iter *from) wake_next_writer = true; } out: - anon_pipe_refill_tmp_pages(pipe, &prealloc); if (pipe_is_full(pipe)) wake_next_writer = false; - mutex_unlock(&pipe->mutex); - anon_pipe_free_pages(&prealloc); + anon_pipe_trim_and_unlock(pipe); /* * If we do do a wakeup event, we do a 'sync' wakeup, because we @@ -956,10 +935,8 @@ void free_pipe_info(struct pipe_inode_info *pipe) if (pipe->watch_queue) put_watch_queue(pipe->watch_queue); #endif - for (i = 0; i < ARRAY_SIZE(pipe->tmp_page); i++) { - if (pipe->tmp_page[i]) - __free_page(pipe->tmp_page[i]); - } + for (i = 0; i < pipe->prealloc.count; i++) + __free_page(pipe->prealloc.pages[i]); kfree(pipe->bufs); kfree(pipe); } diff --git a/fs/posix_acl.c b/fs/posix_acl.c index 3dc62c1c2708..18b302f94174 100644 --- a/fs/posix_acl.c +++ b/fs/posix_acl.c @@ -747,8 +747,6 @@ static int posix_acl_fix_xattr_common(const void *value, size_t size) count = posix_acl_xattr_count(size); if (count < 0) return -EINVAL; - if (count == 0) - return 0; return count; } diff --git a/fs/proc/array.c b/fs/proc/array.c index 479ea8cb4ef4..f6f75d206762 100644 --- a/fs/proc/array.c +++ b/fs/proc/array.c @@ -168,8 +168,8 @@ static inline void task_state(struct seq_file *m, struct pid_namespace *ns, cred = get_task_cred(p); task_lock(p); - if (p->fs) - umask = p->fs->umask; + if (p->real_fs) + umask = p->real_fs->umask; if (p->files) max_fds = files_fdtable(p->files)->max_fds; task_unlock(p); diff --git a/fs/proc/base.c b/fs/proc/base.c index 780f81259052..6a39de424f62 100644 --- a/fs/proc/base.c +++ b/fs/proc/base.c @@ -211,8 +211,8 @@ static int get_task_root(struct task_struct *task, struct path *root) int result = -ENOENT; task_lock(task); - if (task->fs) { - get_fs_root(task->fs, root); + if (task->real_fs) { + get_fs_root(task->real_fs, root); result = 0; } task_unlock(task); @@ -225,8 +225,8 @@ static int proc_cwd_link(struct dentry *dentry, struct path *path, int result = -ENOENT; task_lock(task); - if (task->fs) { - get_fs_pwd(task->fs, path); + if (task->real_fs) { + get_fs_pwd(task->real_fs, path); result = 0; } task_unlock(task); diff --git a/fs/proc_namespace.c b/fs/proc_namespace.c index 5c555db68aa2..036356c0a55b 100644 --- a/fs/proc_namespace.c +++ b/fs/proc_namespace.c @@ -254,13 +254,13 @@ static int mounts_open_common(struct inode *inode, struct file *file, } ns = nsp->mnt_ns; get_mnt_ns(ns); - if (!task->fs) { + if (!task->real_fs) { task_unlock(task); put_task_struct(task); ret = -ENOENT; goto err_put_ns; } - get_fs_root(task->fs, &root); + get_fs_root(task->real_fs, &root); task_unlock(task); put_task_struct(task); diff --git a/fs/ramfs/inode.c b/fs/ramfs/inode.c index 3987639ed132..0a88ede48e0a 100644 --- a/fs/ramfs/inode.c +++ b/fs/ramfs/inode.c @@ -121,14 +121,14 @@ out: static struct dentry *ramfs_mkdir(struct mnt_idmap *idmap, struct inode *dir, struct dentry *dentry, umode_t mode) { - int retval = ramfs_mknod(&nop_mnt_idmap, dir, dentry, mode | S_IFDIR, 0); + int retval = ramfs_mknod(&nop_mnt_idmap, dir, dentry, mode, 0); if (!retval) inc_nlink(dir); return ERR_PTR(retval); } static int ramfs_create(struct mnt_idmap *idmap, struct inode *dir, - struct dentry *dentry, umode_t mode, bool excl) + struct dentry *dentry, umode_t mode) { return ramfs_mknod(&nop_mnt_idmap, dir, dentry, mode | S_IFREG, 0); } diff --git a/fs/romfs/super.c b/fs/romfs/super.c index ac55193bf398..7f783d6173e8 100644 --- a/fs/romfs/super.c +++ b/fs/romfs/super.c @@ -240,6 +240,8 @@ static struct dentry *romfs_lookup(struct inode *dir, struct dentry *dentry, if ((be32_to_cpu(ri.next) & ROMFH_TYPE) == ROMFH_HRD) offset = be32_to_cpu(ri.spec) & ROMFH_MASK; inode = romfs_iget(dir->i_sb, offset); + if (IS_ERR(inode)) + return ERR_CAST(inode); break; } @@ -262,6 +264,8 @@ static const struct inode_operations romfs_dir_inode_operations = { .lookup = romfs_lookup, }; +#define ROMFS_MAX_HARDLINK_DEPTH 64 + /* * get a romfs inode based on its position in the image (which doubles as the * inode number) @@ -273,6 +277,7 @@ static struct inode *romfs_iget(struct super_block *sb, unsigned long pos) struct inode *i; unsigned long nlen; unsigned nextfh; + unsigned int depth = 0; int ret; umode_t mode; @@ -289,6 +294,9 @@ static struct inode *romfs_iget(struct super_block *sb, unsigned long pos) if ((nextfh & ROMFH_TYPE) != ROMFH_HRD) break; + if (++depth > ROMFS_MAX_HARDLINK_DEPTH) + return ERR_PTR(-ELOOP); + pos = be32_to_cpu(ri.spec) & ROMFH_MASK; } @@ -587,7 +595,7 @@ static void romfs_kill_sb(struct super_block *sb) #ifdef CONFIG_ROMFS_ON_BLOCK if (sb->s_bdev) { sync_blockdev(sb->s_bdev); - bdev_fput(sb->s_bdev_file); + fs_bdev_file_release(sb->s_bdev_file, sb); } #endif } diff --git a/fs/seq_file.c b/fs/seq_file.c index 4745db2a34d1..456c78719fd0 100644 --- a/fs/seq_file.c +++ b/fs/seq_file.c @@ -428,7 +428,7 @@ EXPORT_SYMBOL(seq_bprintf); #endif /* CONFIG_BINARY_PRINTF */ /** - * mangle_path - mangle and copy path to buffer beginning + * seq_mangle_path - mangle and copy path to buffer beginning * @s: buffer start * @p: beginning of path in above buffer * @esc: set of characters that need escaping @@ -438,7 +438,7 @@ EXPORT_SYMBOL(seq_bprintf); * Returns pointer past last written character in @s, or NULL in case of * failure. */ -char *mangle_path(char *s, const char *p, const char *esc) +char *seq_mangle_path(char *s, const char *p, const char *esc) { while (s <= p) { char c = *p++; @@ -457,7 +457,6 @@ char *mangle_path(char *s, const char *p, const char *esc) } return NULL; } -EXPORT_SYMBOL(mangle_path); /** * seq_path - seq_file interface to print a pathname @@ -477,7 +476,7 @@ int seq_path(struct seq_file *m, const struct path *path, const char *esc) if (size) { char *p = d_path(path, buf, size); if (!IS_ERR(p)) { - char *end = mangle_path(buf, p, esc); + char *end = seq_mangle_path(buf, p, esc); if (end) res = end - buf; } @@ -520,7 +519,7 @@ int seq_path_root(struct seq_file *m, const struct path *path, return SEQ_SKIP; res = PTR_ERR(p); if (!IS_ERR(p)) { - char *end = mangle_path(buf, p, esc); + char *end = seq_mangle_path(buf, p, esc); if (end) res = end - buf; else @@ -544,7 +543,7 @@ int seq_dentry(struct seq_file *m, struct dentry *dentry, const char *esc) if (size) { char *p = dentry_path(dentry, buf, size); if (!IS_ERR(p)) { - char *end = mangle_path(buf, p, esc); + char *end = seq_mangle_path(buf, p, esc); if (end) res = end - buf; } diff --git a/fs/smb/client/cifsfs.h b/fs/smb/client/cifsfs.h index 854e672a4e37..287c632392c4 100644 --- a/fs/smb/client/cifsfs.h +++ b/fs/smb/client/cifsfs.h @@ -54,7 +54,7 @@ void cifs_sb_deactive(struct super_block *sb); extern const struct inode_operations cifs_dir_inode_ops; struct inode *cifs_root_iget(struct super_block *sb); int cifs_create(struct mnt_idmap *idmap, struct inode *dir, - struct dentry *direntry, umode_t mode, bool excl); + struct dentry *direntry, umode_t mode); int cifs_atomic_open(struct inode *dir, struct dentry *direntry, struct file *file, unsigned int oflags, umode_t mode); int cifs_tmpfile(struct mnt_idmap *idmap, struct inode *dir, diff --git a/fs/smb/client/dir.c b/fs/smb/client/dir.c index 88a4a1787ff0..7803bd5bd01f 100644 --- a/fs/smb/client/dir.c +++ b/fs/smb/client/dir.c @@ -645,7 +645,7 @@ out_free_xid: * hashed-positive by calling d_instantiate(). */ int cifs_create(struct mnt_idmap *idmap, struct inode *dir, - struct dentry *direntry, umode_t mode, bool excl) + struct dentry *direntry, umode_t mode) { struct cifs_sb_info *cifs_sb = CIFS_SB(dir); int rc; diff --git a/fs/smb/client/inode.c b/fs/smb/client/inode.c index 0afff761aab9..18f562ac172e 100644 --- a/fs/smb/client/inode.c +++ b/fs/smb/client/inode.c @@ -2287,6 +2287,13 @@ struct dentry *cifs_mkdir(struct mnt_idmap *idmap, struct inode *inode, const char *full_path; void *page; + /* + * vfs_mkdir() now passes S_IFDIR in @mode, but @mode is forwarded + * verbatim to the server and in the past only contained permission + * bits. Strip the type bit until SMB is verified to deal with it. + */ + mode &= ~S_IFDIR; + cifs_dbg(FYI, "In cifs_mkdir, mode = %04ho inode = 0x%p\n", mode, inode); diff --git a/fs/smb/server/mgmt/share_config.c b/fs/smb/server/mgmt/share_config.c index 6f97f8d39657..e00aee155935 100644 --- a/fs/smb/server/mgmt/share_config.c +++ b/fs/smb/server/mgmt/share_config.c @@ -9,6 +9,7 @@ #include #include #include +#include #include #include @@ -193,7 +194,8 @@ static struct ksmbd_share_config *share_config_request(struct ksmbd_work *work, goto out; } - ret = kern_path(share->path, 0, &share->vfs_path); + scoped_with_init_fs() + ret = kern_path(share->path, 0, &share->vfs_path); ksmbd_revert_fsids(work); if (ret) { ksmbd_debug(SMB, "failed to access '%s'\n", diff --git a/fs/smb/server/smb2pdu.c b/fs/smb/server/smb2pdu.c index 184501e08b29..26506effcd1b 100644 --- a/fs/smb/server/smb2pdu.c +++ b/fs/smb/server/smb2pdu.c @@ -9,6 +9,7 @@ #include #include #include +#include #include #include #include @@ -6051,7 +6052,8 @@ static int smb2_get_info_filesystem(struct ksmbd_work *work, if (!share->path) return -EIO; - rc = kern_path(share->path, LOOKUP_NO_SYMLINKS, &path); + scoped_with_init_fs() + rc = kern_path(share->path, LOOKUP_NO_SYMLINKS, &path); if (rc) { pr_err("cannot create vfs path\n"); return -EIO; diff --git a/fs/smb/server/vfs.c b/fs/smb/server/vfs.c index 6600c2f5a404..2f111b1923d7 100644 --- a/fs/smb/server/vfs.c +++ b/fs/smb/server/vfs.c @@ -7,6 +7,7 @@ #include #include #include +#include #include #include #include @@ -67,8 +68,9 @@ static int ksmbd_vfs_path_lookup(struct ksmbd_share_config *share_conf, } CLASS(filename_kernel, filename)(pathname); - err = vfs_path_parent_lookup(filename, flags, path, &last, - root_share_path); + scoped_with_init_fs() + err = vfs_path_parent_lookup(filename, flags, path, &last, + root_share_path); if (err) return err; @@ -621,7 +623,8 @@ int ksmbd_vfs_link(struct ksmbd_work *work, const char *oldname, if (ksmbd_override_fsids(work)) return -ENOMEM; - err = kern_path(oldname, LOOKUP_NO_SYMLINKS, &oldpath); + scoped_with_init_fs() + err = kern_path(oldname, LOOKUP_NO_SYMLINKS, &oldpath); if (err) { pr_err("cannot get linux path for %s, err = %d\n", oldname, err); diff --git a/fs/stat.c b/fs/stat.c index 89909746bed1..c461c3054234 100644 --- a/fs/stat.c +++ b/fs/stat.c @@ -53,7 +53,7 @@ void fill_mg_cmtime(struct kstat *stat, u32 request_mask, struct inode *inode) } stat->mtime = inode_get_mtime(inode); - stat->ctime.tv_sec = inode->i_ctime_sec; + stat->ctime.tv_sec = inode_get_ctime_sec(inode); stat->ctime.tv_nsec = (u32)atomic_read(pcn); if (!(stat->ctime.tv_nsec & I_CTIME_QUERIED)) stat->ctime.tv_nsec = ((u32)atomic_fetch_or(I_CTIME_QUERIED, pcn)); diff --git a/fs/super.c b/fs/super.c index ffdcc6a2e0de..5feecf5d9038 100644 --- a/fs/super.c +++ b/fs/super.c @@ -24,6 +24,8 @@ #include #include #include +#include +#include #include #include #include /* for the emergency remount stuff */ @@ -102,7 +104,7 @@ static bool super_flags(const struct super_block *sb, unsigned int flags) * creation will succeed and SB_BORN is set by vfs_get_tree() or we're * woken and we'll see SB_DYING. * - * The caller must have acquired a temporary reference on @sb->s_count. + * The caller must have acquired a temporary reference on @sb->s_passive. * * Return: The function returns true if SB_BORN was set and with * s_umount held. The function returns false if SB_DYING was @@ -169,6 +171,19 @@ static void super_wake(struct super_block *sb, unsigned int flag) wake_up_var(&sb->s_flags); } +/* + * The s_op->nr_cached_objects hooks (used for example by btrfs and xfs) + * operate on filesystem-global state and ignore sc->memcg. Driving them + * from per-memcg shrink_slab_memcg() invocations only burns CPU walking + * per-cpu counters and queueing duplicate work: the actual reclaim happens on + * the global path (kswapd or root direct reclaim) regardless. Restrict them + * to that path. + */ +static inline bool super_fs_objects_eligible(struct shrink_control *sc) +{ + return !sc->memcg || mem_cgroup_is_root(sc->memcg); +} + /* * One thing we have to be careful of with a per-sb shrinker is that we don't * drop the last active reference to the superblock from within the shrinker. @@ -198,7 +213,7 @@ static unsigned long super_cache_scan(struct shrinker *shrink, if (!super_trylock_shared(sb)) return SHRINK_STOP; - if (sb->s_op->nr_cached_objects) + if (sb->s_op->nr_cached_objects && super_fs_objects_eligible(sc)) fs_objects = sb->s_op->nr_cached_objects(sb, sc); inodes = list_lru_shrink_count(&sb->s_inode_lru, sc); @@ -259,7 +274,8 @@ static unsigned long super_cache_count(struct shrinker *shrink, return 0; smp_rmb(); - if (sb->s_op && sb->s_op->nr_cached_objects) + if (sb->s_op && sb->s_op->nr_cached_objects && + super_fs_objects_eligible(sc)) total_objects = sb->s_op->nr_cached_objects(sb, sc); total_objects += list_lru_shrink_count(&sb->s_dentry_lru, sc); @@ -272,6 +288,8 @@ static unsigned long super_cache_count(struct shrinker *shrink, return total_objects; } +static struct super_dev *super_dev_alloc(dev_t dev, struct super_block *sb); + static void destroy_super_work(struct work_struct *work) { struct super_block *s = container_of(work, struct super_block, @@ -279,6 +297,8 @@ static void destroy_super_work(struct work_struct *work) fsnotify_sb_free(s); security_sb_free(s); put_user_ns(s->s_user_ns); + /* Only an unregistered entry is still owned by the superblock. */ + kfree(s->s_super_dev); kfree(s->s_subtype); for (int i = 0; i < SB_FREEZE_LEVELS; i++) percpu_free_rwsem(&s->s_writers.rw_sem[i]); @@ -367,7 +387,7 @@ static struct super_block *alloc_super(struct file_system_type *type, int flags, spin_lock_init(&s->s_inode_wblist_lock); fserror_mount(s); - s->s_count = 1; + refcount_set(&s->s_passive, 1); atomic_set(&s->s_active, 1); mutex_init(&s->s_vfs_rename_mutex); lockdep_set_class(&s->s_vfs_rename_mutex, &type->s_vfs_rename_key); @@ -392,6 +412,10 @@ static struct super_block *alloc_super(struct file_system_type *type, int flags, goto fail; if (list_lru_init_memcg(&s->s_inode_lru, s->s_shrink)) goto fail; + s->s_super_dev = super_dev_alloc(0, s); + if (!s->s_super_dev) + goto fail; + s->s_min_writeback_pages = MIN_WRITEBACK_PAGES; return s; @@ -403,12 +427,17 @@ fail: /* Superblock refcounting */ /* - * Drop a superblock's refcount. The caller must hold sb_lock. + * Drop a superblock's passive reference. Must be called WITHOUT sb_lock held; + * put_super() acquires sb_lock itself when the final reference is dropped. */ -static void __put_super(struct super_block *s) +void put_super(struct super_block *s) { - if (!--s->s_count) { + if (refcount_dec_and_test(&s->s_passive)) { + + spin_lock(&sb_lock); list_del_init(&s->s_list); + spin_unlock(&sb_lock); + WARN_ON(s->s_dentry_lru.node); WARN_ON(s->s_inode_lru.node); WARN_ON(s->s_mounts); @@ -416,18 +445,109 @@ static void __put_super(struct super_block *s) } } -/** - * put_super - drop a temporary reference to superblock - * @sb: superblock in question - * - * Drops a temporary reference, frees superblock if there's no - * references left. - */ -void put_super(struct super_block *sb) +struct super_dev { + dev_t sd_dev; + struct super_block *sd_sb; + refcount_t sd_ref; + struct rhlist_head sd_node; + struct rcu_head sd_rcu; +}; + +static struct rhltable super_dev_table; +static const struct rhashtable_params super_dev_params = { + .key_len = sizeof(dev_t), + .key_offset = offsetof(struct super_dev, sd_dev), + .head_offset = offsetof(struct super_dev, sd_node), +}; + +static struct super_dev *super_dev_alloc(dev_t dev, struct super_block *sb) { - spin_lock(&sb_lock); - __put_super(sb); - spin_unlock(&sb_lock); + struct super_dev *fsd; + + fsd = kzalloc_obj(*fsd); + if (!fsd) + return NULL; + fsd->sd_dev = dev; + fsd->sd_sb = sb; + refcount_set(&fsd->sd_ref, 1); + return fsd; +} + +static void super_dev_put(struct super_dev *fsd) +{ + /* Unlink only once unpinned, so a cursor never resumes from a removed node. */ + if (fsd && refcount_dec_and_test(&fsd->sd_ref)) { + rhltable_remove(&super_dev_table, &fsd->sd_node, super_dev_params); + put_super(fsd->sd_sb); + kfree_rcu(fsd, sd_rcu); + } +} + +void __init super_dev_init(void) +{ + if (rhltable_init(&super_dev_table, &super_dev_params)) + panic("VFS: Cannot initialise super_dev_table\n"); +} + +static int super_dev_insert(struct super_dev *fsd) +{ + int err; + + err = rhltable_insert(&super_dev_table, &fsd->sd_node, super_dev_params); + if (!err) + refcount_inc(&fsd->sd_sb->s_passive); + return err; +} + +/* Register @sb under @sb->s_dev as the final fallible act of a set callback. */ +static int super_dev_register(struct super_block *sb) +{ + struct super_dev *fsd = sb->s_super_dev; + int err; + + lockdep_assert_held(&sb_lock); + VFS_WARN_ON_ONCE(!sb->s_dev); + VFS_WARN_ON_ONCE(!fsd || fsd->sd_dev); + + fsd->sd_dev = sb->s_dev; + err = super_dev_insert(fsd); + if (err) + fsd->sd_dev = 0; + return err; +} + +static struct super_dev *super_dev_get(struct rhlist_head *pos) +{ + struct super_dev *sb_dev; + + for (; pos; pos = rcu_dereference_all(pos->next)) { + sb_dev = container_of(pos, struct super_dev, sd_node); + if (refcount_inc_not_zero(&sb_dev->sd_ref)) + return sb_dev; + } + return NULL; +} + +static struct super_dev *super_dev_first(dev_t dev) +{ + struct super_dev *sb_dev; + + rcu_read_lock(); + sb_dev = super_dev_get(rhltable_lookup(&super_dev_table, &dev, super_dev_params)); + rcu_read_unlock(); + return sb_dev; +} + +static struct super_dev *super_dev_next(struct super_dev *prev) +{ + struct super_dev *sb_dev; + + rcu_read_lock(); + sb_dev = super_dev_get(rcu_dereference_all(prev->sd_node.next)); + rcu_read_unlock(); + + super_dev_put(prev); + return sb_dev; } static void kill_super_notify(struct super_block *sb) @@ -449,6 +569,12 @@ static void kill_super_notify(struct super_block *sb) hlist_del_init(&sb->s_instances); spin_unlock(&sb_lock); + /* Drop sget_fc()'s claim; a never-registered entry stays with the sb. */ + if (sb->s_super_dev->sd_dev) { + super_dev_put(sb->s_super_dev); + sb->s_super_dev = NULL; + } + /* * Let concurrent mounts know that this thing is really dead. * We don't need @sb->s_umount here as every concurrent caller @@ -478,11 +604,7 @@ void deactivate_locked_super(struct super_block *s) kill_super_notify(s); - /* - * Since list_lru_destroy() may sleep, we cannot call it from - * put_super(), where we hold the sb_lock. Therefore we destroy - * the lru lists right now. - */ + /* list_lru_destroy() may sleep; put_super() callers may not. */ list_lru_destroy(&s->s_dentry_lru); list_lru_destroy(&s->s_inode_lru); @@ -529,7 +651,7 @@ static bool grab_super(struct super_block *sb) { bool locked; - sb->s_count++; + refcount_inc(&sb->s_passive); spin_unlock(&sb_lock); locked = super_lock_excl(sb); if (locked) { @@ -556,7 +678,7 @@ static bool grab_super(struct super_block *sb) * lock held in read mode in case of success. On successful return, * the caller must drop the s_umount lock when done. * - * Note that unlike get_super() et.al. this one does *not* bump ->s_count. + * Note that unlike get_super() et.al. this one does *not* bump ->s_passive. * The reason why it's safe is that we are OK with doing trylock instead * of down_read(). There's a couple of places that are OK with that, but * it's very much not a general-purpose interface. @@ -763,6 +885,7 @@ retry: } if (!s) { spin_unlock(&sb_lock); + s = alloc_super(fc->fs_type, fc->sb_flags, user_ns); if (!s) return ERR_PTR(-ENOMEM); @@ -772,11 +895,13 @@ retry: s->s_fs_info = fc->s_fs_info; err = set(s, fc); if (err) { + VFS_WARN_ON_ONCE(s->s_super_dev->sd_dev); s->s_fs_info = NULL; spin_unlock(&sb_lock); destroy_unused_super(s); return ERR_PTR(err); } + VFS_WARN_ON_ONCE(!s->s_super_dev->sd_dev); fc->s_fs_info = NULL; s->s_type = fc->fs_type; s->s_iflags |= fc->s_iflags; @@ -851,14 +976,17 @@ static void __iterate_supers(void (*f)(struct super_block *, void *), void *arg, struct super_block *sb, *p = NULL; bool excl = flags & SUPER_ITER_EXCL; - guard(spinlock)(&sb_lock); + spin_lock(&sb_lock); for (sb = first_super(flags); !list_entry_is_head(sb, &super_blocks, s_list); sb = next_super(sb, flags)) { if (super_flags(sb, SB_DYING)) continue; - sb->s_count++; + + if (!refcount_inc_not_zero(&sb->s_passive)) + continue; + spin_unlock(&sb_lock); if (flags & SUPER_ITER_UNLOCKED) { @@ -868,13 +996,14 @@ static void __iterate_supers(void (*f)(struct super_block *, void *), void *arg, super_unlock(sb, excl); } - spin_lock(&sb_lock); if (p) - __put_super(p); + put_super(p); p = sb; + spin_lock(&sb_lock); } + spin_unlock(&sb_lock); if (p) - __put_super(p); + put_super(p); } void iterate_supers(void (*f)(struct super_block *, void *), void *arg) @@ -903,7 +1032,9 @@ void iterate_supers_type(struct file_system_type *type, if (super_flags(sb, SB_DYING)) continue; - sb->s_count++; + if (!refcount_inc_not_zero(&sb->s_passive)) + continue; + spin_unlock(&sb_lock); locked = super_lock_shared(sb); @@ -912,41 +1043,33 @@ void iterate_supers_type(struct file_system_type *type, super_unlock_shared(sb); } - spin_lock(&sb_lock); if (p) - __put_super(p); + put_super(p); p = sb; + spin_lock(&sb_lock); } - if (p) - __put_super(p); spin_unlock(&sb_lock); + if (p) + put_super(p); } EXPORT_SYMBOL(iterate_supers_type); struct super_block *user_get_super(dev_t dev, bool excl) { - struct super_block *sb; + struct super_dev *sb_dev; - spin_lock(&sb_lock); - list_for_each_entry(sb, &super_blocks, s_list) { - bool locked; + for (sb_dev = super_dev_first(dev); sb_dev; sb_dev = super_dev_next(sb_dev)) { + struct super_block *sb = sb_dev->sd_sb; - if (sb->s_dev != dev) + if (!super_lock(sb, excl)) continue; - sb->s_count++; - spin_unlock(&sb_lock); - - locked = super_lock(sb, excl); - if (locked) - return sb; - - spin_lock(&sb_lock); - __put_super(sb); - break; + /* The pinned entry holds a passive reference, take our own. */ + refcount_inc(&sb->s_passive); + super_dev_put(sb_dev); + return sb; } - spin_unlock(&sb_lock); return NULL; } @@ -1228,7 +1351,16 @@ EXPORT_SYMBOL(free_anon_bdev); int set_anon_super(struct super_block *s, void *data) { - return get_anon_bdev(&s->s_dev); + int error; + + error = get_anon_bdev(&s->s_dev); + if (error) + return error; + + error = super_dev_register(s); + if (error) + free_anon_bdev(s->s_dev); + return error; } EXPORT_SYMBOL(set_anon_super); @@ -1314,7 +1446,7 @@ EXPORT_SYMBOL(get_tree_keyed); static int set_bdev_super(struct super_block *s, void *data) { s->s_dev = *(dev_t *)data; - return 0; + return super_dev_register(s); } static int super_s_dev_set(struct super_block *s, struct fs_context *fc) @@ -1356,197 +1488,313 @@ struct super_block *sget_dev(struct fs_context *fc, dev_t dev) EXPORT_SYMBOL(sget_dev); #ifdef CONFIG_BLOCK -/* - * Lock the superblock that is holder of the bdev. Returns the superblock - * pointer if we successfully locked the superblock and it is alive. Otherwise - * we return NULL and just unlock bdev->bd_holder_lock. - * - * The function must be called with bdev->bd_holder_lock and releases it. - */ -static struct super_block *bdev_super_lock(struct block_device *bdev, bool excl) - __releases(&bdev->bd_holder_lock) +static int fs_super_freeze(struct super_block *sb) { - struct super_block *sb = bdev->bd_holder; - bool locked; + if (sb->s_op->freeze_super) + return sb->s_op->freeze_super(sb, + FREEZE_MAY_NEST | FREEZE_HOLDER_USERSPACE, NULL); + return freeze_super(sb, FREEZE_MAY_NEST | FREEZE_HOLDER_USERSPACE, NULL); +} - lockdep_assert_held(&bdev->bd_holder_lock); - lockdep_assert_not_held(&sb->s_umount); - lockdep_assert_not_held(&bdev->bd_disk->open_mutex); - - /* Make sure sb doesn't go away from under us */ - spin_lock(&sb_lock); - sb->s_count++; - spin_unlock(&sb_lock); - - mutex_unlock(&bdev->bd_holder_lock); - - locked = super_lock(sb, excl); - - /* - * If the superblock wasn't already SB_DYING then we hold - * s_umount and can safely drop our temporary reference. - */ - put_super(sb); - - if (!locked) - return NULL; - - if (!sb->s_root || !(sb->s_flags & SB_ACTIVE)) { - super_unlock(sb, excl); - return NULL; - } - - return sb; +static int fs_super_thaw(struct super_block *sb) +{ + if (sb->s_op->thaw_super) + return sb->s_op->thaw_super(sb, + FREEZE_MAY_NEST | FREEZE_HOLDER_USERSPACE, NULL); + return thaw_super(sb, FREEZE_MAY_NEST | FREEZE_HOLDER_USERSPACE, NULL); } static void fs_bdev_mark_dead(struct block_device *bdev, bool surprise) { - struct super_block *sb; + struct super_dev *sb_dev; + dev_t dev = bdev->bd_dev; - sb = bdev_super_lock(bdev, false); - if (!sb) - return; + mutex_unlock(&bdev->bd_holder_lock); - if (sb->s_op->remove_bdev) { - int ret; + for (sb_dev = super_dev_first(dev); sb_dev; sb_dev = super_dev_next(sb_dev)) { + struct super_block *sb = sb_dev->sd_sb; - ret = sb->s_op->remove_bdev(sb, bdev); - if (!ret) { - super_unlock_shared(sb); - return; + if (!super_lock_shared(sb)) + continue; + if (sb->s_root && (sb->s_flags & SB_ACTIVE)) { + if (!sb->s_op->remove_bdev || + sb->s_op->remove_bdev(sb, bdev)) { + if (!surprise) + sync_filesystem(sb); + shrink_dcache_sb(sb); + evict_inodes(sb); + if (sb->s_op->shutdown) + sb->s_op->shutdown(sb); + } } - /* Fallback to shutdown. */ + super_unlock_shared(sb); } - - if (!surprise) - sync_filesystem(sb); - shrink_dcache_sb(sb); - evict_inodes(sb); - if (sb->s_op->shutdown) - sb->s_op->shutdown(sb); - - super_unlock_shared(sb); } static void fs_bdev_sync(struct block_device *bdev) { - struct super_block *sb; + struct super_dev *sb_dev; + dev_t dev = bdev->bd_dev; - sb = bdev_super_lock(bdev, false); - if (!sb) - return; + mutex_unlock(&bdev->bd_holder_lock); - sync_filesystem(sb); - super_unlock_shared(sb); -} + for (sb_dev = super_dev_first(dev); sb_dev; sb_dev = super_dev_next(sb_dev)) { + struct super_block *sb = sb_dev->sd_sb; -static struct super_block *get_bdev_super(struct block_device *bdev) -{ - bool active = false; - struct super_block *sb; - - sb = bdev_super_lock(bdev, true); - if (sb) { - active = atomic_inc_not_zero(&sb->s_active); - super_unlock_excl(sb); + if (!super_lock_shared(sb)) + continue; + if (sb->s_root && (sb->s_flags & SB_ACTIVE)) + sync_filesystem(sb); + super_unlock_shared(sb); } - if (!active) - return NULL; - return sb; } /** - * fs_bdev_freeze - freeze owning filesystem of block device + * fs_bdev_freeze - freeze every superblock using a block device * @bdev: block device * - * Freeze the filesystem that owns this block device if it is still - * active. + * Freeze each live superblock using @bdev. A superblock owning several block + * devices is frozen once per device and stays frozen until all are thawed; the + * block layer nests these freezes so the count stays balanced. * - * A filesystem that owns multiple block devices may be frozen from each - * block device and won't be unfrozen until all block devices are - * unfrozen. Each block device can only freeze the filesystem once as we - * nest freezes for block devices in the block layer. - * - * Return: If the freeze was successful zero is returned. If the freeze - * failed a negative error code is returned. + * Return: 0, or the error from the one superblock on a single-fs device. When + * several superblocks share @bdev a per-superblock failure is swallowed + * (see below), but a sync_blockdev() failure is always reported. */ static int fs_bdev_freeze(struct block_device *bdev) { - struct super_block *sb; - int error = 0; + dev_t dev = bdev->bd_dev; + struct super_dev *sb_dev; + unsigned int count = 0; + int error = 0, err; lockdep_assert_held(&bdev->bd_fsfreeze_mutex); - sb = get_bdev_super(bdev); - if (!sb) - return -EINVAL; + mutex_unlock(&bdev->bd_holder_lock); - if (sb->s_op->freeze_super) - error = sb->s_op->freeze_super(sb, - FREEZE_MAY_NEST | FREEZE_HOLDER_USERSPACE, NULL); - else - error = freeze_super(sb, - FREEZE_MAY_NEST | FREEZE_HOLDER_USERSPACE, NULL); + for (sb_dev = super_dev_first(dev); sb_dev; sb_dev = super_dev_next(sb_dev)) { + if (!get_active_super(sb_dev->sd_sb)) + continue; + err = fs_super_freeze(sb_dev->sd_sb); + if (err && !error) + error = err; + deactivate_super(sb_dev->sd_sb); + count++; + } + + /* + * When several superblocks share the device, keep it frozen even if some + * of them failed to freeze and swallow the error: rolling the rest back + * via thaw_super() can fail too, so neither is a clear win. A single + * filesystem (count == 1) still reports its error. + */ + if (error && count > 1) + error = 0; if (!error) error = sync_blockdev(bdev); - deactivate_super(sb); return error; } /** - * fs_bdev_thaw - thaw owning filesystem of block device + * fs_bdev_thaw - thaw every superblock using a block device * @bdev: block device * - * Thaw the filesystem that owns this block device. + * The counterpart to fs_bdev_freeze(): thaw each live superblock using @bdev. + * A zero return does not imply a superblock is fully unfrozen; it may have been + * frozen more than once (by the kernel or via another device). * - * A filesystem that owns multiple block devices may be frozen from each - * block device and won't be unfrozen until all block devices are - * unfrozen. Each block device can only freeze the filesystem once as we - * nest freezes for block devices in the block layer. - * - * Return: If the thaw was successful zero is returned. If the thaw - * failed a negative error code is returned. If this function - * returns zero it doesn't mean that the filesystem is unfrozen - * as it may have been frozen multiple times (kernel may hold a - * freeze or might be frozen from other block devices). + * Return: 0, or the first error on a single-fs device; a shared device swallows + * per-superblock errors, as fs_bdev_freeze() does. */ static int fs_bdev_thaw(struct block_device *bdev) { - struct super_block *sb; - int error; + dev_t dev = bdev->bd_dev; + struct super_dev *sb_dev; + unsigned int count = 0; + int error = 0, err; lockdep_assert_held(&bdev->bd_fsfreeze_mutex); - /* - * The block device may have been frozen before it was claimed by a - * filesystem. Concurrently another process might try to mount that - * frozen block device and has temporarily claimed the block device for - * that purpose causing a concurrent fs_bdev_thaw() to end up here. The - * mounter is already about to abort mounting because they still saw an - * elevanted bdev->bd_fsfreeze_count so get_bdev_super() will return - * NULL in that case. - */ - sb = get_bdev_super(bdev); - if (!sb) - return -EINVAL; + mutex_unlock(&bdev->bd_holder_lock); - if (sb->s_op->thaw_super) - error = sb->s_op->thaw_super(sb, - FREEZE_MAY_NEST | FREEZE_HOLDER_USERSPACE, NULL); - else - error = thaw_super(sb, - FREEZE_MAY_NEST | FREEZE_HOLDER_USERSPACE, NULL); - deactivate_super(sb); + for (sb_dev = super_dev_first(dev); sb_dev; sb_dev = super_dev_next(sb_dev)) { + if (!get_active_super(sb_dev->sd_sb)) + continue; + err = fs_super_thaw(sb_dev->sd_sb); + if (err && !error) + error = err; + deactivate_super(sb_dev->sd_sb); + count++; + } + + /* Shared device: swallow per-superblock errors, like fs_bdev_freeze(). */ + if (error && count > 1) + error = 0; return error; } -const struct blk_holder_ops fs_holder_ops = { +static const struct blk_holder_ops fs_holder_ops = { .mark_dead = fs_bdev_mark_dead, .sync = fs_bdev_sync, .freeze = fs_bdev_freeze, .thaw = fs_bdev_thaw, }; -EXPORT_SYMBOL_GPL(fs_holder_ops); + +static struct super_dev *super_dev_lookup(dev_t dev, struct super_block *sb) +{ + struct super_dev *it; + struct rhlist_head *list, *pos; + + RCU_LOCKDEP_WARN(!rcu_read_lock_held(), "suspicious super_dev_lookup() usage"); + VFS_WARN_ON_ONCE(!dev); + VFS_WARN_ON_ONCE(!sb); + + list = rhltable_lookup(&super_dev_table, &dev, super_dev_params); + rhl_for_each_entry_rcu(it, pos, list, sd_node) { + if (it->sd_sb == sb) + return it; + } + + return NULL; +} + +static int fs_bdev_register(struct file *bdev_file, struct super_block *sb) +{ + struct super_dev *sb_dev __free(kfree) = NULL; + dev_t dev = file_bdev(bdev_file)->bd_dev; + int err; + + scoped_guard(rcu) { + sb_dev = super_dev_lookup(dev, sb); + if (sb_dev && refcount_inc_not_zero(&sb_dev->sd_ref)) { + retain_and_null_ptr(sb_dev); + return 0; + } + } + + sb_dev = super_dev_alloc(dev, sb); + if (!sb_dev) + return -ENOMEM; + + err = super_dev_insert(sb_dev); + if (err) + return err; + + /* Publish the entry before reading the count; pairs with bdev_freeze(). */ + smp_mb(); + if (atomic_read(&file_bdev(bdev_file)->bd_fsfreeze_count) > 0) { + err = -EBUSY; + super_dev_put(sb_dev); + } + + retain_and_null_ptr(sb_dev); + return err; +} + +/** + * fs_bdev_file_open_by_dev - claim a block device on behalf of a superblock + * @dev: block device number + * @mode: open mode + * @holder: block-layer exclusivity token (a superblock, or the file_system_type + * when the device may be shared by several superblocks of that type) + * @sb: superblock to drive fs_holder_ops events for + * + * Open @dev with &fs_holder_ops and register that @sb uses it, so device + * removal/sync/freeze/thaw are propagated to @sb (and any other superblock + * sharing @dev). Must be paired with fs_bdev_file_release(). + * + * Return: an opened block-device file or an ERR_PTR(). + */ +struct file *fs_bdev_file_open_by_dev(dev_t dev, blk_mode_t mode, void *holder, + struct super_block *sb) +{ + struct file *bdev_file; + int err; + + bdev_file = bdev_file_open_by_dev(dev, mode, holder, &fs_holder_ops); + if (IS_ERR(bdev_file)) + return bdev_file; + + err = fs_bdev_register(bdev_file, sb); + if (err) { + bdev_fput(bdev_file); + return ERR_PTR(err); + } + return bdev_file; +} +EXPORT_SYMBOL_GPL(fs_bdev_file_open_by_dev); + +/** + * fs_bdev_file_open_by_path - claim a block device on behalf of a superblock + * @path: path to the block device + * @mode: open mode + * @holder: block-layer exclusivity token (a superblock, or the file_system_type + * when the device may be shared by several superblocks of that type) + * @sb: superblock to drive fs_holder_ops events for + * + * Open the block device at @path with &fs_holder_ops and register that @sb + * uses it, so device removal/sync/freeze/thaw are propagated to @sb (and any + * other superblock sharing the device). Must be paired with + * fs_bdev_file_release(). + * + * Return: an opened block-device file or an ERR_PTR(). + */ +struct file *fs_bdev_file_open_by_path(const char *path, blk_mode_t mode, + void *holder, struct super_block *sb) +{ + struct file *bdev_file; + int err; + + bdev_file = bdev_file_open_by_path(path, mode, holder, &fs_holder_ops); + if (IS_ERR(bdev_file)) + return bdev_file; + + err = fs_bdev_register(bdev_file, sb); + if (err) { + bdev_fput(bdev_file); + return ERR_PTR(err); + } + return bdev_file; +} +EXPORT_SYMBOL_GPL(fs_bdev_file_open_by_path); + +/** + * fs_bdev_unregister - drop a superblock's claim on a block device + * @bdev_file: file returned by fs_bdev_file_open_by_{dev,path}() + * @sb: superblock the device was claimed for + * + * The inverse of fs_bdev_register(): drop one claim on the {dev, @sb} entry + * (the last claim unregisters it; a pinning cursor defers the actual unlink) + * without closing the device. A caller that must act on the still-open device + * between unregistering and closing - e.g. re-allow freezing one denied for a + * membership change - pairs this with bdev_fput(). fs_bdev_file_release() is + * the common unregister-and-close. + */ +void fs_bdev_unregister(struct file *bdev_file, struct super_block *sb) +{ + dev_t dev = file_bdev(bdev_file)->bd_dev; + struct super_dev *sb_dev; + + rcu_read_lock(); + sb_dev = super_dev_lookup(dev, sb); + rcu_read_unlock(); + super_dev_put(sb_dev); +} +EXPORT_SYMBOL_GPL(fs_bdev_unregister); + +/** + * fs_bdev_file_release - release a block device claimed for a superblock + * @bdev_file: file returned by fs_bdev_file_open_by_{dev,path}() + * @sb: superblock the device was claimed for + * + * Unregister the {dev, @sb} entry, then close the block device. + */ +void fs_bdev_file_release(struct file *bdev_file, struct super_block *sb) +{ + fs_bdev_unregister(bdev_file, sb); + bdev_fput(bdev_file); +} +EXPORT_SYMBOL_GPL(fs_bdev_file_release); int setup_bdev_super(struct super_block *sb, int sb_flags, struct fs_context *fc) @@ -1555,7 +1803,7 @@ int setup_bdev_super(struct super_block *sb, int sb_flags, struct file *bdev_file; struct block_device *bdev; - bdev_file = bdev_file_open_by_dev(sb->s_dev, mode, sb, &fs_holder_ops); + bdev_file = fs_bdev_file_open_by_dev(sb->s_dev, mode, sb, sb); if (IS_ERR(bdev_file)) { if (fc) errorf(fc, "%s: Can't open blockdev", fc->source); @@ -1569,20 +1817,19 @@ int setup_bdev_super(struct super_block *sb, int sb_flags, * writable from userspace even for a read-only block device. */ if ((mode & BLK_OPEN_WRITE) && bdev_read_only(bdev)) { - bdev_fput(bdev_file); + fs_bdev_file_release(bdev_file, sb); return -EACCES; } - /* - * It is enough to check bdev was not frozen before we set - * s_bdev as freezing will wait until SB_BORN is set. - */ + /* The sget_fc() entry is already published; pairs with bdev_freeze(). */ + smp_mb(); if (atomic_read(&bdev->bd_fsfreeze_count) > 0) { if (fc) warnf(fc, "%pg: Can't mount, blockdev is frozen", bdev); - bdev_fput(bdev_file); + fs_bdev_file_release(bdev_file, sb); return -EBUSY; } + spin_lock(&sb_lock); sb->s_bdev_file = bdev_file; sb->s_bdev = bdev; @@ -1671,7 +1918,7 @@ void kill_block_super(struct super_block *sb) generic_shutdown_super(sb); if (bdev) { sync_blockdev(bdev); - bdev_fput(sb->s_bdev_file); + fs_bdev_file_release(sb->s_bdev_file, sb); } } diff --git a/fs/ubifs/dir.c b/fs/ubifs/dir.c index 86d41e077e4d..23ec924162d6 100644 --- a/fs/ubifs/dir.c +++ b/fs/ubifs/dir.c @@ -303,7 +303,7 @@ static int ubifs_prepare_create(struct inode *dir, struct dentry *dentry, } static int ubifs_create(struct mnt_idmap *idmap, struct inode *dir, - struct dentry *dentry, umode_t mode, bool excl) + struct dentry *dentry, umode_t mode) { struct inode *inode; struct ubifs_info *c = dir->i_sb->s_fs_info; @@ -1031,7 +1031,7 @@ static struct dentry *ubifs_mkdir(struct mnt_idmap *idmap, struct inode *dir, sz_change = CALC_DENT_SIZE(fname_len(&nm)); - inode = ubifs_new_inode(c, dir, S_IFDIR | mode, false); + inode = ubifs_new_inode(c, dir, mode, false); if (IS_ERR(inode)) { err = PTR_ERR(inode); goto out_fname; diff --git a/fs/udf/dir.c b/fs/udf/dir.c index ebc9f6a379fe..425e3c162935 100644 --- a/fs/udf/dir.c +++ b/fs/udf/dir.c @@ -157,6 +157,6 @@ const struct file_operations udf_dir_operations = { .read = generic_read_dir, .iterate_shared = udf_readdir, .unlocked_ioctl = udf_ioctl, - .fsync = udf_fsync, + .fsync = simple_fsync, .setlease = generic_setlease, }; diff --git a/fs/udf/file.c b/fs/udf/file.c index f7f1422de30f..57d11606a2a7 100644 --- a/fs/udf/file.c +++ b/fs/udf/file.c @@ -198,13 +198,6 @@ static int udf_file_mmap(struct file *file, struct vm_area_struct *vma) return 0; } -int udf_fsync(struct file *file, loff_t start, loff_t end, int datasync) -{ - return mmb_fsync(file, - &UDF_I(file->f_mapping->host)->i_metadata_bhs, - start, end, datasync); -} - const struct file_operations udf_file_operations = { .read_iter = generic_file_read_iter, .unlocked_ioctl = udf_ioctl, @@ -212,7 +205,7 @@ const struct file_operations udf_file_operations = { .mmap = udf_file_mmap, .write_iter = udf_file_write_iter, .release = udf_release_file, - .fsync = udf_fsync, + .fsync = simple_fsync, .splice_read = filemap_splice_read, .splice_write = iter_file_splice_write, .llseek = generic_file_llseek, @@ -253,6 +246,8 @@ static int udf_setattr(struct mnt_idmap *idmap, struct dentry *dentry, setattr_copy(&nop_mnt_idmap, inode, attr); mark_inode_dirty(inode); + if (IS_SYNC(inode)) + sync_inode_metadata(inode, 1); return 0; } diff --git a/fs/udf/inode.c b/fs/udf/inode.c index 1f131876c345..e45e546a739a 100644 --- a/fs/udf/inode.c +++ b/fs/udf/inode.c @@ -51,8 +51,6 @@ struct udf_map_rq; static umode_t udf_convert_permissions(struct fileEntry *); -static int udf_update_inode(struct inode *, int); -static int udf_sync_inode(struct inode *inode); static int udf_alloc_i_data(struct inode *inode, size_t size); static int inode_getblk(struct inode *inode, struct udf_map_rq *map); static int udf_insert_aext(struct inode *, struct extent_position, @@ -142,7 +140,7 @@ void udf_evict_inode(struct inode *inode) if (!inode->i_nlink) { want_delete = 1; udf_setsize(inode, 0); - udf_update_inode(inode, IS_SYNC(inode)); + sync_inode_metadata(inode, IS_SYNC(inode)); } if (iinfo->i_alloc_type != ICBTAG_FLAG_AD_IN_ICB && inode->i_size != iinfo->i_lenExtents) { @@ -947,10 +945,7 @@ static int inode_getblk(struct inode *inode, struct udf_map_rq *map) iinfo->i_next_alloc_goal = newblocknum + 1; inode_set_ctime_current(inode); - if (IS_SYNC(inode)) - udf_sync_inode(inode); - else - mark_inode_dirty(inode); + mark_inode_dirty(inode); ret = 0; out_free: brelse(prev_epos.bh); @@ -1337,10 +1332,7 @@ set_size: } update_time: inode_set_mtime_to_ts(inode, inode_set_ctime_current(inode)); - if (IS_SYNC(inode)) - udf_sync_inode(inode); - else - mark_inode_dirty(inode); + mark_inode_dirty(inode); return err; } @@ -1721,14 +1713,28 @@ void udf_update_extra_perms(struct inode *inode, umode_t mode) iinfo->i_extraPerms |= FE_PERM_O_DELETE; } -int udf_write_inode(struct inode *inode, struct writeback_control *wbc) +int udf_sync_inode_metadata(struct inode *inode, struct writeback_control *wbc) { - return udf_update_inode(inode, wbc->sync_mode == WB_SYNC_ALL); -} + struct buffer_head *bh; + int err = 0; -static int udf_sync_inode(struct inode *inode) -{ - return udf_update_inode(inode, 1); + bh = sb_getblk(inode->i_sb, + udf_get_lb_pblock(inode->i_sb, + &UDF_I(inode)->i_location, 0)); + if (!bh) + return -EIO; + + sync_dirty_buffer(bh); + if (buffer_write_io_error(bh)) { + udf_warn(inode->i_sb, "IO error syncing udf inode [%08llx]\n", + inode->i_ino); + err = -EIO; + goto out; + } + err = mmb_sync(&UDF_I(inode)->i_metadata_bhs); +out: + brelse(bh); + return err; } static void udf_adjust_time(struct udf_inode_info *iinfo, struct timespec64 time) @@ -1739,7 +1745,7 @@ static void udf_adjust_time(struct udf_inode_info *iinfo, struct timespec64 time iinfo->i_crtime = time; } -static int udf_update_inode(struct inode *inode, int do_sync) +int udf_write_inode(struct inode *inode, struct writeback_control *wbc) { struct buffer_head *bh = NULL; struct fileEntry *fe; @@ -1748,7 +1754,6 @@ static int udf_update_inode(struct inode *inode, int do_sync) uint32_t udfperms; uint16_t icbflags; uint16_t crclen; - int err = 0; struct udf_sb_info *sbi = UDF_SB(inode->i_sb); unsigned char blocksize_bits = inode->i_sb->s_blocksize_bits; struct udf_inode_info *iinfo = UDF_I(inode); @@ -1953,17 +1958,10 @@ finish: /* write the data blocks */ mark_buffer_dirty(bh); - if (do_sync) { - sync_dirty_buffer(bh); - if (buffer_write_io_error(bh)) { - udf_warn(inode->i_sb, "IO error syncing udf inode [%08llx]\n", - inode->i_ino); - err = -EIO; - } - } brelse(bh); + set_inode_metadata_writeback(inode); - return err; + return 0; } struct inode *__udf_iget(struct super_block *sb, struct kernel_lb_addr *ino, diff --git a/fs/udf/namei.c b/fs/udf/namei.c index 9a3b7cef3606..b90841ac0a40 100644 --- a/fs/udf/namei.c +++ b/fs/udf/namei.c @@ -371,7 +371,7 @@ static int udf_add_nondir(struct dentry *dentry, struct inode *inode) } static int udf_create(struct mnt_idmap *idmap, struct inode *dir, - struct dentry *dentry, umode_t mode, bool excl) + struct dentry *dentry, umode_t mode) { struct inode *inode = udf_new_inode(dir, mode); @@ -428,7 +428,7 @@ static struct dentry *udf_mkdir(struct mnt_idmap *idmap, struct inode *dir, struct udf_inode_info *dinfo = UDF_I(dir); struct udf_inode_info *iinfo; - inode = udf_new_inode(dir, S_IFDIR | mode); + inode = udf_new_inode(dir, mode); if (IS_ERR(inode)) return ERR_CAST(inode); diff --git a/fs/udf/super.c b/fs/udf/super.c index 9686078bba64..2ba5973ef4dd 100644 --- a/fs/udf/super.c +++ b/fs/udf/super.c @@ -211,6 +211,7 @@ static const struct super_operations udf_sb_ops = { .alloc_inode = udf_alloc_inode, .free_inode = udf_free_in_core_inode, .write_inode = udf_write_inode, + .sync_inode_metadata = udf_sync_inode_metadata, .evict_inode = udf_evict_inode, .put_super = udf_put_super, .sync_fs = udf_sync_fs, diff --git a/fs/udf/udfdecl.h b/fs/udf/udfdecl.h index 21de6925fc68..7d5a1981434e 100644 --- a/fs/udf/udfdecl.h +++ b/fs/udf/udfdecl.h @@ -137,7 +137,6 @@ static inline unsigned int udf_dir_entry_len(struct fileIdentDesc *cfi) /* file.c */ extern long udf_ioctl(struct file *, unsigned int, unsigned long); -int udf_fsync(struct file *file, loff_t start, loff_t end, int datasync); /* inode.c */ extern struct inode *__udf_iget(struct super_block *, struct kernel_lb_addr *, @@ -158,6 +157,7 @@ extern struct buffer_head *udf_bread(struct inode *inode, udf_pblk_t block, extern int udf_setsize(struct inode *, loff_t); extern void udf_evict_inode(struct inode *); extern int udf_write_inode(struct inode *, struct writeback_control *wbc); +int udf_sync_inode_metadata(struct inode *, struct writeback_control *wbc); extern int inode_bmap(struct inode *inode, sector_t block, struct extent_position *pos, struct kernel_lb_addr *eloc, uint32_t *elen, sector_t *offset, int8_t *etype); diff --git a/fs/ufs/namei.c b/fs/ufs/namei.c index 5b3c85c93242..6703f3bcf76f 100644 --- a/fs/ufs/namei.c +++ b/fs/ufs/namei.c @@ -70,8 +70,7 @@ static struct dentry *ufs_lookup(struct inode * dir, struct dentry *dentry, unsi * with d_instantiate(). */ static int ufs_create (struct mnt_idmap * idmap, - struct inode * dir, struct dentry * dentry, umode_t mode, - bool excl) + struct inode * dir, struct dentry * dentry, umode_t mode) { struct inode *inode; @@ -174,7 +173,7 @@ static struct dentry *ufs_mkdir(struct mnt_idmap * idmap, struct inode * dir, inode_inc_link_count(dir); - inode = ufs_new_inode(dir, S_IFDIR|mode); + inode = ufs_new_inode(dir, mode); err = PTR_ERR(inode); if (IS_ERR(inode)) goto out_dir; diff --git a/fs/ufs/super.c b/fs/ufs/super.c index c4831a8b9b3f..6dcf6d048cce 100644 --- a/fs/ufs/super.c +++ b/fs/ufs/super.c @@ -672,7 +672,7 @@ void ufs_mark_sb_dirty(struct super_block *sb) spin_lock(&sbi->work_lock); if (!sbi->work_queued) { delay = msecs_to_jiffies(dirty_writeback_interval * 10); - queue_delayed_work(system_long_wq, &sbi->sync_work, delay); + queue_delayed_work(system_dfl_long_wq, &sbi->sync_work, delay); sbi->work_queued = 1; } spin_unlock(&sbi->work_lock); diff --git a/fs/vboxsf/dir.c b/fs/vboxsf/dir.c index c5bd3271aa96..0b9eab157432 100644 --- a/fs/vboxsf/dir.c +++ b/fs/vboxsf/dir.c @@ -298,9 +298,9 @@ out: static int vboxsf_dir_mkfile(struct mnt_idmap *idmap, struct inode *parent, struct dentry *dentry, - umode_t mode, bool excl) + umode_t mode) { - return vboxsf_dir_create(parent, dentry, mode, false, excl, NULL); + return vboxsf_dir_create(parent, dentry, mode, false, true, NULL); } static struct dentry *vboxsf_dir_mkdir(struct mnt_idmap *idmap, diff --git a/fs/xfs/xfs_buf.c b/fs/xfs/xfs_buf.c index 48d7dfd3e15f..17b9d643e1a8 100644 --- a/fs/xfs/xfs_buf.c +++ b/fs/xfs/xfs_buf.c @@ -80,7 +80,7 @@ xfs_buf_stale( spin_lock(&bp->b_lockref.lock); atomic_set(&bp->b_lru_ref, 0); - if (!__lockref_is_dead(&bp->b_lockref)) + if (!lockref_is_dead(&bp->b_lockref)) list_lru_del_obj(&bp->b_target->bt_lru, &bp->b_lru); spin_unlock(&bp->b_lockref.lock); } @@ -841,7 +841,7 @@ static void xfs_buf_destroy( struct xfs_buf *bp) { - ASSERT(__lockref_is_dead(&bp->b_lockref)); + ASSERT(lockref_is_dead(&bp->b_lockref)); ASSERT(!(bp->b_flags & _XBF_DELWRI_Q)); if (bp->b_pag) @@ -1631,7 +1631,7 @@ xfs_free_buftarg( fs_put_dax(btp->bt_daxdev, btp->bt_mount); /* the main block device is closed by kill_block_super */ if (btp->bt_bdev != btp->bt_mount->m_super->s_bdev) - bdev_fput(btp->bt_file); + fs_bdev_file_release(btp->bt_file, btp->bt_mount->m_super); kfree(btp); } diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c index 0ade13b31335..cc78924eb1cd 100644 --- a/fs/xfs/xfs_file.c +++ b/fs/xfs/xfs_file.c @@ -251,8 +251,6 @@ xfs_file_dio_read( struct iov_iter *to) { struct xfs_inode *ip = XFS_I(file_inode(iocb->ki_filp)); - unsigned int dio_flags = 0; - const struct iomap_dio_ops *dio_ops = NULL; ssize_t ret; trace_xfs_file_direct_read(iocb, to); @@ -266,11 +264,15 @@ xfs_file_dio_read( if (ret) return ret; if (mapping_stable_writes(iocb->ki_filp->f_mapping)) { - dio_ops = &xfs_dio_read_bounce_ops; - dio_flags |= IOMAP_DIO_BOUNCE; + ret = iomap_dio_rw(iocb, to, &xfs_read_iomap_ops, + &xfs_dio_read_bounce_ops, IOMAP_DIO_BOUNCE, + NULL, 0); + } else { + ret = iomap_dio_read_simple(iocb, to, xfs_read_iomap_begin); + if (ret == -ENOTBLK) + ret = iomap_dio_rw(iocb, to, &xfs_read_iomap_ops, NULL, + 0, NULL, 0); } - ret = iomap_dio_rw(iocb, to, &xfs_read_iomap_ops, dio_ops, dio_flags, - NULL, 0); xfs_iunlock(ip, XFS_IOLOCK_SHARED); return ret; @@ -857,9 +859,9 @@ retry: NULL, 0); /* - * The retry mechanism is based on the ->iomap_begin method returning + * The retry mechanism is based on the ->iomap_next method returning * -ENOPROTOOPT, which would be when the REQ_ATOMIC-based write is not - * possible. The REQ_ATOMIC-based method typically not be possible if + * possible. The REQ_ATOMIC-based method is typically not possible if * the write spans multiple extents or the disk blocks are misaligned. */ if (ret == -ENOPROTOOPT && dops == &xfs_direct_write_iomap_ops) { diff --git a/fs/xfs/xfs_iomap.c b/fs/xfs/xfs_iomap.c index 225c3de88d03..71c45be8c652 100644 --- a/fs/xfs/xfs_iomap.c +++ b/fs/xfs/xfs_iomap.c @@ -1037,8 +1037,11 @@ out_unlock: return error; } +static DEFINE_IOMAP_ITER_NEXT(xfs_direct_write_iomap_next, + xfs_direct_write_iomap_begin); + const struct iomap_ops xfs_direct_write_iomap_ops = { - .iomap_begin = xfs_direct_write_iomap_begin, + .iomap_next = xfs_direct_write_iomap_next, }; #ifdef CONFIG_XFS_RT @@ -1089,8 +1092,11 @@ xfs_zoned_direct_write_iomap_begin( return 0; } +static DEFINE_IOMAP_ITER_NEXT(xfs_zoned_direct_write_iomap_next, + xfs_zoned_direct_write_iomap_begin); + const struct iomap_ops xfs_zoned_direct_write_iomap_ops = { - .iomap_begin = xfs_zoned_direct_write_iomap_begin, + .iomap_next = xfs_zoned_direct_write_iomap_next, }; #endif /* CONFIG_XFS_RT */ @@ -1274,8 +1280,11 @@ out_unlock: return error; } +static DEFINE_IOMAP_ITER_NEXT(xfs_atomic_write_cow_iomap_next, + xfs_atomic_write_cow_iomap_begin); + const struct iomap_ops xfs_atomic_write_cow_iomap_ops = { - .iomap_begin = xfs_atomic_write_cow_iomap_begin, + .iomap_next = xfs_atomic_write_cow_iomap_next, }; static int @@ -1298,9 +1307,11 @@ xfs_dax_write_iomap_end( return xfs_reflink_end_cow(ip, pos, written); } +static DEFINE_IOMAP_ITER_NEXT_END(xfs_dax_write_iomap_next, + xfs_direct_write_iomap_begin, xfs_dax_write_iomap_end); + const struct iomap_ops xfs_dax_write_iomap_ops = { - .iomap_begin = xfs_direct_write_iomap_begin, - .iomap_end = xfs_dax_write_iomap_end, + .iomap_next = xfs_dax_write_iomap_next, }; /* @@ -2168,12 +2179,14 @@ xfs_buffered_write_iomap_end( return 0; } +static DEFINE_IOMAP_ITER_NEXT_END(xfs_buffered_write_iomap_next, + xfs_buffered_write_iomap_begin, xfs_buffered_write_iomap_end); + const struct iomap_ops xfs_buffered_write_iomap_ops = { - .iomap_begin = xfs_buffered_write_iomap_begin, - .iomap_end = xfs_buffered_write_iomap_end, + .iomap_next = xfs_buffered_write_iomap_next, }; -static int +int xfs_read_iomap_begin( struct inode *inode, loff_t offset, @@ -2214,8 +2227,10 @@ xfs_read_iomap_begin( shared ? IOMAP_F_SHARED : 0, seq); } +static DEFINE_IOMAP_ITER_NEXT(xfs_read_iomap_next, xfs_read_iomap_begin); + const struct iomap_ops xfs_read_iomap_ops = { - .iomap_begin = xfs_read_iomap_begin, + .iomap_next = xfs_read_iomap_next, }; static int @@ -2302,8 +2317,10 @@ out_unlock: return error; } +static DEFINE_IOMAP_ITER_NEXT(xfs_seek_iomap_next, xfs_seek_iomap_begin); + const struct iomap_ops xfs_seek_iomap_ops = { - .iomap_begin = xfs_seek_iomap_begin, + .iomap_next = xfs_seek_iomap_next, }; static int @@ -2349,8 +2366,10 @@ out_unlock: return xfs_bmbt_to_iomap(ip, iomap, &imap, flags, IOMAP_F_XATTR, seq); } +static DEFINE_IOMAP_ITER_NEXT(xfs_xattr_iomap_next, xfs_xattr_iomap_begin); + const struct iomap_ops xfs_xattr_iomap_ops = { - .iomap_begin = xfs_xattr_iomap_begin, + .iomap_next = xfs_xattr_iomap_next, }; int diff --git a/fs/xfs/xfs_iomap.h b/fs/xfs/xfs_iomap.h index ebcce7d49446..cffcec532ea6 100644 --- a/fs/xfs/xfs_iomap.h +++ b/fs/xfs/xfs_iomap.h @@ -49,6 +49,10 @@ xfs_aligned_fsb_count( return count_fsb; } +int xfs_read_iomap_begin(struct inode *inode, loff_t offset, + loff_t length, unsigned flags, struct iomap *iomap, + struct iomap *srcmap); + extern const struct iomap_ops xfs_buffered_write_iomap_ops; extern const struct iomap_ops xfs_direct_write_iomap_ops; extern const struct iomap_ops xfs_zoned_direct_write_iomap_ops; diff --git a/fs/xfs/xfs_iops.c b/fs/xfs/xfs_iops.c index 6339f4956ecb..4a3299abf774 100644 --- a/fs/xfs/xfs_iops.c +++ b/fs/xfs/xfs_iops.c @@ -293,8 +293,7 @@ xfs_vn_create( struct mnt_idmap *idmap, struct inode *dir, struct dentry *dentry, - umode_t mode, - bool flags) + umode_t mode) { return xfs_generic_create(idmap, dir, dentry, mode, 0, NULL); } @@ -306,7 +305,7 @@ xfs_vn_mkdir( struct dentry *dentry, umode_t mode) { - return ERR_PTR(xfs_generic_create(idmap, dir, dentry, mode | S_IFDIR, 0, NULL)); + return ERR_PTR(xfs_generic_create(idmap, dir, dentry, mode, 0, NULL)); } STATIC struct dentry * @@ -338,7 +337,7 @@ STATIC struct dentry * xfs_vn_ci_lookup( struct inode *dir, struct dentry *dentry, - unsigned int flags) + unsigned int flags) { struct xfs_inode *ip; struct xfs_name xname; diff --git a/fs/xfs/xfs_qm.c b/fs/xfs/xfs_qm.c index 896b24f87ac9..99a82107b8e6 100644 --- a/fs/xfs/xfs_qm.c +++ b/fs/xfs/xfs_qm.c @@ -128,7 +128,7 @@ xfs_qm_dqpurge( struct xfs_quotainfo *qi = dqp->q_mount->m_quotainfo; spin_lock(&dqp->q_lockref.lock); - if (dqp->q_lockref.count > 0 || __lockref_is_dead(&dqp->q_lockref)) { + if (dqp->q_lockref.count > 0 || lockref_is_dead(&dqp->q_lockref)) { spin_unlock(&dqp->q_lockref.lock); return -EAGAIN; } @@ -429,7 +429,7 @@ xfs_qm_dquot_isolate( * from the LRU, leave it for the freeing task to complete the freeing * process rather than risk it being free from under us here. */ - if (__lockref_is_dead(&dqp->q_lockref)) + if (lockref_is_dead(&dqp->q_lockref)) goto out_miss_unlock; /* diff --git a/fs/xfs/xfs_super.c b/fs/xfs/xfs_super.c index 63c4bcbe6c2b..4b2eeb7783f7 100644 --- a/fs/xfs/xfs_super.c +++ b/fs/xfs/xfs_super.c @@ -400,8 +400,8 @@ xfs_blkdev_get( blk_mode_t mode; mode = sb_open_mode(mp->m_super->s_flags); - *bdev_filep = bdev_file_open_by_path(name, mode, - mp->m_super, &fs_holder_ops); + *bdev_filep = fs_bdev_file_open_by_path(name, mode, + mp->m_super, mp->m_super); if (IS_ERR(*bdev_filep)) { error = PTR_ERR(*bdev_filep); *bdev_filep = NULL; @@ -526,7 +526,7 @@ xfs_open_devices( mp->m_logdev_targp = mp->m_ddev_targp; /* Handle won't be used, drop it */ if (logdev_file) - bdev_fput(logdev_file); + fs_bdev_file_release(logdev_file, mp->m_super); } return 0; @@ -541,10 +541,10 @@ xfs_open_devices( mp->m_ddev_targp = NULL; out_close_rtdev: if (rtdev_file) - bdev_fput(rtdev_file); + fs_bdev_file_release(rtdev_file, mp->m_super); out_close_logdev: if (logdev_file) - bdev_fput(logdev_file); + fs_bdev_file_release(logdev_file, mp->m_super); return error; } diff --git a/fs/zonefs/file.c b/fs/zonefs/file.c index 5ada33f70bb4..5b34849be7a2 100644 --- a/fs/zonefs/file.c +++ b/fs/zonefs/file.c @@ -57,8 +57,10 @@ static int zonefs_read_iomap_begin(struct inode *inode, loff_t offset, return 0; } +static DEFINE_IOMAP_ITER_NEXT(zonefs_read_iomap_next, zonefs_read_iomap_begin); + static const struct iomap_ops zonefs_read_iomap_ops = { - .iomap_begin = zonefs_read_iomap_begin, + .iomap_next = zonefs_read_iomap_next, }; static int zonefs_write_iomap_begin(struct inode *inode, loff_t offset, @@ -106,8 +108,11 @@ static int zonefs_write_iomap_begin(struct inode *inode, loff_t offset, return 0; } +static DEFINE_IOMAP_ITER_NEXT(zonefs_write_iomap_next, + zonefs_write_iomap_begin); + static const struct iomap_ops zonefs_write_iomap_ops = { - .iomap_begin = zonefs_write_iomap_begin, + .iomap_next = zonefs_write_iomap_next, }; static int zonefs_read_folio(struct file *unused, struct folio *folio) diff --git a/include/linux/binfmt_misc.h b/include/linux/binfmt_misc.h new file mode 100644 index 000000000000..072e4b3dd78d --- /dev/null +++ b/include/linux/binfmt_misc.h @@ -0,0 +1,110 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _LINUX_BINFMT_MISC_H +#define _LINUX_BINFMT_MISC_H + +#include + +struct bpf_prog; +struct file; +struct linux_binprm; +struct user_namespace; + +#define BINFMT_MISC_OPS_NAME_MAX 16 + +/* Longest name a 'B' entry can bind an interpreter under. */ +#define BINFMT_MISC_INTERP_NAME_MAX 32 + +/* Most interpreters one entry can bind. */ +#define BINFMT_MISC_INTERP_MAX 100 + +/** + * struct binfmt_misc_interp - an interpreter an entry was registered with + * @list: link in the entry's list, in registration order + * @file: the file, opened at registration and never resolved again + * @path: the path it was registered under, used as the name the interpreter + * runs under; stored after @name in the same allocation + * @name: the name the load program selects it by; empty for the fixed + * interpreter of a static 'F' entry + * + * Owned by the entry and living exactly as long as it does. The list head + * is handed to the handler's load program for the duration of one exec, + * which picks one with bpf_binprm_select_interp(). + */ +struct binfmt_misc_interp { + struct list_head list; + struct file *file; + const char *path; + char name[]; +}; + +const struct binfmt_misc_interp * +binfmt_misc_find_interp(const struct list_head *interps, const char *name); + +/** + * enum bpf_binprm_flags - per-exec invocation flags a load program can request + * @BPF_BINPRM_PRESERVE_ARGV0: keep the caller's argv[0] (like the 'P' flag) + * @BPF_BINPRM_CREDENTIALS: compute credentials from the binary; implies execfd + * (like the 'C' flag) + * @BPF_BINPRM_EXECFD: pass the binary via AT_EXECFD (like the 'O' flag) + * @BPF_BINPRM_TRANSPARENT: leave argv untouched, the interpreter takes the + * binary from AT_EXECFD (like the 'T' flag); implies + * execfd, excludes preserve-argv0 + * @BPF_BINPRM_LOADER: substitute the interpreter for the binary's PT_INTERP + * and run the binary as a native exec (like the 'L' + * flag); excludes every other flag + * + * Set from a load program with bpf_binprm_set_flags(). Unlike a static entry, + * a bpf handler chooses these per exec rather than once at registration. + */ +enum bpf_binprm_flags { + BPF_BINPRM_PRESERVE_ARGV0 = (1ULL << 0), + BPF_BINPRM_CREDENTIALS = (1ULL << 1), + BPF_BINPRM_EXECFD = (1ULL << 2), + BPF_BINPRM_TRANSPARENT = (1ULL << 3), + BPF_BINPRM_LOADER = (1ULL << 4), +}; + +/** + * struct binfmt_misc_ops - bpf-backed binary type handler + * @match: decide whether the handler applies to @bprm; consulted from the + * entry lookup walk like static magic and extension matching, in + * registration order with first-match-wins semantics; sleepable, + * so it can read the binary to decide, but the verifier rejects + * the interpreter selection kfuncs in it + * @load: select an interpreter for the matched @bprm via + * bpf_binprm_set_interp(), or one the entry bound via + * bpf_binprm_select_interp(), and return zero; a match is + * committed, so a failure fails the exec instead of falling + * through to later entries; -ENOEXEC does not fail the exec but + * moves on to the remaining binary formats + * @name: name that 'B' entries reference the handler by + */ +struct binfmt_misc_ops { + bool (*match)(struct linux_binprm *bprm); + int (*load)(struct linux_binprm *bprm); + char name[BINFMT_MISC_OPS_NAME_MAX]; +}; + +#ifdef CONFIG_BINFMT_MISC_BPF +const struct binfmt_misc_ops *binfmt_misc_get_ops(struct user_namespace *user_ns, + const char *name); +void binfmt_misc_put_ops(const struct binfmt_misc_ops *ops); +bool bpf_prog_is_binfmt_misc_ops(const struct bpf_prog *prog); +#else +static inline const struct binfmt_misc_ops * +binfmt_misc_get_ops(struct user_namespace *user_ns, const char *name) +{ + return NULL; +} + +static inline void binfmt_misc_put_ops(const struct binfmt_misc_ops *ops) +{ +} + +static inline bool bpf_prog_is_binfmt_misc_ops(const struct bpf_prog *prog) +{ + return false; +} +#endif /* CONFIG_BINFMT_MISC_BPF */ + +#endif /* _LINUX_BINFMT_MISC_H */ diff --git a/include/linux/binfmts.h b/include/linux/binfmts.h index 2c77e383e737..f686a37f7a0a 100644 --- a/include/linux/binfmts.h +++ b/include/linux/binfmts.h @@ -12,6 +12,16 @@ struct coredump_params; #define CORENAME_MAX_SIZE 128 +/* Interpreter selection staged by a bpf binfmt_misc handler. */ +struct binfmt_misc_bpf { + /* interpreters the matched entry bound, selectable by name */ + const struct list_head *bpf_interps; + const char *bpf_interp; /* interpreter selected by a bpf handler */ + struct file *bpf_interp_file; /* the bound interpreter it selected */ + const char *bpf_interp_arg; /* interpreter argument from a bpf handler */ + u64 bpf_flags; /* enum bpf_binprm_flags from a bpf handler */ +}; + /* * This structure is used to hold the arguments that are used when loading binaries. */ @@ -55,6 +65,7 @@ struct linux_binprm { is_check:1; struct file *executable; /* Executable to pass to the interpreter */ struct file *interpreter; + struct file *loader; struct file *file; struct cred *cred; /* new credentials */ int unsafe; /* how unsafe this exec is (mask of LSM_UNSAFE_*) */ @@ -65,6 +76,7 @@ struct linux_binprm { of the time same as filename, but could be different for binfmt_{misc,script} */ const char *fdpath; /* generated filename for execveat */ + struct binfmt_misc_bpf; /* bpf handler interpreter selection */ unsigned interp_flags; int execfd; /* File descriptor of the executable */ unsigned long exec; @@ -85,6 +97,28 @@ struct linux_binprm { #define BINPRM_FLAGS_PRESERVE_ARGV0_BIT 3 #define BINPRM_FLAGS_PRESERVE_ARGV0 (1 << BINPRM_FLAGS_PRESERVE_ARGV0_BIT) +/* binfmt_misc dispatched to the interpreter transparently */ +#define BINPRM_FLAGS_TRANSPARENT_INTERP_BIT 4 +#define BINPRM_FLAGS_TRANSPARENT_INTERP (1 << BINPRM_FLAGS_TRANSPARENT_INTERP_BIT) + +/** + * bprm_at_flags - the AT_FLAGS this invocation implies + * @bprm: binary that is being executed + * + * Tell the program on the receiving end which dispatch contract it got. + * + * Return: the AT_FLAGS value for this exec + */ +static inline unsigned long bprm_at_flags(const struct linux_binprm *bprm) +{ + /* Transparency preserves the whole argv, argv[0] included. */ + if (bprm->interp_flags & BINPRM_FLAGS_TRANSPARENT_INTERP) + return AT_FLAGS_TRANSPARENT_INTERP; + if (bprm->interp_flags & BINPRM_FLAGS_PRESERVE_ARGV0) + return AT_FLAGS_PRESERVE_ARGV0; + return 0; +} + /* * This structure defines the functions that are used to load the binary formats that * linux accepts. @@ -101,8 +135,8 @@ struct linux_binfmt { #if IS_ENABLED(CONFIG_BINFMT_MISC) struct binfmt_misc { - struct list_head entries; - rwlock_t entries_lock; + struct hlist_head entries; + spinlock_t entries_lock; bool enabled; } __randomize_layout; @@ -129,6 +163,8 @@ extern int begin_new_exec(struct linux_binprm * bprm); extern void setup_new_exec(struct linux_binprm * bprm); extern void finalize_exec(struct linux_binprm *bprm); extern void would_dump(struct linux_binprm *, struct file *); +struct file *bprm_open_interpreter(struct linux_binprm *bprm, const char *path); +void bprm_drop_loader(struct linux_binprm *bprm); extern int suid_dumpable; diff --git a/include/linux/blk_types.h b/include/linux/blk_types.h index 8808ee76e73c..5a725a0cd35f 100644 --- a/include/linux/blk_types.h +++ b/include/linux/blk_types.h @@ -66,7 +66,7 @@ struct block_device { int bd_holders; struct kobject *bd_holder_dir; - atomic_t bd_fsfreeze_count; /* number of freeze requests */ + atomic_t bd_fsfreeze_count; /* >0 freeze requests, <0 freeze deniers */ struct mutex bd_fsfreeze_mutex; /* serialize freeze/thaw */ struct partition_meta_info *bd_meta_info; diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index 9213a5716f95..dbb549cdfb77 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -126,8 +126,6 @@ struct blk_integrity { unsigned char pi_tuple_size; }; -typedef unsigned int __bitwise blk_mode_t; - /* open for reading */ #define BLK_OPEN_READ ((__force blk_mode_t)(1 << 0)) /* open for writing */ @@ -1770,13 +1768,6 @@ struct blk_holder_ops { __releases(&bdev->bd_holder_lock); }; -/* - * For filesystems using @fs_holder_ops, the @holder argument passed to - * helpers used to open and claim block devices via - * bd_prepare_to_claim() must point to a superblock. - */ -extern const struct blk_holder_ops fs_holder_ops; - /* * Return the correct open flags for blkdev_get_by_* for super block flags * as stored in sb->s_flags. @@ -1837,7 +1828,10 @@ static inline int early_lookup_bdev(const char *pathname, dev_t *dev) int bdev_freeze(struct block_device *bdev); int bdev_thaw(struct block_device *bdev); +int bdev_deny_freeze(struct block_device *bdev); +void bdev_allow_freeze(struct block_device *bdev); void bdev_fput(struct file *bdev_file); +void bdev_yield_claim(struct file *bdev_file); struct io_comp_batch { struct rq_list req_list; diff --git a/include/linux/buffer_head.h b/include/linux/buffer_head.h index 8b23bc9a244c..fd2c7115c054 100644 --- a/include/linux/buffer_head.h +++ b/include/linux/buffer_head.h @@ -210,10 +210,6 @@ void bh_end_async_write(struct bio *bio); /* Things to do with metadata buffers list */ void mmb_mark_buffer_dirty(struct buffer_head *bh, struct mapping_metadata_bhs *mmb); -int mmb_fsync_noflush(struct file *file, struct mapping_metadata_bhs *mmb, - loff_t start, loff_t end, bool datasync); -int mmb_fsync(struct file *file, struct mapping_metadata_bhs *mmb, - loff_t start, loff_t end, bool datasync); void clean_bdev_aliases(struct block_device *bdev, sector_t block, sector_t len); static inline void clean_bdev_bh_alias(struct buffer_head *bh) diff --git a/include/linux/efs_vh.h b/include/linux/efs_vh.h deleted file mode 100644 index 206c5270f7b8..000000000000 --- a/include/linux/efs_vh.h +++ /dev/null @@ -1,54 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 */ -/* - * efs_vh.h - * - * Copyright (c) 1999 Al Smith - * - * Portions derived from IRIX header files (c) 1985 MIPS Computer Systems, Inc. - */ - -#ifndef __EFS_VH_H__ -#define __EFS_VH_H__ - -#define VHMAGIC 0xbe5a941 /* volume header magic number */ -#define NPARTAB 16 /* 16 unix partitions */ -#define NVDIR 15 /* max of 15 directory entries */ -#define BFNAMESIZE 16 /* max 16 chars in boot file name */ -#define VDNAMESIZE 8 - -struct volume_directory { - char vd_name[VDNAMESIZE]; /* name */ - __be32 vd_lbn; /* logical block number */ - __be32 vd_nbytes; /* file length in bytes */ -}; - -struct partition_table { /* one per logical partition */ - __be32 pt_nblks; /* # of logical blks in partition */ - __be32 pt_firstlbn; /* first lbn of partition */ - __be32 pt_type; /* use of partition */ -}; - -struct volume_header { - __be32 vh_magic; /* identifies volume header */ - __be16 vh_rootpt; /* root partition number */ - __be16 vh_swappt; /* swap partition number */ - char vh_bootfile[BFNAMESIZE]; /* name of file to boot */ - char pad[48]; /* device param space */ - struct volume_directory vh_vd[NVDIR]; /* other vol hdr contents */ - struct partition_table vh_pt[NPARTAB]; /* device partition layout */ - __be32 vh_csum; /* volume header checksum */ - __be32 vh_fill; /* fill out to 512 bytes */ -}; - -/* partition type sysv is used for EFS format CD-ROM partitions */ -#define SGI_SYSV 0x05 -#define SGI_EFS 0x07 -#define IS_EFS(x) (((x) == SGI_EFS) || ((x) == SGI_SYSV)) - -struct pt_types { - int pt_type; - char *pt_name; -}; - -#endif /* __EFS_VH_H__ */ - diff --git a/include/linux/fs.h b/include/linux/fs.h index aa1d501d2bb6..072d8cd09a0b 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -740,7 +740,8 @@ enum inode_state_flags_enum { I_CREATING = (1U << 15), I_DONTCACHE = (1U << 16), I_SYNC_QUEUED = (1U << 17), - I_PINNING_NETFS_WB = (1U << 18) + I_PINNING_NETFS_WB = (1U << 18), + I_METADATA_WRITEBACK = (1U << 19), }; #define I_DIRTY_INODE (I_DIRTY_SYNC | I_DIRTY_DATASYNC) @@ -1598,12 +1599,12 @@ struct timespec64 inode_set_ctime_deleg(struct inode *inode, static inline time64_t inode_get_atime_sec(const struct inode *inode) { - return inode->i_atime_sec; + return READ_ONCE(inode->i_atime_sec); } static inline long inode_get_atime_nsec(const struct inode *inode) { - return inode->i_atime_nsec; + return READ_ONCE(inode->i_atime_nsec); } static inline struct timespec64 inode_get_atime(const struct inode *inode) @@ -1617,8 +1618,8 @@ static inline struct timespec64 inode_get_atime(const struct inode *inode) static inline struct timespec64 inode_set_atime_to_ts(struct inode *inode, struct timespec64 ts) { - inode->i_atime_sec = ts.tv_sec; - inode->i_atime_nsec = ts.tv_nsec; + WRITE_ONCE(inode->i_atime_sec, ts.tv_sec); + WRITE_ONCE(inode->i_atime_nsec, ts.tv_nsec); return ts; } @@ -1633,12 +1634,12 @@ static inline struct timespec64 inode_set_atime(struct inode *inode, static inline time64_t inode_get_mtime_sec(const struct inode *inode) { - return inode->i_mtime_sec; + return READ_ONCE(inode->i_mtime_sec); } static inline long inode_get_mtime_nsec(const struct inode *inode) { - return inode->i_mtime_nsec; + return READ_ONCE(inode->i_mtime_nsec); } static inline struct timespec64 inode_get_mtime(const struct inode *inode) @@ -1651,8 +1652,8 @@ static inline struct timespec64 inode_get_mtime(const struct inode *inode) static inline struct timespec64 inode_set_mtime_to_ts(struct inode *inode, struct timespec64 ts) { - inode->i_mtime_sec = ts.tv_sec; - inode->i_mtime_nsec = ts.tv_nsec; + WRITE_ONCE(inode->i_mtime_sec, ts.tv_sec); + WRITE_ONCE(inode->i_mtime_nsec, ts.tv_nsec); return ts; } @@ -1677,12 +1678,12 @@ static inline struct timespec64 inode_set_mtime(struct inode *inode, static inline time64_t inode_get_ctime_sec(const struct inode *inode) { - return inode->i_ctime_sec; + return READ_ONCE(inode->i_ctime_sec); } static inline long inode_get_ctime_nsec(const struct inode *inode) { - return inode->i_ctime_nsec & ~I_CTIME_QUERIED; + return READ_ONCE(inode->i_ctime_nsec) & ~I_CTIME_QUERIED; } static inline struct timespec64 inode_get_ctime(const struct inode *inode) @@ -1916,8 +1917,6 @@ struct dir_context { struct io_uring_cmd; struct offset_ctx; -typedef unsigned int __bitwise fop_flags_t; - struct file_operations { struct module *owner; fop_flags_t fop_flags; @@ -2002,7 +2001,7 @@ struct inode_operations { int (*readlink) (struct dentry *, char __user *,int); int (*create) (struct mnt_idmap *, struct inode *,struct dentry *, - umode_t, bool); + umode_t); int (*link) (struct dentry *,struct inode *,struct dentry *); int (*unlink) (struct inode *,struct dentry *); int (*symlink) (struct mnt_idmap *, struct inode *,struct dentry *, @@ -2213,6 +2212,13 @@ static inline void mark_inode_dirty_sync(struct inode *inode) __mark_inode_dirty(inode, I_DIRTY_SYNC); } +static inline void set_inode_metadata_writeback(struct inode *inode) +{ + spin_lock(&inode->i_lock); + inode_state_set(inode, I_METADATA_WRITEBACK); + spin_unlock(&inode->i_lock); +} + /* * returns the refcount on the inode. it can change arbitrarily. */ diff --git a/include/linux/fs/super.h b/include/linux/fs/super.h index 405612678115..733d439f01ed 100644 --- a/include/linux/fs/super.h +++ b/include/linux/fs/super.h @@ -237,4 +237,12 @@ int thaw_super(struct super_block *super, enum freeze_holder who, int sb_init_dio_done_wq(struct super_block *sb); +struct file; +struct file *fs_bdev_file_open_by_dev(dev_t dev, blk_mode_t mode, void *holder, + struct super_block *sb); +struct file *fs_bdev_file_open_by_path(const char *path, blk_mode_t mode, + void *holder, struct super_block *sb); +void fs_bdev_unregister(struct file *bdev_file, struct super_block *sb); +void fs_bdev_file_release(struct file *bdev_file, struct super_block *sb); + #endif /* _LINUX_FS_SUPER_H */ diff --git a/include/linux/fs/super_types.h b/include/linux/fs/super_types.h index 3bdd7f7fb9e5..ecd96aeb1cee 100644 --- a/include/linux/fs/super_types.h +++ b/include/linux/fs/super_types.h @@ -30,6 +30,7 @@ struct mount; struct mtd_info; struct quotactl_ops; struct shrinker; +struct super_dev; struct unicode_map; struct user_namespace; struct workqueue_struct; @@ -86,6 +87,8 @@ struct super_operations { void (*free_inode)(struct inode *inode); void (*dirty_inode)(struct inode *inode, int flags); int (*write_inode)(struct inode *inode, struct writeback_control *wbc); + int (*sync_inode_metadata)(struct inode *inode, + struct writeback_control *wbc); int (*drop_inode)(struct inode *inode); void (*evict_inode)(struct inode *inode); void (*put_super)(struct super_block *sb); @@ -132,6 +135,7 @@ struct super_operations { struct super_block { struct list_head s_list; /* Keep this first */ dev_t s_dev; /* search index; _not_ kdev_t */ + struct super_dev *s_super_dev; /* sget_fc()'s device table claim */ unsigned char s_blocksize_bits; unsigned long s_blocksize; loff_t s_maxbytes; /* Max file size */ @@ -145,7 +149,7 @@ struct super_block { unsigned long s_magic; struct dentry *s_root; struct rw_semaphore s_umount; - int s_count; + refcount_t s_passive; atomic_t s_active; #ifdef CONFIG_SECURITY void *s_security; diff --git a/include/linux/fs_struct.h b/include/linux/fs_struct.h index 0070764b790a..97eef8d3863d 100644 --- a/include/linux/fs_struct.h +++ b/include/linux/fs_struct.h @@ -6,6 +6,7 @@ #include #include #include +#include struct fs_struct { int users; @@ -16,6 +17,7 @@ struct fs_struct { } __randomize_layout; extern struct kmem_cache *fs_cachep; +extern struct fs_struct *userspace_init_fs; extern void exit_fs(struct task_struct *); extern void set_fs_root(struct fs_struct *, const struct path *); @@ -40,6 +42,8 @@ static inline void get_fs_pwd(struct fs_struct *fs, struct path *pwd) read_sequnlock_excl(&fs->seq); } +struct fs_struct *switch_fs_struct(struct fs_struct *new_fs); + extern bool current_chrooted(void); static inline int current_umask(void) @@ -47,4 +51,34 @@ static inline int current_umask(void) return current->fs->umask; } +/* + * Temporarily use userspace_init_fs for path resolution in kthreads. + * Callers should use scoped_with_init_fs() which automatically + * restores the original fs_struct at scope exit. + */ +static inline struct fs_struct *__override_init_fs(void) +{ + struct fs_struct *old_fs; + + old_fs = current->fs; + WRITE_ONCE(current->fs, userspace_init_fs); + return old_fs; +} + +static inline void __revert_init_fs(struct fs_struct *old_fs) +{ + VFS_WARN_ON_ONCE(current->fs != userspace_init_fs); + WRITE_ONCE(current->fs, old_fs); +} + +DEFINE_CLASS(__override_init_fs, + struct fs_struct *, + __revert_init_fs(_T), + __override_init_fs(), void) + +#define scoped_with_init_fs() \ + scoped_class(__override_init_fs, __UNIQUE_ID(label)) + +void __init init_userspace_fs(void); + #endif /* _LINUX_FS_STRUCT_H */ diff --git a/include/linux/init_task.h b/include/linux/init_task.h index a6cb241ea00c..61536be773f5 100644 --- a/include/linux/init_task.h +++ b/include/linux/init_task.h @@ -24,6 +24,7 @@ extern struct files_struct init_files; extern struct fs_struct init_fs; +extern struct fs_struct *userspace_init_fs; extern struct nsproxy init_nsproxy; #ifndef CONFIG_VIRT_CPU_ACCOUNTING_NATIVE diff --git a/include/linux/iomap.h b/include/linux/iomap.h index 56b43d594e6e..8c754eb974fb 100644 --- a/include/linux/iomap.h +++ b/include/linux/iomap.h @@ -10,6 +10,7 @@ #include #include #include +#include struct address_space; struct fiemap_extent_info; @@ -212,24 +213,36 @@ struct iomap_write_ops { #define IOMAP_ATOMIC (1 << 9) /* torn-write protection */ #define IOMAP_DONTCACHE (1 << 10) -struct iomap_ops { - /* - * Return the existing mapping at pos, or reserve space starting at - * pos for up to length, as long as we can do it as a single mapping. - * The actual length is returned in iomap->length. - */ - int (*iomap_begin)(struct inode *inode, loff_t pos, loff_t length, - unsigned flags, struct iomap *iomap, - struct iomap *srcmap); +/* + * Return the existing mapping at pos, or reserve space starting at pos for up + * to length, as long as we can do it as a single mapping. + * The actual length is returned in iomap->length. + */ +typedef int (*iomap_iter_begin_fn)(struct inode *inode, loff_t pos, + loff_t length, unsigned flags, struct iomap *iomap, + struct iomap *srcmap); - /* - * Commit and/or unreserve space previous allocated using iomap_begin. - * Written indicates the length of the successful write operation which - * needs to be commited, while the rest needs to be unreserved. - * Written might be zero if no data was written. - */ - int (*iomap_end)(struct inode *inode, loff_t pos, loff_t length, - ssize_t written, unsigned flags, struct iomap *iomap); +/* + * Commit and/or unreserve space previously allocated by iomap_iter_begin_fn. + * Written indicates the length of the successful write operation which needs + * to be committed, while the rest needs to be unreserved. + * Written might be zero if no data was written. + */ +typedef int (*iomap_iter_end_fn)(struct inode *inode, loff_t pos, loff_t length, + ssize_t written, unsigned flags, struct iomap *iomap); + +/* + * Produce the next mapping (finishing the previous one if needed). + * Return 1 to continue iterating, 0 if the range is fully consumed, or a + * negative error on failure. + */ +typedef int (*iomap_iter_next_fn)(const struct iomap_iter *iter, + struct iomap *iomap, struct iomap *srcmap); + +struct iomap_ops { + iomap_iter_begin_fn iomap_begin; + iomap_iter_end_fn iomap_end; + iomap_iter_next_fn iomap_next; }; /** @@ -317,6 +330,71 @@ static inline const struct iomap *iomap_iter_srcmap(const struct iomap_iter *i) return &i->iomap; } +int iomap_iter_continue(const struct iomap_iter *iter, struct iomap *iomap, + struct iomap *srcmap, int ret); + +/** + * iomap_iter_next - finish the previous mapping and produce the next one + * @iter: iteration structure + * @iomap: mapping to finish and then repopulate + * @srcmap: source mapping to finish and then repopulate + * @begin: callback that produces a mapping for the current position + * @end: optional callback that finishes the previous mapping, or NULL + * + * Inline helper that implements the common body of an ->iomap_next() + * callback: it finishes the previous mapping via @end (if present), decides + * via iomap_iter_continue() whether to keep going, and obtains the next + * mapping via @begin. + * + * This helper is marked __always_inline so that when a caller passes + * compile-time-constant @begin and @end callbacks, the compiler can call them + * directly, avoiding the indirect-call overhead. + * + * Returns 1 to continue iterating, 0 once the range is fully consumed, or a + * negative errno on error. + */ +static __always_inline int iomap_iter_next(const struct iomap_iter *iter, + struct iomap *iomap, struct iomap *srcmap, + iomap_iter_begin_fn begin, iomap_iter_end_fn end) +{ + int ret = 0; + + if (iomap->length) { + if (end) { + /* + * Calculate how far the iter was advanced and the + * original length bytes for end(). + */ + ssize_t advanced = iter->pos - iter->iter_start_pos; + loff_t len; + + len = iomap_length_trim(iter, iter->iter_start_pos, + iter->len + advanced); + + ret = end(iter->inode, iter->iter_start_pos, len, + advanced, iter->flags, iomap); + } + ret = iomap_iter_continue(iter, iomap, srcmap, ret); + if (ret <= 0) + return ret; + } + + ret = begin(iter->inode, iter->pos, iter->len, iter->flags, iomap, + srcmap); + + return ret < 0 ? ret : 1; +} + +#define DEFINE_IOMAP_ITER_NEXT_END(name, begin_fn, end_fn) \ +int name(const struct iomap_iter *iter, struct iomap *iomap, \ + struct iomap *srcmap) \ +{ \ + return iomap_iter_next(iter, iomap, srcmap, begin_fn, end_fn); \ +} + +#define DEFINE_IOMAP_ITER_NEXT(name, begin_fn) \ + DEFINE_IOMAP_ITER_NEXT_END(name, begin_fn, NULL) + /* * Return the file offset for the first unchanged block after a short write. * @@ -365,6 +443,7 @@ struct folio *iomap_get_folio(struct iomap_iter *iter, loff_t pos, size_t len); bool iomap_release_folio(struct folio *folio, gfp_t gfp_flags); void iomap_invalidate_folio(struct folio *folio, size_t offset, size_t len); bool iomap_dirty_folio(struct address_space *mapping, struct folio *folio); +void iomap_folio_mark_uptodate(struct folio *folio); int iomap_file_unshare(struct inode *inode, loff_t pos, loff_t len, const struct iomap_ops *ops, const struct iomap_write_ops *write_ops); @@ -606,6 +685,71 @@ struct iomap_dio *__iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter, ssize_t iomap_dio_complete(struct iomap_dio *dio); void iomap_dio_bio_end_io(struct bio *bio); +/* + * Fast path for small, block-aligned direct I/Os that map to a single + * contiguous on-disk extent. + * + * @iter must describe a non-empty READ no larger than the inode block size: + * writes, zero-length I/O, and larger requests need the generic iomap direct + * I/O path. + * + * Does not support iomap_dio_ops, dio_flags, done_before or private data. + * The range must also stay within i_size and encrypted inodes must use the + * generic iomap direct I/O path. + * + * -ENOTBLK indicates the generic path must be used by the caller instead. + * Any other errno is a real result and is propagated as-is, in particular + * -EAGAIN for IOCB_NOWAIT must reach the caller. + * + * The caller can only provide an iomap begin handler, and the iterator + * is never advanced. + */ +ssize_t __iomap_dio_read_simple(struct kiocb *iocb, struct iov_iter *iter, + struct iomap_iter *iomi); +static __always_inline ssize_t iomap_dio_read_simple(struct kiocb *iocb, + struct iov_iter *iter, iomap_iter_begin_fn begin) +{ + struct iomap_iter iomi = { + .inode = file_inode(iocb->ki_filp), + .pos = iocb->ki_pos, + .len = iov_iter_count(iter), + .flags = IOMAP_DIRECT, + }; + ssize_t ret; + + if (!iomi.len) + return 0; + + /* + * Simple dio is an optimization for small IO. Filter out large IO + * early as it's the most common case to fail for typical direct IO + * workloads. + */ + if (iomi.len > iomi.inode->i_sb->s_blocksize) + return -ENOTBLK; + if (iocb->ki_pos + iomi.len > i_size_read(iomi.inode)) + return -ENOTBLK; + if (IS_ENCRYPTED(iomi.inode)) + return -ENOTBLK; + + ret = kiocb_write_and_wait(iocb, iomi.len); + if (ret) + return ret; + + if (iocb->ki_flags & IOCB_NOWAIT) + iomi.flags |= IOMAP_NOWAIT; + + inode_dio_begin(iomi.inode); + ret = begin(iomi.inode, iomi.pos, iomi.len, iomi.flags, &iomi.iomap, + &iomi.srcmap); + if (ret) { + inode_dio_end(iomi.inode); + return ret; + } + + return __iomap_dio_read_simple(iocb, iter, &iomi); +} + #ifdef CONFIG_SWAP struct file; struct swap_info_struct; diff --git a/include/linux/lockref.h b/include/linux/lockref.h index 6ded24cdb4a8..ddfb7d3b8cec 100644 --- a/include/linux/lockref.h +++ b/include/linux/lockref.h @@ -34,6 +34,8 @@ struct lockref { }; }; +#define __LOCKREF_DEAD_VAL -128 + /** * lockref_init - Initialize a lockref * @lockref: pointer to lockref structure @@ -55,9 +57,15 @@ void lockref_mark_dead(struct lockref *lockref); bool lockref_get_not_dead(struct lockref *lockref); /* Must be called under spinlock for reliable results */ -static inline bool __lockref_is_dead(const struct lockref *l) +static inline bool lockref_is_dead(const struct lockref *l) { - return ((int)l->count < 0); + return (READ_ONCE(l->count) == __LOCKREF_DEAD_VAL); +} + +static inline bool lockref_is_dead_or_zero(const struct lockref *l) +{ + int count = READ_ONCE(l->count); + return (count == __LOCKREF_DEAD_VAL || count == 0); } #endif /* __LINUX_LOCKREF_H */ diff --git a/include/linux/namei.h b/include/linux/namei.h index ebe6e29f7e93..86d657b24fc6 100644 --- a/include/linux/namei.h +++ b/include/linux/namei.h @@ -97,6 +97,9 @@ struct dentry *start_creating_dentry(struct dentry *parent, struct dentry *start_removing_dentry(struct dentry *parent, struct dentry *child); +struct file *vfs_lookup_open(struct path *parent, struct qstr *last, + int open_flag, umode_t mode); + /* end_creating - finish action started with start_creating * @child: dentry returned by start_creating() or vfs_mkdir() * diff --git a/include/linux/net.h b/include/linux/net.h index f268f395ce47..fdcf9956805c 100644 --- a/include/linux/net.h +++ b/include/linux/net.h @@ -285,6 +285,7 @@ int sock_recvmsg(struct socket *sock, struct msghdr *msg, int flags); struct file *sock_alloc_file(struct socket *sock, int flags, const char *dname); struct socket *sockfd_lookup(int fd, int *err); struct socket *sock_from_file(struct file *file); +int sock_read_xattr(struct socket *sock, const char *name, void *value, size_t size); #define sockfd_put(sock) fput(sock->file) int net_ratelimit(void); diff --git a/include/linux/pipe_fs_i.h b/include/linux/pipe_fs_i.h index 7f6a92ac9704..9e8b60ad806a 100644 --- a/include/linux/pipe_fs_i.h +++ b/include/linux/pipe_fs_i.h @@ -14,6 +14,9 @@ #define PIPE_BUF_FLAG_LOSS 0x40 /* Message loss happened after this buffer */ #endif +#define PIPE_PREALLOC_MAX 8 /* max pages in prealloc pool */ +#define PIPE_PREALLOC_KEEP 2 /* keep at least this many after trim */ + /** * struct pipe_buffer - a linux kernel pipe buffer * @page: the page containing the data for the pipe buffer @@ -58,6 +61,21 @@ union pipe_index { }; }; +/** + * struct anon_pipe_prealloc - per-pipe page preallocation pool + * @pages: array of cached pages (pool) + * @count: number of pages currently in the pool + * + * Each pipe keeps a small bounded pool of preallocated pages to reduce + * allocation overhead during writes. The pool is bounded at PIPE_PREALLOC_MAX + * and trimmed down to PIPE_PREALLOC_KEEP after a write completes. + */ +struct anon_pipe_prealloc { + struct page *pages[PIPE_PREALLOC_MAX]; + + unsigned int __data_racy count; +}; + /** * struct pipe_inode_info - a linux kernel pipe * @mutex: mutex protecting the whole thing @@ -68,7 +86,7 @@ union pipe_index { * @max_usage: The maximum number of slots that may be used in the ring * @ring_size: total number of buffers (should be a power of 2) * @nr_accounted: The amount this pipe accounts for in user->pipe_bufs - * @tmp_page: cached released page + * @prealloc: per-pipe page preallocation pool * @readers: number of current readers of this pipe * @writers: number of current writers of this pipe * @files: number of struct file referring this pipe (protected by ->i_lock) @@ -99,7 +117,7 @@ struct pipe_inode_info { #ifdef CONFIG_WATCH_QUEUE bool note_loss; #endif - struct page *tmp_page[2]; + struct anon_pipe_prealloc prealloc; struct fasync_struct *fasync_readers; struct fasync_struct *fasync_writers; struct pipe_buffer *bufs; diff --git a/include/linux/sched.h b/include/linux/sched.h index 373bcc0598d1..1e4136c2b2a3 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -1191,6 +1191,7 @@ struct task_struct { unsigned long last_switch_time; #endif /* Filesystem information: */ + struct fs_struct *real_fs; struct fs_struct *fs; /* Open file information: */ diff --git a/include/linux/sched/task.h b/include/linux/sched/task.h index 41ed884cffc9..e0c1ca8c6a18 100644 --- a/include/linux/sched/task.h +++ b/include/linux/sched/task.h @@ -31,6 +31,7 @@ struct kernel_clone_args { u32 io_thread:1; u32 user_worker:1; u32 no_files:1; + u32 umh:1; unsigned long stack; unsigned long stack_size; unsigned long tls; diff --git a/include/linux/seq_file.h b/include/linux/seq_file.h index 2fb266ea69fa..dc0e8c62d9e0 100644 --- a/include/linux/seq_file.h +++ b/include/linux/seq_file.h @@ -104,7 +104,7 @@ static inline void seq_setwidth(struct seq_file *m, size_t size) } void seq_pad(struct seq_file *m, char c); -char *mangle_path(char *s, const char *p, const char *esc); +char *seq_mangle_path(char *s, const char *p, const char *esc); int seq_open(struct file *, const struct seq_operations *); ssize_t seq_read(struct file *, char __user *, size_t, loff_t *); ssize_t seq_read_iter(struct kiocb *iocb, struct iov_iter *iter); diff --git a/include/linux/syscalls.h b/include/linux/syscalls.h index 874d9067a43b..8413b624ad47 100644 --- a/include/linux/syscalls.h +++ b/include/linux/syscalls.h @@ -457,6 +457,7 @@ asmlinkage long sys_faccessat2(int dfd, const char __user *filename, int mode, asmlinkage long sys_chdir(const char __user *filename); asmlinkage long sys_fchdir(unsigned int fd); asmlinkage long sys_chroot(const char __user *filename); +asmlinkage long sys_fchroot(int fd, unsigned int flags); asmlinkage long sys_fchmod(unsigned int fd, umode_t mode); asmlinkage long sys_fchmodat(int dfd, const char __user *filename, umode_t mode); diff --git a/include/linux/types.h b/include/linux/types.h index 93166b0b0617..bc5dda2a3d86 100644 --- a/include/linux/types.h +++ b/include/linux/types.h @@ -163,6 +163,8 @@ typedef u32 dma_addr_t; typedef unsigned int __bitwise gfp_t; typedef unsigned int __bitwise slab_flags_t; typedef unsigned int __bitwise fmode_t; +typedef unsigned int __bitwise blk_mode_t; +typedef unsigned int __bitwise fop_flags_t; #ifdef CONFIG_PHYS_ADDR_T_64BIT typedef u64 phys_addr_t; diff --git a/include/uapi/asm-generic/errno.h b/include/uapi/asm-generic/errno.h index bd78e69e0a43..c84ebf89c8b6 100644 --- a/include/uapi/asm-generic/errno.h +++ b/include/uapi/asm-generic/errno.h @@ -76,7 +76,7 @@ #define ENOPROTOOPT 92 /* Protocol not available */ #define EPROTONOSUPPORT 93 /* Protocol not supported */ #define ESOCKTNOSUPPORT 94 /* Socket type not supported */ -#define EOPNOTSUPP 95 /* Operation not supported on transport endpoint */ +#define EOPNOTSUPP 95 /* Operation not supported */ #define EPFNOSUPPORT 96 /* Protocol family not supported */ #define EAFNOSUPPORT 97 /* Address family not supported by protocol */ #define EADDRINUSE 98 /* Address already in use */ diff --git a/include/uapi/asm-generic/unistd.h b/include/uapi/asm-generic/unistd.h index a627acc8fb5f..5b7e77a7c736 100644 --- a/include/uapi/asm-generic/unistd.h +++ b/include/uapi/asm-generic/unistd.h @@ -863,8 +863,12 @@ __SYSCALL(__NR_listns, sys_listns) #define __NR_rseq_slice_yield 471 __SYSCALL(__NR_rseq_slice_yield, sys_rseq_slice_yield) +/* fs/open.c */ +#define __NR_fchroot 472 +__SYSCALL(__NR_fchroot, sys_fchroot) + #undef __NR_syscalls -#define __NR_syscalls 472 +#define __NR_syscalls 473 /* * 32 bit systems traditionally used different diff --git a/include/uapi/linux/binfmts.h b/include/uapi/linux/binfmts.h index c6f9450efc12..aafc07d78b80 100644 --- a/include/uapi/linux/binfmts.h +++ b/include/uapi/linux/binfmts.h @@ -22,4 +22,11 @@ struct pt_regs; #define AT_FLAGS_PRESERVE_ARGV0_BIT 0 #define AT_FLAGS_PRESERVE_ARGV0 (1 << AT_FLAGS_PRESERVE_ARGV0_BIT) +/* + * The interpreter runs transparently: the argument vector and the exe + * link belong to the binary passed in AT_EXECFD. + */ +#define AT_FLAGS_TRANSPARENT_INTERP_BIT 1 +#define AT_FLAGS_TRANSPARENT_INTERP (1 << AT_FLAGS_TRANSPARENT_INTERP_BIT) + #endif /* _UAPI_LINUX_BINFMTS_H */ diff --git a/include/uapi/linux/efs_fs_sb.h b/include/uapi/linux/efs_fs_sb.h deleted file mode 100644 index 6bad29a10faa..000000000000 --- a/include/uapi/linux/efs_fs_sb.h +++ /dev/null @@ -1,63 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ -/* - * efs_fs_sb.h - * - * Copyright (c) 1999 Al Smith - * - * Portions derived from IRIX header files (c) 1988 Silicon Graphics - */ - -#ifndef __EFS_FS_SB_H__ -#define __EFS_FS_SB_H__ - -#include -#include - -/* EFS superblock magic numbers */ -#define EFS_MAGIC 0x072959 -#define EFS_NEWMAGIC 0x07295a - -#define IS_EFS_MAGIC(x) ((x == EFS_MAGIC) || (x == EFS_NEWMAGIC)) - -#define EFS_SUPER 1 -#define EFS_ROOTINODE 2 - -/* efs superblock on disk */ -struct efs_super { - __be32 fs_size; /* size of filesystem, in sectors */ - __be32 fs_firstcg; /* bb offset to first cg */ - __be32 fs_cgfsize; /* size of cylinder group in bb's */ - __be16 fs_cgisize; /* bb's of inodes per cylinder group */ - __be16 fs_sectors; /* sectors per track */ - __be16 fs_heads; /* heads per cylinder */ - __be16 fs_ncg; /* # of cylinder groups in filesystem */ - __be16 fs_dirty; /* fs needs to be fsck'd */ - __be32 fs_time; /* last super-block update */ - __be32 fs_magic; /* magic number */ - char fs_fname[6]; /* file system name */ - char fs_fpack[6]; /* file system pack name */ - __be32 fs_bmsize; /* size of bitmap in bytes */ - __be32 fs_tfree; /* total free data blocks */ - __be32 fs_tinode; /* total free inodes */ - __be32 fs_bmblock; /* bitmap location. */ - __be32 fs_replsb; /* Location of replicated superblock. */ - __be32 fs_lastialloc; /* last allocated inode */ - char fs_spare[20]; /* space for expansion - MUST BE ZERO */ - __be32 fs_checksum; /* checksum of volume portion of fs */ -}; - -/* efs superblock information in memory */ -struct efs_sb_info { - __u32 fs_magic; /* superblock magic number */ - __u32 fs_start; /* first block of filesystem */ - __u32 first_block; /* first data block in filesystem */ - __u32 total_blocks; /* total number of blocks in filesystem */ - __u32 group_size; /* # of blocks a group consists of */ - __u32 data_free; /* # of free data blocks */ - __u32 inode_free; /* # of free inodes */ - __u16 inode_blocks; /* # of blocks used for inodes in every grp */ - __u16 total_groups; /* # of groups */ -}; - -#endif /* __EFS_FS_SB_H__ */ - diff --git a/include/uapi/linux/fcntl.h b/include/uapi/linux/fcntl.h index aadfbf6e0cb3..e43e3de3e9ee 100644 --- a/include/uapi/linux/fcntl.h +++ b/include/uapi/linux/fcntl.h @@ -124,6 +124,7 @@ struct delegation { #define FD_PIDFS_ROOT -10002 /* Root of the pidfs filesystem */ #define FD_NSFS_ROOT -10003 /* Root of the nsfs filesystem */ +#define FD_FAILFS_ROOT -10004 /* Root of the failfs filesystem */ #define FD_INVALID -10009 /* Invalid file descriptor: -10000 - EBADF = -10009 */ /* Generic flags for the *at(2) family of syscalls. */ diff --git a/include/uapi/linux/magic.h b/include/uapi/linux/magic.h index 4f2da935a76c..fd5f0e95648e 100644 --- a/include/uapi/linux/magic.h +++ b/include/uapi/linux/magic.h @@ -105,5 +105,6 @@ #define PID_FS_MAGIC 0x50494446 /* "PIDF" */ #define GUEST_MEMFD_MAGIC 0x474d454d /* "GMEM" */ #define NULL_FS_MAGIC 0x4E554C4C /* "NULL" */ +#define FAIL_FS_MAGIC 0x4641494C /* "FAIL" */ #endif /* __LINUX_MAGIC_H__ */ diff --git a/include/uapi/linux/nsfs.h b/include/uapi/linux/nsfs.h index a25e38d1c874..007fed5971b4 100644 --- a/include/uapi/linux/nsfs.h +++ b/include/uapi/linux/nsfs.h @@ -96,9 +96,10 @@ enum ns_type { * struct ns_id_req - namespace ID request structure * @size: size of this structure * @spare: reserved for future use - * @filter: filter mask - * @ns_id: last namespace id - * @user_ns_id: owning user namespace ID + * @ns_id: last namespace ID + * @ns_type: bit mask of namespace types to include + * @spare2: reserved for future use + * @user_ns_id: filter on this user namespace ID (or 0) * * Structure for passing namespace ID and miscellaneous parameters to * statns(2) and listns(2). diff --git a/init/init_task.c b/init/init_task.c index b67ef6040a65..ba5c2523f7e0 100644 --- a/init/init_task.c +++ b/init/init_task.c @@ -162,6 +162,7 @@ struct task_struct init_task __aligned(L1_CACHE_BYTES) = { RCU_POINTER_INITIALIZER(cred, &init_cred), .comm = INIT_TASK_COMM, .thread = INIT_THREAD, + .real_fs = &init_fs, .fs = &init_fs, .files = &init_files, #ifdef CONFIG_IO_URING diff --git a/init/initramfs.c b/init/initramfs.c index 20a18fcda48e..3cee8b50ad82 100644 --- a/init/initramfs.c +++ b/init/initramfs.c @@ -6,6 +6,7 @@ #include #include #include +#include #include #include #include @@ -618,7 +619,7 @@ void __init reserve_initrd_mem(void) phys_addr_t start; unsigned long size; - /* Ignore the virtul address computed during device tree parsing */ + /* Ignore the virtual address computed during device tree parsing */ initrd_start = initrd_end = 0; if (!phys_initrd_size) @@ -716,7 +717,7 @@ static void __init populate_initrd_image(char *err) } #endif /* CONFIG_BLK_DEV_RAM */ -static void __init do_populate_rootfs(void *unused, async_cookie_t cookie) +static void __init unpack_initramfs(async_cookie_t cookie) { /* Load the built in initramfs */ char *err = unpack_to_rootfs(__initramfs_start, __initramfs_size); @@ -724,7 +725,7 @@ static void __init do_populate_rootfs(void *unused, async_cookie_t cookie) panic_show_mem("%s", err); /* Failed to decompress INTERNAL initramfs */ if (!initrd_start || IS_ENABLED(CONFIG_INITRAMFS_FORCE)) - goto done; + return; if (IS_ENABLED(CONFIG_BLK_DEV_RAM)) printk(KERN_INFO "Trying to unpack rootfs image as initramfs...\n"); @@ -739,9 +740,14 @@ static void __init do_populate_rootfs(void *unused, async_cookie_t cookie) printk(KERN_EMERG "Initramfs unpacking failed: %s\n", err); #endif } +} -done: - security_initramfs_populated(); +static void __init do_populate_rootfs(void *unused, async_cookie_t cookie) +{ + scoped_with_init_fs() { + unpack_initramfs(cookie); + security_initramfs_populated(); + } /* * If the initrd region is overlapped with crashkernel reserved region, diff --git a/init/initramfs_test.c b/init/initramfs_test.c index bc55306d226d..9cf316c13ffa 100644 --- a/init/initramfs_test.c +++ b/init/initramfs_test.c @@ -3,6 +3,7 @@ #include #include #include +#include #include #include #include @@ -562,7 +563,7 @@ static struct kunit_case __refdata initramfs_test_cases[] = { {}, }; -static int __init initramfs_test_init(struct kunit_suite *suite) +static int __init initramfs_suite_init(struct kunit_suite *suite) { /* * unpack_to_rootfs() uses module-static state (victim, byte_count, @@ -574,9 +575,23 @@ static int __init initramfs_test_init(struct kunit_suite *suite) return 0; } +/* Tests run in a nullfs kthread; always use the init fs for path resolution. */ +static int __init initramfs_test_init(struct kunit *test) +{ + test->priv = __override_init_fs(); + return 0; +} + +static void __init initramfs_test_exit(struct kunit *test) +{ + __revert_init_fs(test->priv); +} + static struct kunit_suite __refdata initramfs_test_suite = { .name = "initramfs", - .suite_init = initramfs_test_init, + .suite_init = initramfs_suite_init, + .init = initramfs_test_init, + .exit = initramfs_test_exit, .test_cases = initramfs_test_cases, }; kunit_test_init_section_suites(&initramfs_test_suite); diff --git a/init/main.c b/init/main.c index e363232b428b..92d34e496a33 100644 --- a/init/main.c +++ b/init/main.c @@ -103,6 +103,7 @@ #include #include #include +#include #include #include #include @@ -670,6 +671,11 @@ static __initdata DECLARE_COMPLETION(kthreadd_done); static noinline void __ref __noreturn rest_init(void) { + struct kernel_clone_args init_args = { + .flags = (CLONE_VM | CLONE_UNTRACED), + .fn = kernel_init, + .fn_arg = NULL, + }; struct task_struct *tsk; int pid; @@ -679,7 +685,7 @@ static noinline void __ref __noreturn rest_init(void) * the init task will end up wanting to create kthreads, which, if * we schedule it before we create kthreadd, will OOPS. */ - pid = user_mode_thread(kernel_init, NULL, CLONE_FS); + pid = kernel_clone(&init_args); /* * Pin init on the boot CPU. Task migration is not properly working * until sched_init_smp() has been run. It will set the allowed @@ -1540,6 +1546,8 @@ static int __ref kernel_init(void *unused) { int ret; + init_userspace_fs(); + /* * Wait until kthreadd is all set-up. */ diff --git a/ipc/mqueue.c b/ipc/mqueue.c index 4798b375972b..d1dd36a651b0 100644 --- a/ipc/mqueue.c +++ b/ipc/mqueue.c @@ -608,7 +608,7 @@ out_unlock: } static int mqueue_create(struct mnt_idmap *idmap, struct inode *dir, - struct dentry *dentry, umode_t mode, bool excl) + struct dentry *dentry, umode_t mode) { return mqueue_create_attr(dentry, mode, NULL); } @@ -790,7 +790,6 @@ static void __do_notify(struct mqueue_inode_info *info) struct kernel_siginfo sig_i; struct task_struct *task; - /* do_mq_notify() accepts sigev_signo == 0, why?? */ if (!info->notify.sigev_signo) break; @@ -1281,9 +1280,9 @@ static int do_mq_notify(mqd_t mqdes, const struct sigevent *notification) notification->sigev_notify != SIGEV_THREAD)) return -EINVAL; if (notification->sigev_notify == SIGEV_SIGNAL && - !valid_signal(notification->sigev_signo)) { + (!notification->sigev_signo || + !valid_signal(notification->sigev_signo))) return -EINVAL; - } if (notification->sigev_notify == SIGEV_THREAD) { long timeo; diff --git a/kernel/exit.c b/kernel/exit.c index 2c0b1c02920f..f812df279ea7 100644 --- a/kernel/exit.c +++ b/kernel/exit.c @@ -1116,7 +1116,7 @@ void __noreturn make_task_dead(int signr) SYSCALL_DEFINE1(exit, int, error_code) { - do_exit((error_code&0xff)<<8); + do_exit((error_code & 0xff) << 8); } /* diff --git a/kernel/fork.c b/kernel/fork.c index f0e2e131a9a5..94e021eabf1d 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -1618,9 +1618,27 @@ static int copy_exec_state(u64 clone_flags, struct task_struct *tsk) return task_exec_state_copy(tsk); } -static int copy_fs(u64 clone_flags, struct task_struct *tsk) +static int copy_fs(u64 clone_flags, struct task_struct *tsk, bool umh) { - struct fs_struct *fs = current->fs; + struct fs_struct *fs; + + /* + * Usermodehelper may copy userspace_init_fs filesystem state but + * they don't get to create mount namespaces, share the + * filesystem state, or be started from a non-initial mount + * namespace. + */ + if (umh) { + if (clone_flags & (CLONE_NEWNS | CLONE_FS)) + return -EINVAL; + if (current->nsproxy->mnt_ns != &init_mnt_ns) + return -EINVAL; + fs = userspace_init_fs; + } else { + fs = current->fs; + VFS_WARN_ON_ONCE(current->fs != current->real_fs); + } + if (clone_flags & CLONE_FS) { /* tsk->fs is already what we want */ read_seqlock_excl(&fs->seq); @@ -1633,7 +1651,7 @@ static int copy_fs(u64 clone_flags, struct task_struct *tsk) read_sequnlock_excl(&fs->seq); return 0; } - tsk->fs = copy_fs_struct(fs); + tsk->real_fs = tsk->fs = copy_fs_struct(fs); if (!tsk->fs) return -ENOMEM; return 0; @@ -2277,7 +2295,7 @@ __latent_entropy struct task_struct *copy_process( retval = copy_files(clone_flags, p, args->no_files); if (retval) goto bad_fork_cleanup_semundo; - retval = copy_fs(clone_flags, p); + retval = copy_fs(clone_flags, p, args->umh); if (retval) goto bad_fork_cleanup_files; retval = copy_sighand(clone_flags, p); @@ -2819,6 +2837,7 @@ pid_t user_mode_thread(int (*fn)(void *), void *arg, unsigned long flags) .exit_signal = (flags & CSIGNAL), .fn = fn, .fn_arg = arg, + .umh = 1, }; return kernel_clone(&args); @@ -3216,7 +3235,7 @@ static int unshare_fd(unsigned long unshare_flags, struct files_struct **new_fdp */ int ksys_unshare(unsigned long unshare_flags) { - struct fs_struct *fs, *new_fs = NULL; + struct fs_struct *new_fs = NULL; struct files_struct *new_fd = NULL; struct cred *new_cred = NULL; struct nsproxy *new_nsproxy = NULL; @@ -3247,6 +3266,10 @@ int ksys_unshare(unsigned long unshare_flags) if (unshare_flags & CLONE_NEWNS) unshare_flags |= CLONE_FS; + /* No unsharing with overriden fs state */ + VFS_WARN_ON_ONCE(unshare_flags & (CLONE_NEWNS | CLONE_FS) && + current->fs != current->real_fs); + err = check_unshare_flags(unshare_flags); if (err) goto bad_unshare_out; @@ -3294,23 +3317,13 @@ int ksys_unshare(unsigned long unshare_flags) new_nsproxy = NULL; } - task_lock(current); + if (new_fs) + new_fs = switch_fs_struct(new_fs); - if (new_fs) { - fs = current->fs; - read_seqlock_excl(&fs->seq); - current->fs = new_fs; - if (--fs->users) - new_fs = NULL; - else - new_fs = fs; - read_sequnlock_excl(&fs->seq); - } - - if (new_fd) + if (new_fd) { + guard(task_lock)(current); swap(current->files, new_fd); - - task_unlock(current); + } if (new_cred) { /* Install the new user namespace */ diff --git a/kernel/kcmp.c b/kernel/kcmp.c index 7c1a65bd5f8d..76476aeee067 100644 --- a/kernel/kcmp.c +++ b/kernel/kcmp.c @@ -186,7 +186,7 @@ SYSCALL_DEFINE5(kcmp, pid_t, pid1, pid_t, pid2, int, type, ret = kcmp_ptr(task1->files, task2->files, KCMP_FILES); break; case KCMP_FS: - ret = kcmp_ptr(task1->fs, task2->fs, KCMP_FS); + ret = kcmp_ptr(task1->real_fs, task2->real_fs, KCMP_FS); break; case KCMP_SIGHAND: ret = kcmp_ptr(task1->sighand, task2->sighand, KCMP_SIGHAND); diff --git a/kernel/umh.c b/kernel/umh.c index 48117c569e1a..6e2c7bb315c6 100644 --- a/kernel/umh.c +++ b/kernel/umh.c @@ -71,10 +71,8 @@ static int call_usermodehelper_exec_async(void *data) spin_unlock_irq(¤t->sighand->siglock); /* - * Initial kernel threads share ther FS with init, in order to - * get the init root directory. But we've now created a new - * thread that is going to execve a user process and has its own - * 'struct fs_struct'. Reset umask to the default. + * Usermodehelper threads get a copy of userspace init's + * fs_struct. Reset umask to the default. */ current->fs->umask = 0022; diff --git a/kernel/user.c b/kernel/user.c index 7aef4e679a6a..21bafdc11379 100644 --- a/kernel/user.c +++ b/kernel/user.c @@ -23,9 +23,9 @@ #if IS_ENABLED(CONFIG_BINFMT_MISC) struct binfmt_misc init_binfmt_misc = { - .entries = LIST_HEAD_INIT(init_binfmt_misc.entries), + .entries = HLIST_HEAD_INIT, .enabled = true, - .entries_lock = __RW_LOCK_UNLOCKED(init_binfmt_misc.entries_lock), + .entries_lock = __SPIN_LOCK_UNLOCKED(init_binfmt_misc.entries_lock), }; EXPORT_SYMBOL_GPL(init_binfmt_misc); #endif diff --git a/lib/lockref.c b/lib/lockref.c index 5d8e3ef3860e..9b3dd688d8cd 100644 --- a/lib/lockref.c +++ b/lib/lockref.c @@ -131,7 +131,7 @@ EXPORT_SYMBOL(lockref_put_or_lock); void lockref_mark_dead(struct lockref *lockref) { assert_spin_locked(&lockref->lock); - lockref->count = -128; + lockref->count = __LOCKREF_DEAD_VAL; } EXPORT_SYMBOL(lockref_mark_dead); diff --git a/lib/seq_buf.c b/lib/seq_buf.c index b59488fa8135..a92093f346da 100644 --- a/lib/seq_buf.c +++ b/lib/seq_buf.c @@ -321,7 +321,7 @@ int seq_buf_path(struct seq_buf *s, const struct path *path, const char *esc) if (size) { char *p = d_path(path, buf, size); if (!IS_ERR(p)) { - char *end = mangle_path(buf, p, esc); + char *end = seq_mangle_path(buf, p, esc); if (end) res = end - buf; } diff --git a/mm/shmem.c b/mm/shmem.c index 9001aaf3b7b9..dc8cd4f563f4 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -3874,7 +3874,7 @@ static struct dentry *shmem_mkdir(struct mnt_idmap *idmap, struct inode *dir, } static int shmem_create(struct mnt_idmap *idmap, struct inode *dir, - struct dentry *dentry, umode_t mode, bool excl) + struct dentry *dentry, umode_t mode) { return shmem_mknod(idmap, dir, dentry, mode | S_IFREG, 0); } diff --git a/net/socket.c b/net/socket.c index 63c69a0fa74e..b0256cd222f8 100644 --- a/net/socket.c +++ b/net/socket.c @@ -465,6 +465,31 @@ static const struct xattr_handler sockfs_user_xattr_handler = { .set = sockfs_user_xattr_set, }; +/** + * sock_read_xattr - read a user.* xattr from a socket's sockfs inode + * @sock: socket whose inode holds the xattr + * @name: full xattr name, e.g. "user.bpf_test" + * @value: output buffer + * @size: size of @value in bytes + * + * SOCK_INODE() is valid only for sockfs sockets; sock_from_file() rejects + * anything else (e.g. tun, tap). + * Lockless: simple_xattr_get() looks up the value under RCU, no inode lock. + * + * Return: length of the value on success, a negative errno on error. + */ +int sock_read_xattr(struct socket *sock, const char *name, void *value, size_t size) +{ + struct file *file = sock->file; + struct sockfs_inode *si; + + if (!file || sock_from_file(file) != sock) + return -EOPNOTSUPP; + + si = SOCKFS_I(SOCK_INODE(sock)); + return simple_xattr_get(&sockfs_xa_cache, &si->xattrs, name, value, size); +} + static const struct xattr_handler * const sockfs_xattr_handlers[] = { &sockfs_xattr_handler, &sockfs_security_xattr_handler, diff --git a/net/unix/af_unix.c b/net/unix/af_unix.c index 10ed9421e43a..7794740fa80f 100644 --- a/net/unix/af_unix.c +++ b/net/unix/af_unix.c @@ -1197,17 +1197,12 @@ static struct sock *unix_find_bsd(struct sockaddr_un *sunaddr, int addr_len, unix_mkname_bsd(sunaddr, addr_len); if (flags & SOCK_COREDUMP) { - struct path root; - - task_lock(&init_task); - get_fs_root(init_task.fs, &root); - task_unlock(&init_task); - - scoped_with_kernel_creds() - err = vfs_path_lookup(root.dentry, root.mnt, sunaddr->sun_path, - LOOKUP_BENEATH | LOOKUP_NO_SYMLINKS | - LOOKUP_NO_MAGICLINKS, &path); - path_put(&root); + scoped_with_init_fs() { + scoped_with_kernel_creds() + err = kern_path(sunaddr->sun_path, + LOOKUP_NO_SYMLINKS | + LOOKUP_NO_MAGICLINKS, &path); + } if (err) goto fail; } else { diff --git a/scripts/syscall.tbl b/scripts/syscall.tbl index 7a42b32b6577..0ab531605120 100644 --- a/scripts/syscall.tbl +++ b/scripts/syscall.tbl @@ -412,3 +412,4 @@ 469 common file_setattr sys_file_setattr 470 common listns sys_listns 471 common rseq_slice_yield sys_rseq_slice_yield +472 common fchroot sys_fchroot diff --git a/tools/include/uapi/asm-generic/unistd.h b/tools/include/uapi/asm-generic/unistd.h index a627acc8fb5f..5b7e77a7c736 100644 --- a/tools/include/uapi/asm-generic/unistd.h +++ b/tools/include/uapi/asm-generic/unistd.h @@ -863,8 +863,12 @@ __SYSCALL(__NR_listns, sys_listns) #define __NR_rseq_slice_yield 471 __SYSCALL(__NR_rseq_slice_yield, sys_rseq_slice_yield) +/* fs/open.c */ +#define __NR_fchroot 472 +__SYSCALL(__NR_fchroot, sys_fchroot) + #undef __NR_syscalls -#define __NR_syscalls 472 +#define __NR_syscalls 473 /* * 32 bit systems traditionally used different diff --git a/tools/perf/arch/arm/entry/syscalls/syscall.tbl b/tools/perf/arch/arm/entry/syscalls/syscall.tbl index 94351e22bfcf..55717ed32c27 100644 --- a/tools/perf/arch/arm/entry/syscalls/syscall.tbl +++ b/tools/perf/arch/arm/entry/syscalls/syscall.tbl @@ -486,3 +486,4 @@ 469 common file_setattr sys_file_setattr 470 common listns sys_listns 471 common rseq_slice_yield sys_rseq_slice_yield +472 common fchroot sys_fchroot diff --git a/tools/perf/arch/mips/entry/syscalls/syscall_n64.tbl b/tools/perf/arch/mips/entry/syscalls/syscall_n64.tbl index 630aab9e5425..83dc93a0712f 100644 --- a/tools/perf/arch/mips/entry/syscalls/syscall_n64.tbl +++ b/tools/perf/arch/mips/entry/syscalls/syscall_n64.tbl @@ -386,3 +386,4 @@ 469 n64 file_setattr sys_file_setattr 470 n64 listns sys_listns 471 n64 rseq_slice_yield sys_rseq_slice_yield +472 n64 fchroot sys_fchroot diff --git a/tools/perf/arch/powerpc/entry/syscalls/syscall.tbl b/tools/perf/arch/powerpc/entry/syscalls/syscall.tbl index 4fcc7c58a105..cfbb70039ff0 100644 --- a/tools/perf/arch/powerpc/entry/syscalls/syscall.tbl +++ b/tools/perf/arch/powerpc/entry/syscalls/syscall.tbl @@ -562,3 +562,4 @@ 469 common file_setattr sys_file_setattr 470 common listns sys_listns 471 nospu rseq_slice_yield sys_rseq_slice_yield +472 common fchroot sys_fchroot diff --git a/tools/perf/arch/s390/entry/syscalls/syscall.tbl b/tools/perf/arch/s390/entry/syscalls/syscall.tbl index 09a7ef04d979..1b45e68a217b 100644 --- a/tools/perf/arch/s390/entry/syscalls/syscall.tbl +++ b/tools/perf/arch/s390/entry/syscalls/syscall.tbl @@ -398,3 +398,4 @@ 469 common file_setattr sys_file_setattr 470 common listns sys_listns 471 common rseq_slice_yield sys_rseq_slice_yield +472 common fchroot sys_fchroot diff --git a/tools/perf/arch/sh/entry/syscalls/syscall.tbl b/tools/perf/arch/sh/entry/syscalls/syscall.tbl index 70b315cbe710..ace068dff0de 100644 --- a/tools/perf/arch/sh/entry/syscalls/syscall.tbl +++ b/tools/perf/arch/sh/entry/syscalls/syscall.tbl @@ -475,3 +475,4 @@ 469 common file_setattr sys_file_setattr 470 common listns sys_listns 471 common rseq_slice_yield sys_rseq_slice_yield +472 common fchroot sys_fchroot diff --git a/tools/perf/arch/sparc/entry/syscalls/syscall.tbl b/tools/perf/arch/sparc/entry/syscalls/syscall.tbl index 7e71bf7fcd14..5b9fe0e8140f 100644 --- a/tools/perf/arch/sparc/entry/syscalls/syscall.tbl +++ b/tools/perf/arch/sparc/entry/syscalls/syscall.tbl @@ -517,3 +517,4 @@ 469 common file_setattr sys_file_setattr 470 common listns sys_listns 471 common rseq_slice_yield sys_rseq_slice_yield +472 common fchroot sys_fchroot diff --git a/tools/perf/arch/x86/entry/syscalls/syscall_32.tbl b/tools/perf/arch/x86/entry/syscalls/syscall_32.tbl index f832ebd2d79b..2c172ef48dfd 100644 --- a/tools/perf/arch/x86/entry/syscalls/syscall_32.tbl +++ b/tools/perf/arch/x86/entry/syscalls/syscall_32.tbl @@ -477,3 +477,4 @@ 469 i386 file_setattr sys_file_setattr 470 i386 listns sys_listns 471 i386 rseq_slice_yield sys_rseq_slice_yield +472 i386 fchroot sys_fchroot diff --git a/tools/perf/arch/x86/entry/syscalls/syscall_64.tbl b/tools/perf/arch/x86/entry/syscalls/syscall_64.tbl index 524155d655da..d5b6045b0090 100644 --- a/tools/perf/arch/x86/entry/syscalls/syscall_64.tbl +++ b/tools/perf/arch/x86/entry/syscalls/syscall_64.tbl @@ -396,6 +396,7 @@ 469 common file_setattr sys_file_setattr 470 common listns sys_listns 471 common rseq_slice_yield sys_rseq_slice_yield +472 common fchroot sys_fchroot # # Due to a historical design error, certain syscalls are numbered differently diff --git a/tools/perf/arch/xtensa/entry/syscalls/syscall.tbl b/tools/perf/arch/xtensa/entry/syscalls/syscall.tbl index a9bca4e484de..d354bb231796 100644 --- a/tools/perf/arch/xtensa/entry/syscalls/syscall.tbl +++ b/tools/perf/arch/xtensa/entry/syscalls/syscall.tbl @@ -442,3 +442,4 @@ 469 common file_setattr sys_file_setattr 470 common listns sys_listns 471 common rseq_slice_yield sys_rseq_slice_yield +472 common fchroot sys_fchroot diff --git a/tools/scripts/syscall.tbl b/tools/scripts/syscall.tbl index 7a42b32b6577..0ab531605120 100644 --- a/tools/scripts/syscall.tbl +++ b/tools/scripts/syscall.tbl @@ -412,3 +412,4 @@ 469 common file_setattr sys_file_setattr 470 common listns sys_listns 471 common rseq_slice_yield sys_rseq_slice_yield +472 common fchroot sys_fchroot diff --git a/tools/testing/selftests/Makefile b/tools/testing/selftests/Makefile index 8d4db2241cc2..e23ed889a16a 100644 --- a/tools/testing/selftests/Makefile +++ b/tools/testing/selftests/Makefile @@ -33,6 +33,7 @@ TARGETS += fchmodat2 TARGETS += filesystems TARGETS += filesystems/binderfs TARGETS += filesystems/epoll +TARGETS += filesystems/failfs TARGETS += filesystems/fat TARGETS += filesystems/overlayfs TARGETS += filesystems/statmount @@ -42,6 +43,7 @@ TARGETS += filesystems/fuse TARGETS += filesystems/move_mount TARGETS += filesystems/empty_mntns TARGETS += filesystems/fsmount_ns +TARGETS += filesystems/mntns_cleanup TARGETS += firmware TARGETS += fpu TARGETS += ftrace diff --git a/tools/testing/selftests/bpf/bpf_experimental.h b/tools/testing/selftests/bpf/bpf_experimental.h index 67ff7882299e..63e4e472fe36 100644 --- a/tools/testing/selftests/bpf/bpf_experimental.h +++ b/tools/testing/selftests/bpf/bpf_experimental.h @@ -364,6 +364,9 @@ extern void bpf_iter_dmabuf_destroy(struct bpf_iter_dmabuf *it) __weak __ksym; extern int bpf_cgroup_read_xattr(struct cgroup *cgroup, const char *name__str, struct bpf_dynptr *value_p) __weak __ksym; +extern int bpf_sock_read_xattr(struct socket *sock, const char *name__str, + struct bpf_dynptr *value_p) __weak __ksym; + #define PREEMPT_BITS 8 #define SOFTIRQ_BITS 8 #define HARDIRQ_BITS 4 diff --git a/tools/testing/selftests/bpf/prog_tests/sock_xattr.c b/tools/testing/selftests/bpf/prog_tests/sock_xattr.c new file mode 100644 index 000000000000..b5816e90f01a --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/sock_xattr.c @@ -0,0 +1,67 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* Copyright (c) 2026 Christian Brauner */ + +#include +#include +#include +#include +#include +#include +#include + +#include "sock_read_xattr.skel.h" + +static const char xattr_value[] = "bpf_sock_value"; +static const char xattr_name[] = "user.bpf_test"; + +static void test_read_sock_xattr(void) +{ + struct sockaddr_in addr = {}; + struct sock_read_xattr *skel = NULL; + struct bpf_link *link = NULL; + int sock_fd = -1, err; + + sock_fd = socket(AF_INET, SOCK_STREAM, 0); + if (!ASSERT_OK_FD(sock_fd, "socket")) + return; + + err = fsetxattr(sock_fd, xattr_name, xattr_value, sizeof(xattr_value), 0); + if (!ASSERT_OK(err, "fsetxattr")) + goto out; + + skel = sock_read_xattr__open_and_load(); + if (!ASSERT_OK_PTR(skel, "sock_read_xattr__open_and_load")) + goto out; + + skel->bss->monitored_pid = sys_gettid(); + + /* Only attach the functional program; the verifier-only programs + * above are not pid-gated and would clobber the shared globals. + */ + link = bpf_program__attach(skel->progs.read_sock_xattr); + if (!ASSERT_OK_PTR(link, "attach read_sock_xattr")) + goto out; + + addr.sin_family = AF_INET; + addr.sin_port = htons(1234); + addr.sin_addr.s_addr = htonl(INADDR_LOOPBACK); + /* Only the lsm/socket_connect hook matters; the connect may fail. */ + connect(sock_fd, (struct sockaddr *)&addr, sizeof(addr)); + + ASSERT_EQ(skel->data->read_ret, sizeof(xattr_value), "read_ret"); + ASSERT_STREQ(skel->bss->value, xattr_value, "value"); + +out: + bpf_link__destroy(link); + if (sock_fd >= 0) + close(sock_fd); + sock_read_xattr__destroy(skel); +} + +void test_sock_xattr(void) +{ + RUN_TESTS(sock_read_xattr); + + if (test__start_subtest("read_sock_xattr")) + test_read_sock_xattr(); +} diff --git a/tools/testing/selftests/bpf/progs/sock_read_xattr.c b/tools/testing/selftests/bpf/progs/sock_read_xattr.c new file mode 100644 index 000000000000..c4a8eae8cc3c --- /dev/null +++ b/tools/testing/selftests/bpf/progs/sock_read_xattr.c @@ -0,0 +1,54 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2026 Christian Brauner */ + +#include +#include +#include +#include +#include "bpf_experimental.h" +#include "bpf_misc.h" + +char _license[] SEC("license") = "GPL"; + +char value[16]; +int read_ret = -1; +__u32 monitored_pid = 0; + +static __always_inline void read_xattr(struct socket *sock) +{ + struct bpf_dynptr value_ptr; + + bpf_dynptr_from_mem(value, sizeof(value), 0, &value_ptr); + bpf_sock_read_xattr(sock, "user.bpf_test", &value_ptr); +} + +SEC("lsm.s/socket_connect") +__success +int BPF_PROG(trusted_sock_ptr_sleepable, struct socket *sock) +{ + read_xattr(sock); + return 0; +} + +SEC("lsm/socket_connect") +__success +int BPF_PROG(trusted_sock_ptr_non_sleepable, struct socket *sock) +{ + read_xattr(sock); + return 0; +} + +SEC("lsm.s/socket_connect") +__success +int BPF_PROG(read_sock_xattr, struct socket *sock) +{ + struct bpf_dynptr value_ptr; + __u32 pid = bpf_get_current_pid_tgid() >> 32; + + if (pid != monitored_pid) + return 0; + + bpf_dynptr_from_mem(value, sizeof(value), 0, &value_ptr); + read_ret = bpf_sock_read_xattr(sock, "user.bpf_test", &value_ptr); + return 0; +} diff --git a/tools/testing/selftests/exec/.gitignore b/tools/testing/selftests/exec/.gitignore index 7f3d1ae762ec..fbbb1600ddb9 100644 --- a/tools/testing/selftests/exec/.gitignore +++ b/tools/testing/selftests/exec/.gitignore @@ -19,3 +19,13 @@ null-argv xxxxxxxx* pipe S_I*.test +binfmt_misc_bpf +binfmt_bpf_interp +binfmt_bpf_app +binfmt_misc_transparent +binfmt_transparent_interp +binfmt_misc_loader +binfmt_loader_payload +binfmt_loader_payload_static +*.bpf.o +vmlinux.h diff --git a/tools/testing/selftests/exec/Makefile b/tools/testing/selftests/exec/Makefile index 45a3cfc435cf..410c93606a0c 100644 --- a/tools/testing/selftests/exec/Makefile +++ b/tools/testing/selftests/exec/Makefile @@ -21,9 +21,52 @@ TEST_GEN_PROGS += recursion-depth TEST_GEN_PROGS += null-argv TEST_GEN_PROGS += check-exec +# binfmt_misc must not be reachable as an exec source or as a stacking layer, +# or an 'F' entry can pin the instance that owns it. Unprivileged, no bpf. +TEST_GEN_PROGS += binfmt_misc_selfpin + +# 'D' (register disabled) binfmt_misc test: an entry that exists but does +# not dispatch until it is enabled. Static magic entry, no bpf toolchain. +TEST_GEN_PROGS += binfmt_misc_disabled + +# Static ('T' flag) transparent binfmt_misc test; the asserting interpreter +# is shared with the bpf harness's transparent case. No bpf toolchain needed. +TEST_GEN_PROGS += binfmt_misc_transparent +TEST_GEN_FILES += binfmt_transparent_interp + +# 'L' (loader substitution) binfmt_misc test: the payload runs as the main +# image with a copy of the system loader substituted for its PT_INTERP and +# asserts the native identity from inside; the static build proves the +# override is dropped for a binary without PT_INTERP. +TEST_GEN_PROGS += binfmt_misc_loader +TEST_GEN_FILES += binfmt_loader_payload binfmt_loader_payload_static + +# binfmt_misc bpf-backed ('B') handler test: a libbpf harness plus its +# struct_ops objects and the test interpreter/app it routes between. Only +# built when clang, bpftool, the vmlinux BTF and libbpf are all present +# (HAVE_BPF_TOOLCHAIN=y forces it) so the other exec selftests don't grow +# a bpf toolchain dependency. +CLANG ?= clang +BPFTOOL ?= bpftool +VMLINUX_BTF ?= /sys/kernel/btf/vmlinux +HAVE_BPF_TOOLCHAIN ?= $(shell command -v $(CLANG) >/dev/null 2>&1 && \ + command -v $(BPFTOOL) >/dev/null 2>&1 && \ + test -r $(VMLINUX_BTF) && \ + pkg-config --exists libbpf 2>/dev/null && echo y) +ifeq ($(HAVE_BPF_TOOLCHAIN),y) +TEST_GEN_PROGS += binfmt_misc_bpf +TEST_GEN_FILES += bpf_interp.bpf.o nix_origin.bpf.o transparent.bpf.o +TEST_GEN_FILES += loader.bpf.o interp_bind.bpf.o +TEST_GEN_FILES += binfmt_bpf_interp binfmt_bpf_app binfmt_bind_interp +else +$(info exec selftests: skipping binfmt_misc_bpf, needs clang, bpftool, vmlinux BTF and libbpf) +endif + EXTRA_CLEAN := $(OUTPUT)/subdir.moved $(OUTPUT)/execveat.moved $(OUTPUT)/xxxxx* \ $(OUTPUT)/S_I*.test +LOCAL_HDRS += binfmt_misc_common.h + include ../lib.mk CHECK_EXEC_SAMPLES := $(top_srcdir)/samples/check-exec @@ -55,3 +98,47 @@ $(OUTPUT)/script-exec.inc: $(CHECK_EXEC_SAMPLES)/script-exec.inc cp $< $@ $(OUTPUT)/script-noexec.inc: $(CHECK_EXEC_SAMPLES)/script-noexec.inc cp $< $@ + +# Reuses setup_userns()/write_file() from the filesystems selftests. Their +# wrappers.h wants the uapi headers, so ask for them here rather than widening +# CFLAGS for every program in this directory. +$(OUTPUT)/binfmt_misc_selfpin: CFLAGS += $(TOOLS_INCLUDES) +$(OUTPUT)/binfmt_misc_selfpin: ../filesystems/utils.c + +# --- binfmt_misc bpf ('B') handler test --------------------------------- +# The struct_ops bpf objects are compiled against the running kernel's BTF. +# CLANG/BPFTOOL/VMLINUX_BTF are set above next to the toolchain check; +# override LIBBPF_CFLAGS/LDLIBS to point at a libbpf install. +BPF_CFLAGS ?= -I$(OUTPUT) +LIBBPF_CFLAGS ?= +LIBBPF_LDLIBS ?= -lbpf -lelf -lz + +$(OUTPUT)/vmlinux.h: + $(BPFTOOL) btf dump file $(VMLINUX_BTF) format c > $@ + +# BPF_NO_KFUNC_PROTOTYPES: the programs declare the kfuncs they use themselves. +$(OUTPUT)/%.bpf.o: %.bpf.c $(OUTPUT)/vmlinux.h + $(CLANG) -g -O2 -target bpf -mcpu=v3 -DBPF_NO_KFUNC_PROTOTYPES \ + $(BPF_CFLAGS) $(LIBBPF_CFLAGS) -c $< -o $@ + +$(OUTPUT)/binfmt_misc_bpf: binfmt_misc_bpf.c binfmt_misc_common.h + $(CC) $(CFLAGS) $(LIBBPF_CFLAGS) $(LDFLAGS) $< $(LIBBPF_LDLIBS) -o $@ + +$(OUTPUT)/binfmt_bpf_interp: binfmt_bpf_interp.c + $(CC) $(CFLAGS) $(LDFLAGS) $< -o $@ + +$(OUTPUT)/binfmt_bind_interp: binfmt_bind_interp.c + $(CC) $(CFLAGS) $(LDFLAGS) $< -o $@ + +$(OUTPUT)/binfmt_loader_payload: binfmt_loader_payload.c binfmt_misc_common.h + $(CC) $(CFLAGS) $(LDFLAGS) -fPIE -pie $< -o $@ + +$(OUTPUT)/binfmt_loader_payload_static: binfmt_loader_payload.c binfmt_misc_common.h + $(CC) $(CFLAGS) $(LDFLAGS) -static $< -o $@ + +# PT_INTERP is set to the literal "$ORIGIN/binfmt_bpf_interp"; the nix_origin +# handler resolves it relative to the binary at run time. +$(OUTPUT)/binfmt_bpf_app: binfmt_bpf_app.c + $(CC) $(CFLAGS) $(LDFLAGS) -Wl,--dynamic-linker,'$$ORIGIN/binfmt_bpf_interp' $< -o $@ + +EXTRA_CLEAN += $(OUTPUT)/vmlinux.h $(OUTPUT)/*.bpf.o diff --git a/tools/testing/selftests/exec/binfmt_bind_interp.c b/tools/testing/selftests/exec/binfmt_bind_interp.c new file mode 100644 index 000000000000..06d65062856b --- /dev/null +++ b/tools/testing/selftests/exec/binfmt_bind_interp.c @@ -0,0 +1,14 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Test interpreter for the bound-interpreter case of the binfmt_misc_bpf + * selftest. Two copies are installed at different paths and bound to one + * entry under different names; printing argv[0] - the path the kernel ran + * this copy under - tells the harness which of them the load program picked. + */ +#include + +int main(int argc, char **argv) +{ + printf("BIND_RAN %s\n", argc > 0 ? argv[0] : ""); + return 0; +} diff --git a/tools/testing/selftests/exec/binfmt_bpf_app.c b/tools/testing/selftests/exec/binfmt_bpf_app.c new file mode 100644 index 000000000000..472270f148bc --- /dev/null +++ b/tools/testing/selftests/exec/binfmt_bpf_app.c @@ -0,0 +1,12 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * A relocatable binary for the binfmt_misc_bpf $ORIGIN case. The Makefile + * links it with PT_INTERP set to the literal "$ORIGIN/binfmt_bpf_interp" + * (-Wl,--dynamic-linker), which the kernel ELF loader cannot resolve. The + * nix_origin bpf handler resolves it relative to this binary's directory and + * routes execution to the co-located interpreter. + */ +int main(void) +{ + return 0; +} diff --git a/tools/testing/selftests/exec/binfmt_bpf_interp.c b/tools/testing/selftests/exec/binfmt_bpf_interp.c new file mode 100644 index 000000000000..2db205f095b2 --- /dev/null +++ b/tools/testing/selftests/exec/binfmt_bpf_interp.c @@ -0,0 +1,15 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Test interpreter for the binfmt_misc_bpf selftest. A bpf-backed 'B' handler + * routes a matched binary here; printing this marker proves the program's + * chosen interpreter actually ran. + */ +#include + +int main(int argc, char **argv) +{ + (void)argc; + (void)argv; + write(1, "BPF_INTERP_RAN\n", 15); + return 0; +} diff --git a/tools/testing/selftests/exec/binfmt_loader_payload.c b/tools/testing/selftests/exec/binfmt_loader_payload.c new file mode 100644 index 000000000000..272db8efb4b5 --- /dev/null +++ b/tools/testing/selftests/exec/binfmt_loader_payload.c @@ -0,0 +1,146 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Payload for the binfmt_misc 'L' (loader substitution) selftest. It is + * executed as the MAIN image - a fully native exec - with the registered + * interpreter substituted for its PT_INTERP, and asserts the native + * identity from the inside. Exits 0 when every surface checks out. + * + * Modes, selected by the orchestrator via the environment: + * - default: full assertions, path-based ones included + * - BINFMT_TEST_MEMFD=1: executed from an inaccessible memfd, skip + * the path-based assertions + * - BINFMT_TEST_STATIC=1: static build; the override was dropped, so + * expect no interpreter at all + */ +#define _GNU_SOURCE +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "binfmt_misc_common.h" + +/* Start of our own mapped image, courtesy of the linker. */ +extern const char __ehdr_start[]; + +/* An image is never this large; used to bracket "within our image". */ +#define IMAGE_SPAN (16UL << 20) + +static int failed; + +static void check(int cond, const char *what) +{ + if (cond) + return; + fprintf(stderr, "[payload] FAILED: %s (errno %d)\n", what, errno); + failed = 1; +} + +/* Return whether /proc/self/maps names a path starting with @prefix. */ +static int maps_has_prefix(const char *prefix) +{ + char *line = NULL; + size_t len = 0; + int found = 0; + FILE *f; + + f = fopen("/proc/self/maps", "r"); + if (!f) + return -1; + while (getline(&line, &len, f) > 0) { + char *path = strchr(line, '/'); + + if (path && !strncmp(path, prefix, strlen(prefix))) { + found = 1; + break; + } + } + free(line); + fclose(f); + return found; +} + +int main(int argc, char *argv[]) +{ + const char *binary = getenv("BINFMT_TEST_BINARY"); + const char *interp = getenv("BINFMT_TEST_INTERP"); + int memfd_mode = getenv("BINFMT_TEST_MEMFD") != NULL; + int static_mode = getenv("BINFMT_TEST_STATIC") != NULL; + unsigned long self = (unsigned long)__ehdr_start; + unsigned long base = getauxval(AT_BASE); + unsigned long phdr = getauxval(AT_PHDR); + unsigned long entry = getauxval(AT_ENTRY); + unsigned long start_code, end_code; + + /* The argument vector is exactly what the caller built. */ + check(argc == 3 && !strcmp(argv[0], PAYLOAD_ARGV0) && + !strcmp(argv[1], PAYLOAD_ARG1) && !strcmp(argv[2], PAYLOAD_ARG2), + "argv was rewritten"); + + /* Native from birth: no execfd, no dispatch marker. */ + check(getauxval(AT_EXECFD) == 0, "AT_EXECFD present"); + check(getauxval(AT_FLAGS) == 0, "AT_FLAGS not native"); + + if (static_mode) { + /* The override was dropped: no interpreter was loaded. */ + check(base == 0, "AT_BASE set for a static payload"); + } else { + /* A loader is mapped in the interpreter slot, not our image. */ + check(base != 0, "AT_BASE missing"); + check(base < self || base >= self + IMAGE_SPAN, + "AT_BASE inside our own image"); + } + + /* We occupy the main-image slot. */ + check(phdr >= self && phdr < self + IMAGE_SPAN, + "AT_PHDR outside our image"); + check(entry >= self && entry < self + IMAGE_SPAN, + "AT_ENTRY outside our image"); + + /* The code statistics markers describe our image, natively placed. */ + if (stat_codes(getpid(), &start_code, &end_code) == 0) { + check(start_code >= self && start_code < end_code && + end_code < self + IMAGE_SPAN, + "stat start_code/end_code not our image"); + check(entry >= start_code && entry < end_code, + "AT_ENTRY outside [start_code, end_code)"); + } else { + check(0, "cannot parse /proc/self/stat"); + } + + if (!memfd_mode && binary) { + const char *execfn = (const char *)getauxval(AT_EXECFN); + const char *base_name = strrchr(binary, '/'); + + base_name = base_name ? base_name + 1 : binary; + + /* exe link, AT_EXECFN and comm all follow the binary. */ + check(exe_is(binary), "/proc/self/exe"); + check(execfn && !strcmp(execfn, binary), "AT_EXECFN"); + check(comm_is(base_name), "comm"); + + /* The running binary is write-denied, natively. */ + check(write_denied(binary), "no ETXTBSY on the binary"); + } + + if (interp) { + int found = maps_has_prefix(interp); + + if (static_mode) + /* Nothing was substituted, nothing may be mapped. */ + check(found == 0, "loader mapped for a static payload"); + else + /* The substituted loader shows under its real path. */ + check(found == 1, "loader path not in /proc/self/maps"); + } + + if (failed) + return 1; + printf("[payload] native identity checks out\n"); + return 0; +} diff --git a/tools/testing/selftests/exec/binfmt_misc_bpf.c b/tools/testing/selftests/exec/binfmt_misc_bpf.c new file mode 100644 index 000000000000..2c7b63075f1d --- /dev/null +++ b/tools/testing/selftests/exec/binfmt_misc_bpf.c @@ -0,0 +1,568 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Selftest for binfmt_misc bpf-backed ('B') handlers. + * + * A handler is a struct binfmt_misc_ops struct_ops map with a sleepable match + * and a sleepable load program. Attaching it publishes it by name in the + * caller's user namespace; a 'B' entry referencing it by name in the + * interpreter field activates it: + * + * echo ':name:B:::::' > /proc/sys/fs/binfmt_misc/register + * + * Five self-contained cases are exercised: + * + * 1. bpf_interp: the match program matches a synthetic aarch64 ELF header + * from the prefetched bprm->buf and the load program routes it to a + * fixed interpreter of its choosing. + * 2. nix_origin: the match program reads the binary's program headers to + * commit only to a "$ORIGIN/..."-relative PT_INTERP and the load program + * resolves it to an interpreter co-located with the binary (the + * relocatable-loader case the kernel ELF loader cannot express). + * 3. transparent: the load program sets BPF_BINPRM_TRANSPARENT; the + * asserting interpreter (binfmt_transparent_interp) verifies the + * identity the kernel constructed (exe link, argv, cmdline, comm, + * AT_EXECFD, write denial) from inside the process. + * 4. loader: the load program sets BPF_BINPRM_LOADER; the payload + * (binfmt_loader_payload) runs as the main image with the selected + * interpreter substituted for its PT_INTERP and asserts the native + * identity from inside. + * 5. interp_bind: an entry registered disabled with 'D' is given its + * interpreters one write at a time, and the load program picks one by + * name per exec. Replacing what the path holds afterwards changes + * nothing, which is the point of binding a file rather than resolving + * a name at exec time. Enabling the entry seals it. + * + * The first two route to a test interpreter that prints BPF_INTERP_RAN, + * proving the program's chosen interpreter actually ran. + */ +#define _GNU_SOURCE +#include +#include +#include +#include +#include +#include +#include + +#include +#include + +#include "binfmt_misc_common.h" +#include "kselftest_harness.h" + +#define INTERP_PATH "/tmp/binfmt_bpf_interp" +#define AARCH64_PATH "/tmp/binfmt_bpf_aarch64" +#define RELOC_TEMPLATE "/tmp/binfmt_relocXXXXXX" +#define TRANS_INTERP "/tmp/binfmt_transparent_interp" +#define TRANS_PATH "/tmp/binfmt_bpf_riscv" +#define EXPECT "BPF_INTERP_RAN" +#define TRANS_EXPECT "TRANSPARENT_OK" +#define LOADER_INTERP "/tmp/binfmt_loader_interp" +#define LOADER_PATH "/tmp/binfmt_bpf_loader.ldrtest" +#define BIND_FIRST "/tmp/binfmt_bind_first" +#define BIND_SECOND "/tmp/binfmt_bind_second" +#define BIND_ARM_PATH "/tmp/binfmt_bind_arm" +#define BIND_RISCV_PATH "/tmp/binfmt_bind_riscv" +#define BIND_EXPECT "BIND_RAN " +#define BIND_MAX 100 + +/* A minimal 64-bit little-endian ELF header, padded to the read size. */ +static int create_fake_elf(const char *path, unsigned short machine) +{ + unsigned char hdr[256] = {0}; + int fd; + + hdr[0] = 0x7f; hdr[1] = 'E'; hdr[2] = 'L'; hdr[3] = 'F'; + hdr[4] = ELFCLASS64; + hdr[5] = ELFDATA2LSB; + hdr[6] = EV_CURRENT; + hdr[16] = ET_EXEC; + hdr[18] = machine & 0xff; /* e_machine, little-endian */ + hdr[19] = machine >> 8; + hdr[20] = EV_CURRENT; + + unlink(path); + fd = open(path, O_WRONLY | O_CREAT | O_EXCL, 0755); + if (fd < 0) + return -1; + if (write(fd, hdr, sizeof(hdr)) != (ssize_t)sizeof(hdr)) { + close(fd); + return -1; + } + close(fd); + return 0; +} + +/* + * Register a 'B' entry for @handler. With @flags "D" the entry is created + * disabled, which is what leaves it open to being given interpreters. + */ +static int register_entry(const char *name, const char *handler, + const char *flags) +{ + char rule[PATH_MAX]; + + snprintf(rule, sizeof(rule), ":%s:B::::%s:%s", name, handler, + flags ? flags : ""); + return write_reg(rule); +} + +static int check_output(const char *cmd, const char *expected) +{ + char buf[128]; + FILE *fp; + + fp = popen(cmd, "r"); + if (!fp) + return -1; + if (!fgets(buf, sizeof(buf), fp)) { + pclose(fp); + return -1; + } + pclose(fp); + return strncmp(buf, expected, strlen(expected)) ? -1 : 0; +} + +/* Does the kernel BTF know struct binfmt_misc_ops (CONFIG_BINFMT_MISC_BPF)? */ +static bool have_binfmt_misc_ops(void) +{ + struct btf *btf = btf__load_vmlinux_btf(); + bool have; + + have = btf && btf__find_by_name_kind(btf, "binfmt_misc_ops", + BTF_KIND_STRUCT) >= 0; + btf__free(btf); + return have; +} + +/* The reason bpf handler cases cannot run here, NULL if they can. */ +static const char *bpf_handler_unsupported(void) +{ + if (getuid() != 0) + return "test must be run as root"; + if (!have_binfmt_misc_ops()) + return "no struct binfmt_misc_ops in the kernel BTF (CONFIG_BINFMT_MISC_BPF)"; + if (!binfmt_misc_available()) + return "no binfmt_misc"; + return NULL; +} + +/* An attached handler with its 'B' entry activated. */ +struct bpf_case { + struct bpf_object *obj; + struct bpf_link *link; + const char *entry; +}; + +/* + * Load @objfile, attach its struct_ops map @handler (which publishes the + * handler) and register a 'B' entry named @entry that references it, with + * @flags as the entry's register-string flags. + */ +static int bpf_case_start_flags(struct bpf_case *c, const char *objfile, + const char *handler, const char *entry, + const char *flags) +{ + struct bpf_map *map; + + c->obj = NULL; + c->link = NULL; + c->entry = entry; + + c->obj = bpf_object__open_file(objfile, NULL); + if (!c->obj || libbpf_get_error(c->obj)) { + fprintf(stderr, "open %s failed\n", objfile); + c->obj = NULL; + return -1; + } + if (bpf_object__load(c->obj)) { + fprintf(stderr, "load %s failed (check dmesg for the verifier log)\n", + objfile); + goto fail; + } + map = bpf_object__find_map_by_name(c->obj, handler); + if (!map) { + fprintf(stderr, "no struct_ops map '%s' in %s\n", handler, objfile); + goto fail; + } + c->link = bpf_map__attach_struct_ops(map); + if (!c->link || libbpf_get_error(c->link)) { + fprintf(stderr, "attach struct_ops '%s' failed\n", handler); + c->link = NULL; + goto fail; + } + if (register_entry(entry, handler, flags)) { + fprintf(stderr, "register 'B' entry '%s' failed\n", entry); + goto fail; + } + return 0; + +fail: + bpf_link__destroy(c->link); + bpf_object__close(c->obj); + c->obj = NULL; + c->link = NULL; + return -1; +} + +static int bpf_case_start(struct bpf_case *c, const char *objfile, + const char *handler, const char *entry) +{ + return bpf_case_start_flags(c, objfile, handler, entry, NULL); +} + +static void bpf_case_stop(struct bpf_case *c) +{ + unregister(c->entry); + bpf_link__destroy(c->link); + bpf_object__close(c->obj); +} + +/* Activate @handler, run @target and check it produced @expect. */ +static int run_case(const char *objfile, const char *handler, + const char *entry, const char *target, const char *expect) +{ + struct bpf_case c; + int ret; + + if (bpf_case_start(&c, objfile, handler, entry)) + return -1; + ret = check_output(target, expect); + bpf_case_stop(&c); + return ret; +} + +FIXTURE(bpf_handler) { + char obj[PATH_MAX]; /* struct_ops object of the case under test */ +}; + +FIXTURE_SETUP(bpf_handler) +{ + char src[PATH_MAX]; + const char *why = bpf_handler_unsupported(); + + if (why) + SKIP(return, "%s", why); + + /* Shared test interpreter. */ + ASSERT_EQ(artifact_path(src, sizeof(src), "binfmt_bpf_interp"), 0); + ASSERT_EQ(copy_file(src, INTERP_PATH), 0); +} + +FIXTURE_TEARDOWN(bpf_handler) +{ + unlink(INTERP_PATH); +} + +/* The match program matches a synthetic header, the load program routes it. */ +TEST_F(bpf_handler, fixed_interpreter) +{ + ASSERT_EQ(create_fake_elf(AARCH64_PATH, EM_AARCH64), 0); + ASSERT_EQ(artifact_path(self->obj, sizeof(self->obj), + "bpf_interp.bpf.o"), 0); + EXPECT_EQ(run_case(self->obj, "bpf_interp", "test_bpf_interp", + AARCH64_PATH, EXPECT), 0); + unlink(AARCH64_PATH); +} + +/* A "$ORIGIN/..." PT_INTERP resolved to an interpreter next to the binary. */ +TEST_F(bpf_handler, origin_relative_interpreter) +{ + char src[PATH_MAX], app[PATH_MAX], interp[PATH_MAX]; + char dir[] = RELOC_TEMPLATE; + + ASSERT_NE(mkdtemp(dir), NULL); + snprintf(app, sizeof(app), "%s/app", dir); + snprintf(interp, sizeof(interp), "%s/binfmt_bpf_interp", dir); + ASSERT_EQ(artifact_path(src, sizeof(src), "binfmt_bpf_app"), 0); + ASSERT_EQ(copy_file(src, app), 0); + ASSERT_EQ(copy_file(INTERP_PATH, interp), 0); + + ASSERT_EQ(artifact_path(self->obj, sizeof(self->obj), + "nix_origin.bpf.o"), 0); + EXPECT_EQ(run_case(self->obj, "nix_origin", "test_bpf_origin", + app, EXPECT), 0); + + unlink(app); + unlink(interp); + rmdir(dir); +} + +/* A transparent dispatch: the process presents as the binary, not the interp. */ +TEST_F(bpf_handler, transparent_dispatch) +{ + char src[PATH_MAX], cmd[PATH_MAX + 16]; + + /* Probe for transparent-mode support via its static counterpart. */ + if (!binfmt_flag_supported('T')) + SKIP(return, "kernel without transparent mode"); + + ASSERT_EQ(artifact_path(src, sizeof(src), "binfmt_transparent_interp"), 0); + ASSERT_EQ(copy_file(src, TRANS_INTERP), 0); + ASSERT_EQ(create_fake_elf(TRANS_PATH, EM_RISCV), 0); + + setenv("BINFMT_TEST_BINARY", TRANS_PATH, 1); + snprintf(cmd, sizeof(cmd), "%s argone argtwo", TRANS_PATH); + ASSERT_EQ(artifact_path(self->obj, sizeof(self->obj), + "transparent.bpf.o"), 0); + EXPECT_EQ(run_case(self->obj, "transparent", "test_bpf_transparent", + cmd, TRANS_EXPECT), 0); + + unlink(TRANS_PATH); + unlink(TRANS_INTERP); +} + +/* A per-exec loader substitution: the payload runs as a native exec. */ +TEST_F(bpf_handler, loader_substitution) +{ + char src[PATH_MAX], loader[PATH_MAX]; + struct bpf_case c; + int status; + + if (find_loader(loader, sizeof(loader))) + SKIP(return, "cannot determine own PT_INTERP"); + + ASSERT_EQ(copy_file(loader, LOADER_INTERP), 0); + ASSERT_EQ(artifact_path(src, sizeof(src), "binfmt_loader_payload"), 0); + ASSERT_EQ(copy_file(src, LOADER_PATH), 0); + ASSERT_EQ(patch_file(LOADER_PATH, EI_PAD, LOADER_MARKER, + strlen(LOADER_MARKER)), 0); + ASSERT_EQ(artifact_path(self->obj, sizeof(self->obj), + "loader.bpf.o"), 0); + + setenv("BINFMT_TEST_BINARY", LOADER_PATH, 1); + setenv("BINFMT_TEST_INTERP", LOADER_INTERP, 1); + + ASSERT_EQ(bpf_case_start(&c, self->obj, "loader", "test_bpf_loader"), 0); + status = run_payload(LOADER_PATH); + bpf_case_stop(&c); + EXPECT_EQ(status, 0); + + unsetenv("BINFMT_TEST_INTERP"); + unlink(LOADER_PATH); + unlink(LOADER_INTERP); +} + +/* The errno an exec of @path fails with, 0 if it succeeded. */ +static int exec_errno(const char *path) +{ + int status; + pid_t pid; + + pid = fork(); + if (pid == 0) { + execl(path, path, (char *)NULL); + _exit(errno); + } + if (pid < 0 || waitpid(pid, &status, 0) != pid || !WIFEXITED(status)) + return -1; + return WEXITSTATUS(status); +} + +/* Install a copy of the bound-interpreter test binary at @path. */ +static int install_interp(const char *path) +{ + char src[PATH_MAX]; + + if (artifact_path(src, sizeof(src), "binfmt_bind_interp")) + return -1; + return copy_file(src, path); +} + +/* Bind @path to @entry under @name, the '+' command of a disabled entry. */ +static int entry_bind(const char *entry, const char *name, const char *path) +{ + char cmd[PATH_MAX]; + + snprintf(cmd, sizeof(cmd), "+%s %s\n", name, path); + return entry_command(entry, cmd); +} + +FIXTURE(bound_interp) { + char obj[PATH_MAX]; + struct bpf_case c; + bool started; +}; + +FIXTURE_SETUP(bound_interp) +{ + const char *why = bpf_handler_unsupported(); + + if (why) + SKIP(return, "%s", why); + if (!binfmt_flag_supported('D')) { + ASSERT_EQ(errno, EINVAL); + SKIP(return, "kernel without the 'D' flag"); + } + + ASSERT_EQ(install_interp(BIND_FIRST), 0); + ASSERT_EQ(install_interp(BIND_SECOND), 0); + + ASSERT_EQ(artifact_path(self->obj, sizeof(self->obj), + "interp_bind.bpf.o"), 0); + + /* + * Registered disabled, so it cannot be matched yet and can still be + * given interpreters. Each path is resolved once, by its write(2); + * from here on the entry holds the files themselves. + */ + ASSERT_EQ(bpf_case_start_flags(&self->c, self->obj, "interp_bind", + "test_interp_bind", "D"), 0); + self->started = true; + + ASSERT_EQ(entry_bind("test_interp_bind", "first", BIND_FIRST), 0); + ASSERT_EQ(entry_bind("test_interp_bind", "second", BIND_SECOND), 0); +} + +FIXTURE_TEARDOWN(bound_interp) +{ + if (self->started) + bpf_case_stop(&self->c); + unlink(BIND_FIRST); + unlink(BIND_SECOND); + unlink(AARCH64_PATH); + unlink(BIND_RISCV_PATH); + unlink(BIND_ARM_PATH); +} + +/* Enabling is what makes the configured entry matchable. */ +static int activate(const char *entry) +{ + return entry_command(entry, "1\n"); +} + +/* One entry, one interpreter per guest architecture, picked per exec. */ +TEST_F(bound_interp, selects_by_name) +{ + ASSERT_EQ(create_fake_elf(AARCH64_PATH, EM_AARCH64), 0); + ASSERT_EQ(create_fake_elf(BIND_RISCV_PATH, EM_RISCV), 0); + + /* Disabled, so it does not match and no format claims the binary. */ + EXPECT_EQ(exec_errno(AARCH64_PATH), ENOEXEC); + + ASSERT_EQ(activate("test_interp_bind"), 0); + EXPECT_EQ(check_output(AARCH64_PATH, BIND_EXPECT BIND_FIRST), 0); + EXPECT_EQ(check_output(BIND_RISCV_PATH, BIND_EXPECT BIND_SECOND), 0); +} + +/* What was bound is what runs, whatever the path holds afterwards. */ +TEST_F(bound_interp, path_no_longer_decides) +{ + char other[PATH_MAX]; + + ASSERT_EQ(create_fake_elf(AARCH64_PATH, EM_AARCH64), 0); + ASSERT_EQ(activate("test_interp_bind"), 0); + + /* Bound interpreters are pinned against writes, exactly like 'F'. */ + EXPECT_TRUE(write_denied(BIND_FIRST)); + + /* Replace the path with a different binary: a new file, new inode. */ + ASSERT_EQ(artifact_path(other, sizeof(other), "binfmt_bpf_interp"), 0); + ASSERT_EQ(unlink(BIND_FIRST), 0); + ASSERT_EQ(copy_file(other, BIND_FIRST), 0); + + EXPECT_EQ(check_output(AARCH64_PATH, BIND_EXPECT BIND_FIRST), 0); +} + +/* The entry reports what it bound, under the names it bound them as. */ +TEST_F(bound_interp, entry_reports_bindings) +{ + EXPECT_TRUE(entry_shows("test_interp_bind", + "bpf-interpreter first " BIND_FIRST)); + EXPECT_TRUE(entry_shows("test_interp_bind", + "bpf-interpreter second " BIND_SECOND)); +} + +/* Selecting a name the entry did not bind fails the exec. */ +TEST_F(bound_interp, unbound_name_fails) +{ + ASSERT_EQ(create_fake_elf(BIND_ARM_PATH, EM_ARM), 0); + ASSERT_EQ(activate("test_interp_bind"), 0); + + EXPECT_EQ(exec_errno(BIND_ARM_PATH), ENOENT); +} + +/* Activating seals it: what can be matched cannot be changed. */ +TEST_F(bound_interp, sealed_once_active) +{ + ASSERT_EQ(activate("test_interp_bind"), 0); + + EXPECT_EQ(entry_bind("test_interp_bind", "third", BIND_SECOND), -EBUSY); + EXPECT_FALSE(entry_shows("test_interp_bind", + "bpf-interpreter third " BIND_SECOND)); +} + +/* The seal is for good: disabling the entry again reopens nothing. */ +TEST_F(bound_interp, disable_does_not_unseal) +{ + ASSERT_EQ(activate("test_interp_bind"), 0); + ASSERT_EQ(entry_command("test_interp_bind", "0\n"), 0); + + EXPECT_EQ(entry_bind("test_interp_bind", "third", BIND_SECOND), -EBUSY); +} + +/* An entry registered without 'D' is sealed from the start. */ +TEST_F(bound_interp, born_sealed) +{ + /* A second entry for the handler the fixture already published. */ + ASSERT_EQ(register_entry("test_born_sealed", "interp_bind", NULL), 0); + + EXPECT_EQ(entry_bind("test_born_sealed", "first", BIND_FIRST), -EBUSY); + unregister("test_born_sealed"); +} + +/* A name is bound once; a second use of it is refused. */ +TEST_F(bound_interp, duplicate_name_refused) +{ + EXPECT_EQ(entry_bind("test_interp_bind", "first", BIND_SECOND), -EEXIST); +} + +/* A name is a printable word: the entry file reports 'name path' lines. */ +TEST_F(bound_interp, name_must_be_printable) +{ + /* A control character would forge a line into the entry file. */ + EXPECT_EQ(entry_bind("test_interp_bind", "a\tb", BIND_FIRST), -EINVAL); + EXPECT_EQ(entry_bind("test_interp_bind", "a\nb", BIND_FIRST), -EINVAL); + + /* A space cannot even be spelled: the path starts after the first one. */ + EXPECT_EQ(entry_bind("test_interp_bind", "a b", BIND_FIRST), -EINVAL); +} + +/* The command ends at the write: bytes past an embedded nul are refused. */ +TEST_F(bound_interp, trailing_bytes_refused) +{ + char cmd[PATH_MAX]; + size_t len; + int fd; + + /* entry_command() cannot spell a nul, so write the buffer raw. */ + snprintf(cmd, sizeof(cmd), "+nul %s", BIND_FIRST); + len = strlen(cmd) + 1; + memcpy(cmd + len, "junk", sizeof("junk")); + len += sizeof("junk"); + + fd = open(BINFMT_DIR "/test_interp_bind", O_WRONLY | O_CLOEXEC); + ASSERT_GE(fd, 0); + EXPECT_EQ(write(fd, cmd, len), -1); + EXPECT_EQ(errno, EINVAL); + close(fd); + + EXPECT_FALSE(entry_shows("test_interp_bind", + "bpf-interpreter nul " BIND_FIRST)); +} + +/* An entry binds at most BIND_MAX interpreters. */ +TEST_F(bound_interp, capped_bindings) +{ + char name[16]; + int i; + + /* The fixture bound "first" and "second" already. */ + for (i = 2; i < BIND_MAX; i++) { + snprintf(name, sizeof(name), "n%d", i); + ASSERT_EQ(entry_bind("test_interp_bind", name, BIND_FIRST), 0); + } + EXPECT_EQ(entry_bind("test_interp_bind", "over", BIND_FIRST), -ENOSPC); +} + +TEST_HARNESS_MAIN diff --git a/tools/testing/selftests/exec/binfmt_misc_common.h b/tools/testing/selftests/exec/binfmt_misc_common.h new file mode 100644 index 000000000000..745aff84dc78 --- /dev/null +++ b/tools/testing/selftests/exec/binfmt_misc_common.h @@ -0,0 +1,315 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Helpers shared by the binfmt_misc selftests. */ +#ifndef __SELFTESTS_EXEC_BINFMT_MISC_COMMON_H +#define __SELFTESTS_EXEC_BINFMT_MISC_COMMON_H + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#define BINFMT_DIR "/proc/sys/fs/binfmt_misc" +#define BINFMT_REG BINFMT_DIR "/register" + +/* comm holds 15 usable chars; a read of /proc/self/comm appends a newline. */ +#define TASK_COMM_LEN 16 + +/* The canonical payload argv: run_payload() passes it, the payloads assert it. */ +#define PAYLOAD_ARGV0 "payload-argv0" +#define PAYLOAD_ARG1 "argone" +#define PAYLOAD_ARG2 "argtwo" + +/* Marker the loader tests poke into the payload's e_ident padding. */ +#define LOADER_MARKER "LDRTST" + +/* Exit status run_payload() reports when the exec was refused as unhandled. */ +#define RUN_ENOEXEC 42 + +static inline int copy_file(const char *src, const char *dst) +{ + char buf[4096]; + int in, out; + ssize_t n; + + in = open(src, O_RDONLY); + if (in < 0) + return -1; + /* The tests share /tmp, so never write through a name they don't own. */ + unlink(dst); + out = open(dst, O_WRONLY | O_CREAT | O_EXCL, 0755); + if (out < 0) { + close(in); + return -1; + } + while ((n = read(in, buf, sizeof(buf))) > 0) { + if (write(out, buf, n) != n) { + close(in); + close(out); + return -1; + } + } + close(in); + close(out); + return n < 0 ? -1 : 0; +} + +/* Write @rule to the register file, preserving the write's errno. */ +static inline int write_reg(const char *rule) +{ + int fd, saved; + ssize_t n; + + fd = open(BINFMT_REG, O_WRONLY); + if (fd < 0) + return -1; + n = write(fd, rule, strlen(rule)); + saved = errno; + close(fd); + errno = saved; + return n < 0 ? -1 : 0; +} + +static inline void unregister(const char *name) +{ + char path[PATH_MAX]; + int fd; + + snprintf(path, sizeof(path), BINFMT_DIR "/%s", name); + fd = open(path, O_WRONLY); + if (fd >= 0) { + if (write(fd, "-1", 2) < 0) + ; /* best effort */ + close(fd); + } +} + +/* Write @line to @entry's file, reporting the errno it was refused with. */ +static inline int entry_command(const char *entry, const char *line) +{ + char path[PATH_MAX]; + int fd, retval = 0; + size_t len = strlen(line); + + snprintf(path, sizeof(path), BINFMT_DIR "/%s", entry); + fd = open(path, O_WRONLY | O_CLOEXEC); + if (fd < 0) + return -errno; + if (write(fd, line, len) != (ssize_t)len) + retval = -errno; + close(fd); + return retval; +} + +/* Does @entry's file report @line? */ +static inline bool entry_shows(const char *entry, const char *line) +{ + char path[PATH_MAX], buf[PATH_MAX]; + bool found = false; + FILE *fp; + + snprintf(path, sizeof(path), BINFMT_DIR "/%s", entry); + fp = fopen(path, "r"); + if (!fp) + return false; + while (fgets(buf, sizeof(buf), fp)) { + buf[strcspn(buf, "\n")] = '\0'; + if (!strcmp(buf, line)) { + found = true; + break; + } + } + fclose(fp); + return found; +} + +/* Mount binfmt_misc unless it already is, and report whether it is usable. */ +static inline bool binfmt_misc_available(void) +{ + if (access(BINFMT_REG, F_OK) < 0) + mount("binfmt_misc", BINFMT_DIR, "binfmt_misc", 0, NULL); + return access(BINFMT_REG, F_OK) == 0; +} + +/* Absolute path of @name in the directory this test was built into. */ +static inline int artifact_path(char *out, size_t sz, const char *name) +{ + char exe[PATH_MAX]; + ssize_t n; + + n = readlink("/proc/self/exe", exe, sizeof(exe) - 1); + if (n < 0) + return -1; + exe[n] = '\0'; + if ((size_t)snprintf(out, sz, "%s/%s", dirname(exe), name) >= sz) + return -1; + return 0; +} + +/* Probe kernel support for a registration flag with a throwaway entry. */ +static inline bool binfmt_flag_supported(char flag) +{ + char rule[64]; + + snprintf(rule, sizeof(rule), ":bm_flag_probe:E::bmprobe::/bin/true:%c", + flag); + if (write_reg(rule)) + return false; + unregister("bm_flag_probe"); + return true; +} + +/* + * Run @path with the canonical payload argv and return its exit status, or + * RUN_ENOEXEC when the exec itself was refused as unhandled. + */ +static inline int run_payload(const char *path) +{ + int status; + pid_t pid; + + pid = fork(); + if (pid == 0) { + execl(path, PAYLOAD_ARGV0, PAYLOAD_ARG1, PAYLOAD_ARG2, + (char *)NULL); + _exit(errno == ENOEXEC ? RUN_ENOEXEC : 126); + } + if (pid < 0 || waitpid(pid, &status, 0) != pid || !WIFEXITED(status)) + return -1; + return WEXITSTATUS(status); +} + +/* Does the exe link name @path? */ +static inline bool exe_is(const char *path) +{ + char exe[PATH_MAX], real[PATH_MAX]; + ssize_t n; + + n = readlink("/proc/self/exe", exe, sizeof(exe) - 1); + if (n <= 0 || !realpath(path, real)) + return false; + exe[n] = '\0'; + return !strcmp(exe, real); +} + +/* Is comm @name truncated to what a comm can hold? */ +static inline bool comm_is(const char *name) +{ + char comm[TASK_COMM_LEN + 2], expect[TASK_COMM_LEN]; + ssize_t n; + int fd; + + fd = open("/proc/self/comm", O_RDONLY); + if (fd < 0) + return false; + n = read(fd, comm, sizeof(comm) - 1); + close(fd); + if (n <= 0) + return false; + if (comm[n - 1] == '\n') + n--; + comm[n] = '\0'; + snprintf(expect, sizeof(expect), "%s", name); + return !strcmp(comm, expect); +} + +/* Opening @path for writing has to fail with ETXTBSY. */ +static inline bool write_denied(const char *path) +{ + int fd = open(path, O_WRONLY); + + if (fd >= 0) { + close(fd); + return false; + } + return errno == ETXTBSY; +} + +static inline int patch_file(const char *path, off_t off, const void *data, size_t len) +{ + ssize_t n; + int fd; + + fd = open(path, O_WRONLY); + if (fd < 0) + return -1; + n = pwrite(fd, data, len, off); + close(fd); + return n == (ssize_t)len ? 0 : -1; +} + +/* start_code and end_code are the 26th and 27th fields of /proc/pid/stat. */ +static inline int stat_codes(pid_t pid, unsigned long *start_code, + unsigned long *end_code) +{ + char buf[4096], path[64], *p; + ssize_t n; + int fd, i; + + snprintf(path, sizeof(path), "/proc/%d/stat", pid); + fd = open(path, O_RDONLY); + if (fd < 0) + return -1; + n = read(fd, buf, sizeof(buf) - 1); + close(fd); + if (n <= 0) + return -1; + buf[n] = '\0'; + + /* Skip "pid (comm)", then start_code is the 24th field after it. */ + p = strrchr(buf, ')'); + if (!p) + return -1; + p++; + for (i = 0; i < 23; i++) { + p = strchr(p + 1, ' '); + if (!p) + return -1; + } + if (sscanf(p, " %lu %lu", start_code, end_code) != 2) + return -1; + return 0; +} + +/* Find the system loader through our own PT_INTERP. */ +static inline int find_loader(char *out, size_t sz) +{ + ElfW(Ehdr) eh; + ElfW(Phdr) ph; + int fd, i, ret = -1; + + fd = open("/proc/self/exe", O_RDONLY); + if (fd < 0) + return -1; + if (pread(fd, &eh, sizeof(eh), 0) != sizeof(eh)) + goto out; + for (i = 0; i < eh.e_phnum; i++) { + if (pread(fd, &ph, sizeof(ph), + eh.e_phoff + i * eh.e_phentsize) != sizeof(ph)) + goto out; + if (ph.p_type != PT_INTERP) + continue; + if (!ph.p_filesz || ph.p_filesz > sz) + goto out; + if (pread(fd, out, ph.p_filesz, ph.p_offset) != + (ssize_t)ph.p_filesz) + goto out; + out[ph.p_filesz - 1] = '\0'; + ret = 0; + break; + } +out: + close(fd); + return ret; +} + +#endif /* __SELFTESTS_EXEC_BINFMT_MISC_COMMON_H */ diff --git a/tools/testing/selftests/exec/binfmt_misc_disabled.c b/tools/testing/selftests/exec/binfmt_misc_disabled.c new file mode 100644 index 000000000000..47c9e8a4ee42 --- /dev/null +++ b/tools/testing/selftests/exec/binfmt_misc_disabled.c @@ -0,0 +1,172 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Test the 'D' (register disabled) flag of binfmt_misc. An entry + * registered with it exists but cannot be matched until userspace enables + * it, which splits a registration into create and activate. + * + * Needs root for the registration; no bpf toolchain involved. + */ +#define _GNU_SOURCE +#include +#include + +#include "binfmt_misc_common.h" +#include "kselftest_harness.h" + +#define MAGIC "#DISABLED-SELFTEST#" +#define TARGET_PATH "/tmp/binfmt_disabled_target" +#define INTERP_PATH "/tmp/binfmt_disabled_interp.sh" +#define ENTRY "test_disabled" +#define RULE(flags) ":" ENTRY ":M:0:" MAGIC "::" INTERP_PATH ":" flags + +/* The interpreter exits with a code the harness can recognise. */ +#define EXIT_INTERP 7 + +/* The target only has to carry the magic; it is never actually loaded. */ +static int create_target(void) +{ + char buf[128] = MAGIC "\n"; + int fd; + + unlink(TARGET_PATH); + fd = open(TARGET_PATH, O_WRONLY | O_CREAT | O_EXCL, 0755); + if (fd < 0) + return -1; + if (write(fd, buf, sizeof(buf)) != (ssize_t)sizeof(buf)) { + close(fd); + return -1; + } + close(fd); + return 0; +} + +static int create_interp(void) +{ + char buf[64]; + int fd; + + unlink(INTERP_PATH); + fd = open(INTERP_PATH, O_WRONLY | O_CREAT | O_EXCL, 0755); + if (fd < 0) + return -1; + snprintf(buf, sizeof(buf), "#!/bin/sh\nexit %d\n", EXIT_INTERP); + if (write(fd, buf, strlen(buf)) != (ssize_t)strlen(buf)) { + close(fd); + return -1; + } + return close(fd); +} + +FIXTURE(disabled) { +}; + +FIXTURE_SETUP(disabled) +{ + if (getuid() != 0) + SKIP(return, "test must be run as root"); + if (!binfmt_misc_available()) + SKIP(return, "no binfmt_misc"); + + /* Skip the whole suite on a kernel that does not know 'D'. */ + if (!binfmt_flag_supported('D')) { + ASSERT_EQ(errno, EINVAL); + SKIP(return, "kernel without the 'D' flag"); + } + + ASSERT_EQ(create_interp(), 0); + ASSERT_EQ(create_target(), 0); +} + +FIXTURE_TEARDOWN(disabled) +{ + unregister(ENTRY); + unlink(TARGET_PATH); + unlink(INTERP_PATH); +} + +/* The entry exists but does not dispatch until it is enabled. */ +TEST_F(disabled, inert_until_enabled) +{ + ASSERT_EQ(write_reg(RULE("D")), 0); + EXPECT_TRUE(entry_shows(ENTRY, "disabled")); + + /* Nothing matches it, so no binary format claims the target. */ + EXPECT_EQ(run_payload(TARGET_PATH), RUN_ENOEXEC); + + ASSERT_EQ(entry_command(ENTRY, "1\n"), 0); + EXPECT_TRUE(entry_shows(ENTRY, "enabled")); + EXPECT_EQ(run_payload(TARGET_PATH), EXIT_INTERP); +} + +/* Without 'D' an entry is matchable the moment it is registered. */ +TEST_F(disabled, enabled_without_the_flag) +{ + ASSERT_EQ(write_reg(RULE("")), 0); + EXPECT_TRUE(entry_shows(ENTRY, "enabled")); + EXPECT_EQ(run_payload(TARGET_PATH), EXIT_INTERP); +} + +/* 'D' is spent on the registration: the entry does not report it back. */ +TEST_F(disabled, flag_not_reported) +{ + ASSERT_EQ(write_reg(RULE("D")), 0); + EXPECT_FALSE(entry_shows(ENTRY, "flags: D")); + EXPECT_TRUE(entry_shows(ENTRY, "flags: ")); +} + +/* A disabled entry can be disabled and enabled like any other. */ +TEST_F(disabled, toggles_like_any_entry) +{ + ASSERT_EQ(write_reg(RULE("D")), 0); + + ASSERT_EQ(entry_command(ENTRY, "1\n"), 0); + ASSERT_EQ(run_payload(TARGET_PATH), EXIT_INTERP); + ASSERT_EQ(entry_command(ENTRY, "0\n"), 0); + EXPECT_EQ(run_payload(TARGET_PATH), RUN_ENOEXEC); + ASSERT_EQ(entry_command(ENTRY, "1\n"), 0); + EXPECT_EQ(run_payload(TARGET_PATH), EXIT_INTERP); +} + +/* 'D' composes with the invocation flags a static entry can carry. */ +TEST_F(disabled, composes_with_invocation_flags) +{ + ASSERT_EQ(write_reg(RULE("PD")), 0); + EXPECT_TRUE(entry_shows(ENTRY, "disabled")); + EXPECT_TRUE(entry_shows(ENTRY, "flags: P")); +} + +/* '-1' to the status file sweeps a staged entry with everything else. */ +TEST_F(disabled, removed_by_remove_all) +{ + int fd; + + ASSERT_EQ(write_reg(RULE("D")), 0); + EXPECT_TRUE(entry_shows(ENTRY, "disabled")); + + fd = open(BINFMT_DIR "/status", O_WRONLY | O_CLOEXEC); + ASSERT_GE(fd, 0); + ASSERT_EQ(write(fd, "-1", 2), 2); + close(fd); + + EXPECT_NE(access(BINFMT_DIR "/" ENTRY, F_OK), 0); +} + +/* A file handle held across a removal cannot resurrect the entry. */ +TEST_F(disabled, no_resurrection_after_remove) +{ + int fd; + + ASSERT_EQ(write_reg(RULE("D")), 0); + fd = open(BINFMT_DIR "/" ENTRY, O_WRONLY | O_CLOEXEC); + ASSERT_GE(fd, 0); + + ASSERT_EQ(write(fd, "-1", 2), 2); + EXPECT_NE(access(BINFMT_DIR "/" ENTRY, F_OK), 0); + + /* Accepted like any toggle of a removed entry, but publishes nothing. */ + EXPECT_EQ(write(fd, "1", 1), 1); + EXPECT_EQ(run_payload(TARGET_PATH), RUN_ENOEXEC); + close(fd); +} + +TEST_HARNESS_MAIN diff --git a/tools/testing/selftests/exec/binfmt_misc_loader.c b/tools/testing/selftests/exec/binfmt_misc_loader.c new file mode 100644 index 000000000000..1e14dcd274af --- /dev/null +++ b/tools/testing/selftests/exec/binfmt_misc_loader.c @@ -0,0 +1,372 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Test the 'L' (loader substitution) flag of binfmt_misc. A matched + * binary runs as the MAIN image - a fully native exec - with the + * registered interpreter substituted for its PT_INTERP. The payload + * (binfmt_loader_payload) asserts the native identity from inside. + * + * The substitute is a copy of the system loader found via our own + * PT_INTERP; magic matching pokes a marker into the ELF header's + * e_ident padding, which kernel and loader ignore. + * + * Needs root for the registration; no bpf toolchain involved. + */ +#define _GNU_SOURCE +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "binfmt_misc_common.h" +#include "kselftest_harness.h" + +#define ENTRY "test_loader" +#define INTERP_PATH "/tmp/binfmt_loader_interp" +#define MOVED_PATH INTERP_PATH ".moved" +#define TARGET_PATH "/tmp/binfmt_loader_target.ldrtest" +#define STATIC_PATH "/tmp/binfmt_loader_static.ldrtest" +#define FOREIGN_PATH "/tmp/binfmt_loader_foreign.ldrtest" +#define SCRIPT_PATH "/tmp/binfmt_loader_script.ldrtest" +#define M_RULE ":" ENTRY ":M:9:" LOADER_MARKER "::" INTERP_PATH ":L" +#define E_RULE ":" ENTRY ":E::ldrtest::" INTERP_PATH ":L" +#define FL_RULE ":" ENTRY ":E::ldrtest::" INTERP_PATH ":FL" + +/* Execute the binary from an inaccessible O_CLOEXEC memfd. */ +static int run_memfd(const char *path) +{ + int status; + pid_t pid; + + pid = fork(); + if (pid == 0) { + char *argv[] = { PAYLOAD_ARGV0, PAYLOAD_ARG1, PAYLOAD_ARG2, NULL }; + char buf[4096]; + int in, mfd; + ssize_t n; + + mfd = memfd_create("loader-test", MFD_CLOEXEC); + in = open(path, O_RDONLY); + if (mfd < 0 || in < 0) + _exit(125); + while ((n = read(in, buf, sizeof(buf))) > 0) + if (write(mfd, buf, n) != n) + _exit(125); + close(in); + setenv("BINFMT_TEST_MEMFD", "1", 1); + unsetenv("BINFMT_TEST_BINARY"); + syscall(SYS_execveat, mfd, "", argv, environ, AT_EMPTY_PATH); + _exit(126); + } + if (pid < 0 || waitpid(pid, &status, 0) != pid || !WIFEXITED(status)) + return -1; + return WEXITSTATUS(status); +} + +/* + * The differentiator against the transparent mode: at PTRACE_EVENT_EXEC + * the identity is already complete - exe, auxv and the stat code markers + * are mutually consistent with no window a debugger could observe. + */ +static int ptrace_probe(const char *target) +{ + unsigned long auxv[2 * 64], base = 0, entry = 0, at_flags = 0; + unsigned long start_code = 0, end_code = 0; + int status, fd, execfd_seen = 0, failed = 0; + char path[64], buf[PATH_MAX]; + ssize_t n; + pid_t pid; + int i; + + pid = fork(); + if (pid == 0) { + ptrace(PTRACE_TRACEME, 0, NULL, NULL); + raise(SIGSTOP); + execl(target, PAYLOAD_ARGV0, PAYLOAD_ARG1, PAYLOAD_ARG2, (char *)NULL); + _exit(126); + } + if (pid < 0) + return -1; + if (waitpid(pid, &status, 0) != pid || !WIFSTOPPED(status)) + goto fail_kill; + if (ptrace(PTRACE_SETOPTIONS, pid, NULL, (void *)PTRACE_O_TRACEEXEC)) + goto fail_kill; + if (ptrace(PTRACE_CONT, pid, NULL, NULL)) + goto fail_kill; + if (waitpid(pid, &status, 0) != pid || !WIFSTOPPED(status) || + status >> 8 != (SIGTRAP | (PTRACE_EVENT_EXEC << 8))) { + fprintf(stderr, "no exec stop (status %#x)\n", status); + goto fail_kill; + } + + snprintf(path, sizeof(path), "/proc/%d/exe", pid); + n = readlink(path, buf, sizeof(buf) - 1); + if (n <= 0) { + failed = 1; + } else { + buf[n] = '\0'; + if (strcmp(buf, target)) { + fprintf(stderr, "exe at exec stop: %s\n", buf); + failed = 1; + } + } + + snprintf(path, sizeof(path), "/proc/%d/auxv", pid); + fd = open(path, O_RDONLY); + if (fd < 0) { + n = -1; + } else { + n = read(fd, auxv, sizeof(auxv)); + close(fd); + } + if (n <= 0) { + failed = 1; + n = 0; + } + for (i = 0; i + 1 < (int)(n / sizeof(unsigned long)); i += 2) { + switch (auxv[i]) { + case AT_BASE: + base = auxv[i + 1]; + break; + case AT_ENTRY: + entry = auxv[i + 1]; + break; + case AT_FLAGS: + at_flags = auxv[i + 1]; + break; + case AT_EXECFD: + execfd_seen = 1; + break; + } + } + + if (stat_codes(pid, &start_code, &end_code)) + failed = 1; + + if (!base || execfd_seen || at_flags) { + fprintf(stderr, "auxv at exec stop not native\n"); + failed = 1; + } + if (!start_code || entry < start_code || entry >= end_code) { + fprintf(stderr, "auxv/stat inconsistent at exec stop\n"); + failed = 1; + } + + if (ptrace(PTRACE_CONT, pid, NULL, NULL)) + goto fail_kill; + if (waitpid(pid, &status, 0) != pid || !WIFEXITED(status) || + WEXITSTATUS(status)) + failed = 1; + return failed ? -1 : 0; + +fail_kill: + kill(pid, SIGKILL); + waitpid(pid, &status, 0); + return -1; +} + +FIXTURE(loader) { + bool have_static; +}; + +FIXTURE_SETUP(loader) +{ + unsigned short foreign_machine = 0xdead; + char src[PATH_MAX], loader[PATH_MAX]; + + if (getuid() != 0) + SKIP(return, "test must be run as root"); + if (!binfmt_misc_available()) + SKIP(return, "no binfmt_misc"); + if (find_loader(loader, sizeof(loader))) + SKIP(return, "cannot determine own PT_INTERP"); + + ASSERT_EQ(copy_file(loader, INTERP_PATH), 0); + + ASSERT_EQ(artifact_path(src, sizeof(src), "binfmt_loader_payload"), 0); + ASSERT_EQ(copy_file(src, TARGET_PATH), 0); + ASSERT_EQ(patch_file(TARGET_PATH, EI_PAD, LOADER_MARKER, + strlen(LOADER_MARKER)), 0); + + /* The same payload with a machine type this kernel cannot load. */ + ASSERT_EQ(copy_file(src, FOREIGN_PATH), 0); + ASSERT_EQ(patch_file(FOREIGN_PATH, EI_PAD, LOADER_MARKER, + strlen(LOADER_MARKER)), 0); + ASSERT_EQ(patch_file(FOREIGN_PATH, offsetof(ElfW(Ehdr), e_machine), + &foreign_machine, sizeof(foreign_machine)), 0); + + self->have_static = + artifact_path(src, sizeof(src), "binfmt_loader_payload_static") == 0 && + copy_file(src, STATIC_PATH) == 0; + + setenv("BINFMT_TEST_BINARY", TARGET_PATH, 1); + setenv("BINFMT_TEST_INTERP", INTERP_PATH, 1); + + /* Everything below needs the flag; find out once. */ + if (write_reg(E_RULE)) { + ASSERT_EQ(errno, EINVAL); + SKIP(return, "kernel without the 'L' flag"); + } + unregister(ENTRY); +} + +FIXTURE_TEARDOWN(loader) +{ + unregister(ENTRY); + if (access(MOVED_PATH, F_OK) == 0) + rename(MOVED_PATH, INTERP_PATH); + unlink(TARGET_PATH); + unlink(STATIC_PATH); + unlink(FOREIGN_PATH); + unlink(SCRIPT_PATH); + unlink(INTERP_PATH); +} + +/* Grammar sanity check: the same entry without 'L' has to register. */ +TEST_F(loader, plain_entry_registers) +{ + ASSERT_EQ(write_reg(":" ENTRY ":E::ldrtest::" INTERP_PATH ":"), 0); +} + +/* 'L' is a native exec: every classic-dispatch flag is rejected. */ +TEST_F(loader, rejects_classic_flags) +{ + static const char * const combos[] = { "LT", "LP", "LC", "LO" }; + char rule[PATH_MAX]; + unsigned int i; + + for (i = 0; i < ARRAY_SIZE(combos); i++) { + int rc; + + snprintf(rule, sizeof(rule), + ":" ENTRY ":E::ldrtest::" INTERP_PATH ":%s", combos[i]); + rc = write_reg(rule); + EXPECT_EQ(rc, -1) + TH_LOG("'%s' was not rejected", combos[i]); + if (rc == 0) { + unregister(ENTRY); + continue; + } + EXPECT_EQ(errno, EINVAL); + } +} + +/* + * Without 'F' the interpreter is opened when the binary is executed, so a + * relative path would be resolved against the caller's working directory. + */ +TEST_F(loader, rejects_relative_interpreter) +{ + static const char * const flags[] = { "L", "C" }; + char rule[PATH_MAX]; + unsigned int i; + + for (i = 0; i < ARRAY_SIZE(flags); i++) { + int rc; + + snprintf(rule, sizeof(rule), + ":" ENTRY ":E::ldrtest::binfmt_loader_interp:%s", + flags[i]); + rc = write_reg(rule); + EXPECT_EQ(rc, -1) + TH_LOG("'%s' accepted a relative interpreter", flags[i]); + if (rc == 0) { + unregister(ENTRY); + continue; + } + EXPECT_EQ(errno, EINVAL); + } +} + +TEST_F(loader, extension_matched) +{ + ASSERT_EQ(write_reg(E_RULE), 0); + EXPECT_EQ(run_payload(TARGET_PATH), 0); +} + +TEST_F(loader, magic_matched) +{ + ASSERT_EQ(write_reg(M_RULE), 0); + EXPECT_EQ(run_payload(TARGET_PATH), 0); +} + +/* + * The differentiator against the transparent mode: at PTRACE_EVENT_EXEC the + * identity is already complete, with no window a debugger could observe. + */ +TEST_F(loader, exec_stop_consistency) +{ + ASSERT_EQ(write_reg(E_RULE), 0); + EXPECT_EQ(ptrace_probe(TARGET_PATH), 0); +} + +/* A binary without PT_INTERP drops the override and runs natively. */ +TEST_F(loader, static_binary_runs_natively) +{ + if (!self->have_static) + SKIP(return, "no static payload built"); + + ASSERT_EQ(write_reg(E_RULE), 0); + setenv("BINFMT_TEST_BINARY", STATIC_PATH, 1); + setenv("BINFMT_TEST_STATIC", "1", 1); + EXPECT_EQ(run_payload(STATIC_PATH), 0); + unsetenv("BINFMT_TEST_STATIC"); + setenv("BINFMT_TEST_BINARY", TARGET_PATH, 1); +} + +/* + * A '#!' file that matched an 'L' entry is claimed by binfmt_script, which + * sits ahead of binfmt_elf. The substitute the entry staged has to be + * released when the interpreter replaces the file, not leaked. + */ +TEST_F(loader, script_claims_the_file) +{ + static const char script[] = "#!/bin/sh\nexit 0\n"; + int fd; + + unlink(SCRIPT_PATH); + fd = open(SCRIPT_PATH, O_WRONLY | O_CREAT | O_EXCL, 0755); + ASSERT_GE(fd, 0); + ASSERT_EQ(write(fd, script, sizeof(script) - 1), + (ssize_t)sizeof(script) - 1); + ASSERT_EQ(close(fd), 0); + + ASSERT_EQ(write_reg(E_RULE), 0); + EXPECT_EQ(run_payload(SCRIPT_PATH), 0); + + /* A leaked substitute keeps its write denial on the loader. */ + fd = open(INTERP_PATH, O_WRONLY); + EXPECT_GE(fd, 0) + TH_LOG("loader still write denied (errno %d)", errno); + if (fd >= 0) + close(fd); +} + +/* Nothing needs the binary's path, so an inaccessible fd works. */ +TEST_F(loader, inaccessible_memfd) +{ + ASSERT_EQ(write_reg(M_RULE), 0); + EXPECT_EQ(run_memfd(TARGET_PATH), 0); +} + +/* The whole exec of a wrong-arch binary fails as if unhandled. */ +TEST_F(loader, foreign_arch_enoexec) +{ + ASSERT_EQ(write_reg(M_RULE), 0); + EXPECT_EQ(run_payload(FOREIGN_PATH), RUN_ENOEXEC); +} + +/* 'F' pre-opens the substitute, so it survives losing its path. */ +TEST_F(loader, fixed_interpreter_survives_rename) +{ + ASSERT_EQ(write_reg(FL_RULE), 0); + ASSERT_EQ(rename(INTERP_PATH, MOVED_PATH), 0); + EXPECT_EQ(run_payload(TARGET_PATH), 0); +} + +TEST_HARNESS_MAIN diff --git a/tools/testing/selftests/exec/binfmt_misc_selfpin.c b/tools/testing/selftests/exec/binfmt_misc_selfpin.c new file mode 100644 index 000000000000..5286b0604eed --- /dev/null +++ b/tools/testing/selftests/exec/binfmt_misc_selfpin.c @@ -0,0 +1,158 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * An 'F' entry keeps its interpreter open for as long as the entry exists, + * and the entry only goes away when the binfmt_misc superblock is destroyed. + * An interpreter that lives on a mount which in turn keeps that superblock + * alive therefore pins the instance that owns it, and nothing can break the + * cycle. Check the two ways userspace could arrange for that: an interpreter + * on the binfmt_misc instance itself, and one on a filesystem stacked on it. + * + * Runs unprivileged in a user namespace; binfmt_misc is FS_USERNS_MOUNT. + */ +#define _GNU_SOURCE +#include +#include +#include +#include +#include + +#include "../filesystems/utils.h" +#include "kselftest_harness.h" + +#define MNT "/tmp/binfmt_selfpin" +#define BACKING "/tmp/binfmt_selfpin_back" +#define LOWER BACKING "/lower" +#define MERGED "/tmp/binfmt_selfpin_merged" + +#define MAGIC "\\xde\\xad" +#define RULE(interp) ":selfpin:M::" MAGIC "::" interp ":F" +/* Not on the instance, and unlike /bin/true it always exists. */ +#define INTERP "/proc/self/exe" + +#define OPTS_MAX (3 * PATH_MAX + 64) + +static int ensure_dir(const char *path) +{ + if (mkdir(path, 0755) && errno != EEXIST) + return -1; + return 0; +} + +/* Write @rule to this instance's register file, preserving write(2)'s errno. */ +static int register_at(struct __test_metadata *_metadata, const char *rule) +{ + int fd, saved; + ssize_t n; + + fd = open(MNT "/register", O_WRONLY); + ASSERT_GE(fd, 0); + n = write(fd, rule, strlen(rule)); + saved = errno; + close(fd); + errno = saved; + return n < 0 ? -1 : 0; +} + +/* + * Mount an overlay over @lower using a private upper/work pair, so the two + * mounts this test performs cannot interfere with each other and neither + * overlaps the lower layer. + */ +static int mount_overlay(const char *lower, int nr) +{ + char opts[OPTS_MAX], upper[PATH_MAX], work[PATH_MAX]; + + snprintf(upper, sizeof(upper), "%s/upper%d", BACKING, nr); + snprintf(work, sizeof(work), "%s/work%d", BACKING, nr); + if (mkdir(upper, 0755) || mkdir(work, 0755)) + return -1; + + snprintf(opts, sizeof(opts), "lowerdir=%s,upperdir=%s,workdir=%s", + lower, upper, work); + return mount("ovl", MERGED, "overlay", 0, opts); +} + +FIXTURE(selfpin) { +}; + +FIXTURE_SETUP(selfpin) +{ + /* setup_userns() exits rather than returns if this is not there. */ + if (access("/proc/self/ns/user", F_OK)) + SKIP(return, "kernel without user namespaces"); + ASSERT_EQ(setup_userns(), 0); + + ASSERT_EQ(ensure_dir(MNT), 0); + if (mount("binfmt_misc", MNT, "binfmt_misc", 0, NULL)) { + int saved = errno; + + /* Teardown doesn't run when setup skips, so clean up here. */ + rmdir(MNT); + SKIP(return, "no binfmt_misc: %s", strerror(saved)); + } +} + +FIXTURE_TEARDOWN(selfpin) +{ + /* The namespaces go with the process; just don't litter /tmp. */ + umount2(MERGED, MNT_DETACH); + umount2(BACKING, MNT_DETACH); + umount2(MNT, MNT_DETACH); + rmdir(MERGED); + rmdir(BACKING); + rmdir(MNT); +} + +/* + * The instance's own files are regular files the mounter owns, so they can be + * made executable. Opening one for exec still has to fail, otherwise the entry + * pins the very superblock it lives in. + */ +TEST_F(selfpin, interpreter_on_the_instance) +{ + ASSERT_EQ(chmod(MNT "/status", 0755), 0); + + ASSERT_NE(register_at(_metadata, RULE(MNT "/status")), 0); + EXPECT_EQ(errno, EACCES); +} + +/* Same for an entry file rather than one of the control files. */ +TEST_F(selfpin, interpreter_on_an_entry) +{ + ASSERT_EQ(register_at(_metadata, ":victim:M::" MAGIC "::" INTERP ":"), 0); + ASSERT_EQ(chmod(MNT "/victim", 0755), 0); + + ASSERT_NE(register_at(_metadata, RULE(MNT "/victim")), 0); + EXPECT_EQ(errno, EACCES); +} + +/* + * A stacking filesystem holds a private clone of each layer for its whole + * lifetime, so an instance used as a layer can be pinned by an interpreter + * that does not live on it at all. Refuse to be a layer. + */ +TEST_F(selfpin, refuses_to_be_stacked_on) +{ + ASSERT_EQ(ensure_dir(BACKING), 0); + ASSERT_EQ(mount("tmpfs", BACKING, "tmpfs", 0, NULL), 0); + ASSERT_EQ(mkdir(LOWER, 0755), 0); + ASSERT_EQ(ensure_dir(MERGED), 0); + + /* Nothing to prove unless overlayfs works here at all. */ + if (mount_overlay(LOWER, 1)) { + if (errno == ENODEV || errno == EPERM) + SKIP(return, "no unprivileged overlayfs"); + SKIP(return, "overlayfs unusable here: %s", strerror(errno)); + } + ASSERT_EQ(umount(MERGED), 0); + + EXPECT_NE(mount_overlay(MNT, 2), 0); +} + +/* An ordinary interpreter still registers with 'F'. */ +TEST_F(selfpin, ordinary_interpreter_still_works) +{ + EXPECT_EQ(register_at(_metadata, RULE(INTERP)), 0); +} + +TEST_HARNESS_MAIN diff --git a/tools/testing/selftests/exec/binfmt_misc_transparent.c b/tools/testing/selftests/exec/binfmt_misc_transparent.c new file mode 100644 index 000000000000..2ebf73de8018 --- /dev/null +++ b/tools/testing/selftests/exec/binfmt_misc_transparent.c @@ -0,0 +1,95 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Test the static transparent flag 'T' of binfmt_misc. A magic-matched + * binary is dispatched to an interpreter with the argument vector left + * untouched, the binary passed through AT_EXECFD and mm->exe_file labeled + * with the binary. The asserting interpreter (binfmt_transparent_interp) + * verifies the constructed identity from inside the process and exits 0. + * + * Needs root for the registration; no bpf toolchain involved. + */ +#define _GNU_SOURCE +#include +#include + +#include "binfmt_misc_common.h" +#include "kselftest_harness.h" + +#define MAGIC "#TRANSPARENT-SELFTEST#" +#define TARGET_PATH "/tmp/binfmt_transparent_target" +#define INTERP_PATH "/tmp/binfmt_transparent_interp" +#define ENTRY "test_transparent" +#define RULE(flags) ":" ENTRY ":M:0:" MAGIC "::" INTERP_PATH ":" flags + +/* The target only has to carry the magic; it is never actually loaded. */ +static int create_target(void) +{ + char buf[128] = MAGIC "\n"; + int fd; + + unlink(TARGET_PATH); + fd = open(TARGET_PATH, O_WRONLY | O_CREAT | O_EXCL, 0755); + if (fd < 0) + return -1; + if (write(fd, buf, sizeof(buf)) != (ssize_t)sizeof(buf)) { + close(fd); + return -1; + } + close(fd); + return 0; +} + +FIXTURE(transparent) { +}; + +FIXTURE_SETUP(transparent) +{ + char src[PATH_MAX]; + + if (getuid() != 0) + SKIP(return, "test must be run as root"); + if (!binfmt_misc_available()) + SKIP(return, "no binfmt_misc"); + + ASSERT_EQ(artifact_path(src, sizeof(src), "binfmt_transparent_interp"), 0); + ASSERT_EQ(copy_file(src, INTERP_PATH), 0); + ASSERT_EQ(create_target(), 0); + + /* Skip the whole suite on a kernel that does not know 'T'. */ + if (!binfmt_flag_supported('T')) { + ASSERT_EQ(errno, EINVAL); + SKIP(return, "kernel without the 'T' flag"); + } +} + +FIXTURE_TEARDOWN(transparent) +{ + unregister(ENTRY); + unlink(TARGET_PATH); + unlink(INTERP_PATH); +} + +/* Grammar sanity check: the same entry without 'T' has to register. */ +TEST_F(transparent, plain_entry_registers) +{ + ASSERT_EQ(write_reg(RULE("")), 0); +} + +/* 'T' preserves the whole argv, so combining it with 'P' is rejected. */ +TEST_F(transparent, rejects_preserve_argv0) +{ + ASSERT_NE(write_reg(RULE("TP")), 0); + EXPECT_EQ(errno, EINVAL); +} + +/* The interpreter asserts the identity the kernel built for it. */ +TEST_F(transparent, dispatch) +{ + ASSERT_EQ(write_reg(RULE("T")), 0); + + setenv("BINFMT_TEST_BINARY", TARGET_PATH, 1); + setenv("BINFMT_TEST_ARGV0", PAYLOAD_ARGV0, 1); + EXPECT_EQ(run_payload(TARGET_PATH), 0); +} + +TEST_HARNESS_MAIN diff --git a/tools/testing/selftests/exec/binfmt_transparent_interp.c b/tools/testing/selftests/exec/binfmt_transparent_interp.c new file mode 100644 index 000000000000..d4c4a538c9aa --- /dev/null +++ b/tools/testing/selftests/exec/binfmt_transparent_interp.c @@ -0,0 +1,112 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Asserting interpreter for the transparent binfmt_misc mode. It runs in + * place of the dispatched binary and verifies the identity the kernel + * constructed: the aux vector contract, the exe link, argv, cmdline, comm + * and the write denial on the binary. BINFMT_TEST_BINARY names the binary; + * the harness execs it with the arguments "argone argtwo". Prints + * TRANSPARENT_OK and exits 0 when every check holds. + */ +#define _GNU_SOURCE +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "binfmt_misc_common.h" +#include "kselftest.h" + +#ifndef AT_FLAGS_TRANSPARENT_INTERP +#define AT_FLAGS_TRANSPARENT_INTERP (1 << 1) +#endif + +static int fail; + +static void ok(int cond, const char *what) +{ + if (!cond) { + fprintf(stderr, "TRANSPARENT_FAIL: %s (errno %d)\n", what, errno); + fail = 1; + } +} + +int main(int argc, char **argv) +{ + const char *binary = getenv("BINFMT_TEST_BINARY"); + const char *argv0 = getenv("BINFMT_TEST_ARGV0"); + char expect[PATH_MAX + 32], buf[PATH_MAX]; + unsigned long execfd; + struct stat stb, stfd; + const char *want[3]; + const char *base; + size_t expect_len, i; + int fd, have_stb, have_stfd; + ssize_t n; + + if (!binary) { + fprintf(stderr, "TRANSPARENT_FAIL: BINFMT_TEST_BINARY unset\n"); + return 1; + } + /* Distinct from the binary path, so a classic argv splice is caught. */ + want[0] = argv0 ? argv0 : binary; + want[1] = PAYLOAD_ARG1; + want[2] = PAYLOAD_ARG2; + + /* The aux vector announces the transparent contract. */ + ok(getauxval(AT_FLAGS) & AT_FLAGS_TRANSPARENT_INTERP, + "AT_FLAGS lacks AT_FLAGS_TRANSPARENT_INTERP"); + + /* AT_EXECFD refers to the very file that was executed. */ + execfd = getauxval(AT_EXECFD); + ok(execfd > 2, "no AT_EXECFD"); + have_stb = !stat(binary, &stb); + ok(have_stb, "cannot stat the binary"); + have_stfd = !fstat((int)execfd, &stfd); + ok(have_stfd, "cannot fstat AT_EXECFD"); + ok(have_stb && have_stfd && stb.st_dev == stfd.st_dev && + stb.st_ino == stfd.st_ino, "AT_EXECFD is not the binary"); + + /* The exe link names the binary, not this interpreter. */ + ok(exe_is(binary), "/proc/self/exe is not the binary"); + + /* argv arrived unspliced. */ + ok(argc == (int)ARRAY_SIZE(want), "argv was rewritten"); + for (i = 0; i < ARRAY_SIZE(want) && i < (size_t)argc; i++) + ok(!strcmp(argv[i], want[i]), "argv was rewritten"); + + /* And so did the kernel's copy of it: the same strings, NUL separated. */ + for (i = 0, expect_len = 0; i < ARRAY_SIZE(want); i++) { + size_t len = strlen(want[i]) + 1; + + if (expect_len + len > sizeof(expect)) { + ok(0, "argv does not fit the expectation buffer"); + break; + } + memcpy(expect + expect_len, want[i], len); + expect_len += len; + } + fd = open("/proc/self/cmdline", O_RDONLY); + n = fd >= 0 ? read(fd, buf, sizeof(buf)) : -1; + if (fd >= 0) + close(fd); + ok(n == (ssize_t)expect_len && !memcmp(buf, expect, expect_len), + "/proc/self/cmdline was rewritten"); + + /* comm is the binary's basename. */ + base = strrchr(binary, '/'); + base = base ? base + 1 : binary; + ok(comm_is(base), "comm is not the binary's basename"); + + /* The binary is write-denied while it runs, like a direct exec. */ + ok(write_denied(binary), "binary is writable while running"); + ok(write_denied("/proc/self/exe"), "exe link is writable while running"); + + if (!fail) + printf("TRANSPARENT_OK\n"); + return fail; +} diff --git a/tools/testing/selftests/exec/bpf_interp.bpf.c b/tools/testing/selftests/exec/bpf_interp.bpf.c new file mode 100644 index 000000000000..8df2d2d01e25 --- /dev/null +++ b/tools/testing/selftests/exec/bpf_interp.bpf.c @@ -0,0 +1,61 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * binfmt_misc_ops handler for the selftest's fixed-interpreter case: match a + * 64-bit aarch64 ELF header from the prefetched buffer and route it to a fixed + * interpreter chosen by the program. This is the portable, self-contained + * equivalent of routing a foreign binary to an emulator: it matches + * programmatically and computes the interpreter, but points at a test binary + * the harness installs rather than a system emulator. + */ +#include "vmlinux.h" +#include +#include + +char _license[] SEC("license") = "GPL"; + +#define EI_CLASS 4 +#define ELFCLASS64 2 +#define EM_AARCH64 183 + +extern int bpf_binprm_set_interp(struct linux_binprm *bprm, const char *path, + size_t path__sz) __ksym; + +/* + * A magic-style decision needs nothing beyond the prefetched bprm->buf, + * even though the match program could read the file. + */ +SEC("struct_ops.s/match") +bool BPF_PROG(bpf_interp_match, struct linux_binprm *bprm) +{ + __u16 machine; + + if (bprm->buf[0] != 0x7f || bprm->buf[1] != 'E' || + bprm->buf[2] != 'L' || bprm->buf[3] != 'F' || + bprm->buf[EI_CLASS] != ELFCLASS64) + return false; + + /* e_machine is a 16-bit little-endian field at offset 18. */ + machine = (__u8)bprm->buf[18] | ((__u16)(__u8)bprm->buf[19] << 8); + return machine == EM_AARCH64; +} + +SEC("struct_ops.s/load") +int BPF_PROG(bpf_interp_load, struct linux_binprm *bprm) +{ + /* + * Keep the path on the (writable) stack: bpf_binprm_set_interp() takes + * a sized memory arg and the verifier rejects a read-only .rodata + * buffer for it. The harness installs the interpreter at this path. + */ + char interp[] = "/tmp/binfmt_bpf_interp"; + + /* @path__sz includes the terminating NUL; 0 commits the selection. */ + return bpf_binprm_set_interp(bprm, interp, sizeof(interp)); +} + +SEC(".struct_ops.link") +struct binfmt_misc_ops bpf_interp = { + .match = (void *)bpf_interp_match, + .load = (void *)bpf_interp_load, + .name = "bpf_interp", +}; diff --git a/tools/testing/selftests/exec/config b/tools/testing/selftests/exec/config index c308079867b3..ea359a929ae8 100644 --- a/tools/testing/selftests/exec/config +++ b/tools/testing/selftests/exec/config @@ -1,2 +1,12 @@ CONFIG_BLK_DEV=y CONFIG_BLK_DEV_LOOP=y +CONFIG_BINFMT_MISC=y +CONFIG_BINFMT_MISC_BPF=y +CONFIG_BPF_JIT=y +CONFIG_BPF_SYSCALL=y +CONFIG_DEBUG_INFO=y +CONFIG_DEBUG_INFO_BTF=y +CONFIG_DEBUG_INFO_DWARF4=y +CONFIG_OVERLAY_FS=y +CONFIG_TMPFS=y +CONFIG_USER_NS=y diff --git a/tools/testing/selftests/exec/interp_bind.bpf.c b/tools/testing/selftests/exec/interp_bind.bpf.c new file mode 100644 index 000000000000..1ce45cca215f --- /dev/null +++ b/tools/testing/selftests/exec/interp_bind.bpf.c @@ -0,0 +1,76 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * binfmt_misc_ops handler for the selftest's bound-interpreter case: one + * handler, one entry, an interpreter per guest architecture - each bound to + * a file when the entry was registered rather than to a path resolved at + * exec time. The load program names the one it wants; a name the entry did + * not bind fails the exec, which the harness checks too. + */ +#include "vmlinux.h" +#include +#include + +char _license[] SEC("license") = "GPL"; + +#define EI_CLASS 4 +#define ELFCLASS64 2 +#define E_MACHINE_OFF 18 +#define EM_ARM 40 +#define EM_AARCH64 183 +#define EM_RISCV 243 + +extern int bpf_binprm_select_interp(struct linux_binprm *bprm, + const char *name, size_t name__sz) __ksym; + +/* The guest architecture of a 64-bit ELF, or zero if it is not one. */ +static __u16 elf_machine(struct linux_binprm *bprm) +{ + if (bprm->buf[0] != 0x7f || bprm->buf[1] != 'E' || + bprm->buf[2] != 'L' || bprm->buf[3] != 'F' || + bprm->buf[EI_CLASS] != ELFCLASS64) + return 0; + + /* Little-endian 16-bit field, read byte-wise for the verifier. */ + return (__u8)bprm->buf[E_MACHINE_OFF] | + ((__u16)(__u8)bprm->buf[E_MACHINE_OFF + 1] << 8); +} + +SEC("struct_ops.s/match") +bool BPF_PROG(interp_bind_match, struct linux_binprm *bprm) +{ + __u16 machine = elf_machine(bprm); + + return machine == EM_AARCH64 || machine == EM_RISCV || + machine == EM_ARM; +} + +SEC("struct_ops.s/load") +int BPF_PROG(interp_bind_load, struct linux_binprm *bprm) +{ + /* + * Names, not paths: each one selects a file the entry pre-opened, so + * nothing is resolved here or later, in any namespace. The buffers + * are on the stack because the verifier rejects .rodata for a sized + * memory argument. + */ + char first[] = "first"; + char second[] = "second"; + char unbound[] = "unbound"; + + switch (elf_machine(bprm)) { + case EM_AARCH64: + return bpf_binprm_select_interp(bprm, first, sizeof(first)); + case EM_RISCV: + return bpf_binprm_select_interp(bprm, second, sizeof(second)); + } + + /* The entry bound nothing under this name: -ENOENT fails the exec. */ + return bpf_binprm_select_interp(bprm, unbound, sizeof(unbound)); +} + +SEC(".struct_ops.link") +struct binfmt_misc_ops interp_bind = { + .match = (void *)interp_bind_match, + .load = (void *)interp_bind_load, + .name = "interp_bind", +}; diff --git a/tools/testing/selftests/exec/loader.bpf.c b/tools/testing/selftests/exec/loader.bpf.c new file mode 100644 index 000000000000..108e51dd4961 --- /dev/null +++ b/tools/testing/selftests/exec/loader.bpf.c @@ -0,0 +1,56 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * binfmt_misc_ops handler for the loader-substitution case: match the + * marker the harness poked into the payload's e_ident padding and ask for + * the selected interpreter to be substituted for the binary's PT_INTERP, + * so the binary itself runs as a fully native exec. + */ +#include "vmlinux.h" +#include +#include + +char _license[] SEC("license") = "GPL"; + +#define EI_CLASS 4 +#define EI_PAD 9 +#define ELFCLASS64 2 + +extern int bpf_binprm_set_interp(struct linux_binprm *bprm, const char *path, + size_t path__sz) __ksym; +extern int bpf_binprm_set_flags(struct linux_binprm *bprm, + enum bpf_binprm_flags flags) __ksym; + +SEC("struct_ops.s/match") +bool BPF_PROG(loader_match, struct linux_binprm *bprm) +{ + if (bprm->buf[0] != 0x7f || bprm->buf[1] != 'E' || + bprm->buf[2] != 'L' || bprm->buf[3] != 'F' || + bprm->buf[EI_CLASS] != ELFCLASS64) + return false; + + /* The harness marks the payload with "LDRTST" at EI_PAD. */ + return bprm->buf[EI_PAD + 0] == 'L' && bprm->buf[EI_PAD + 1] == 'D' && + bprm->buf[EI_PAD + 2] == 'R' && bprm->buf[EI_PAD + 3] == 'T' && + bprm->buf[EI_PAD + 4] == 'S' && bprm->buf[EI_PAD + 5] == 'T'; +} + +SEC("struct_ops.s/load") +int BPF_PROG(loader_load, struct linux_binprm *bprm) +{ + char interp[] = "/tmp/binfmt_loader_interp"; + int err; + + err = bpf_binprm_set_flags(bprm, BPF_BINPRM_LOADER); + if (err) + return err; + + /* @path__sz includes the terminating NUL; 0 commits the selection. */ + return bpf_binprm_set_interp(bprm, interp, sizeof(interp)); +} + +SEC(".struct_ops.link") +struct binfmt_misc_ops loader = { + .match = (void *)loader_match, + .load = (void *)loader_load, + .name = "loader", +}; diff --git a/tools/testing/selftests/exec/nix_origin.bpf.c b/tools/testing/selftests/exec/nix_origin.bpf.c new file mode 100644 index 000000000000..378e22a4c43b --- /dev/null +++ b/tools/testing/selftests/exec/nix_origin.bpf.c @@ -0,0 +1,224 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * nix_origin.bpf.c - $ORIGIN-relative PT_INTERP resolution + * + * A binfmt_misc_ops handler that makes relocatable (Nix-style) ELF + * binaries work: if PT_INTERP starts with "$ORIGIN/", the loader is + * resolved relative to the directory of the binary being executed and + * selected via bpf_binprm_set_interp(). The match program reads the + * program headers itself, so anything else never commits to this + * handler and passes through untouched. + * + * Activate with: + * bpftool struct_ops register nix_origin.bpf.o /sys/fs/bpf + * echo ':nix-origin:B::::nix_origin:' > /proc/sys/fs/binfmt_misc/register + */ +#include "vmlinux.h" +#include +#include + +char _license[] SEC("license") = "GPL"; + +#define PATH_MAX 4096 +#define EI_CLASS 4 +#define ELFCLASSXX 2 /* ELFCLASS64; flip to 1 for 32-bit */ +#define PT_INTERP 3 +#define MAX_PHDRS 64 + +#define ORIGIN "$ORIGIN" +#define ORIGIN_LEN (sizeof(ORIGIN) - 1) + +#define ENOENT 2 +#define ENOEXEC 8 +#define ENAMETOOLONG 36 + +extern int bpf_dynptr_from_file(struct file *file, __u32 flags, + struct bpf_dynptr *ptr__uninit) __ksym; +extern int bpf_dynptr_file_discard(struct bpf_dynptr *dynptr) __ksym; +extern int bpf_path_d_path(const struct path *path, char *buf, + size_t buf__sz) __ksym; +extern int bpf_binprm_set_interp(struct linux_binprm *bprm, const char *path, + size_t path__sz) __ksym; + +struct scratch { + char interp[PATH_MAX]; /* PT_INTERP as embedded in the binary */ + char path[PATH_MAX]; /* d_path of the binary, becomes the result */ +}; + +/* Keyed by pid: execs run concurrently and the programs can sleep. */ +struct { + __uint(type, BPF_MAP_TYPE_HASH); + __uint(max_entries, 512); + __type(key, __u64); + __type(value, struct scratch); +} scratch_map SEC(".maps"); + +static const struct scratch zero_scratch; + +/* An ELF64 binary per the prefetched header? */ +static bool is_elf64(struct linux_binprm *bprm) +{ + return bprm->buf[0] == 0x7f && bprm->buf[1] == 'E' && + bprm->buf[2] == 'L' && bprm->buf[3] == 'F' && + bprm->buf[EI_CLASS] == ELFCLASSXX; +} + +/* Locate PT_INTERP; false if the file has none or looks malformed. */ +static bool find_pt_interp(struct bpf_dynptr *dp, struct elf64_phdr *phdr) +{ + struct elf64_hdr ehdr; + bool found = false; + int i; + + if (bpf_dynptr_read(&ehdr, sizeof(ehdr), dp, 0, 0)) + return false; + if (ehdr.e_phentsize != sizeof(struct elf64_phdr)) + return false; + + bpf_for(i, 0, ehdr.e_phnum) { + if (i >= MAX_PHDRS) + break; + if (bpf_dynptr_read(phdr, sizeof(*phdr), dp, + ehdr.e_phoff + i * sizeof(*phdr), 0)) + return false; + if (phdr->p_type == PT_INTERP) { + found = true; + break; + } + } + return found; +} + +/* + * An ELF64 binary whose PT_INTERP starts with "$ORIGIN/" is ours. The + * match can sleep and read the file, so the decision is made here and + * regular binaries never commit to this handler: later binfmt_misc + * entries and binfmt_elf see them as if we did not exist. + */ +SEC("struct_ops.s/match") +bool BPF_PROG(nix_origin_match, struct linux_binprm *bprm) +{ + char prefix[ORIGIN_LEN + 1] = {}; + struct elf64_phdr phdr; + struct bpf_dynptr dp; + bool ours = false; + + if (!is_elf64(bprm)) + return false; + + /* The dynptr must be discarded on every path once requested. */ + if (bpf_dynptr_from_file(bprm->file, 0, &dp)) + goto out; + if (find_pt_interp(&dp, &phdr) && + phdr.p_filesz > ORIGIN_LEN + 1 && + !bpf_dynptr_read(prefix, sizeof(prefix), &dp, phdr.p_offset, 0)) + ours = !bpf_strncmp(prefix, sizeof(prefix), ORIGIN "/"); +out: + bpf_dynptr_file_discard(&dp); + return ours; +} + +/* + * The match is committed and already vetted the "$ORIGIN/" prefix, so + * everything here reads the file again from scratch: -ENOEXEC only + * covers a binary that changed under us and stopped being ours. + */ +SEC("struct_ops.s/load") +int BPF_PROG(nix_origin_load, struct linux_binprm *bprm) +{ + __u32 isz, sfx, rsz, slash; + struct elf64_phdr phdr; + struct bpf_dynptr dp; + struct scratch *sc; + __u64 id; + int ret = -ENOEXEC, len, i; + + if (bpf_dynptr_from_file(bprm->file, 0, &dp)) + goto out; + + if (!find_pt_interp(&dp, &phdr)) + goto out; + + isz = phdr.p_filesz; + if (isz <= ORIGIN_LEN + 1 || isz >= sizeof(sc->interp)) + goto out; + /* + * The range check above compiles to a test on a zero-extended copy of + * the u64 p_filesz, so the verifier does not carry the bound to the + * dynptr_read() length below ("unbounded memory access"). Mask isz to + * the buffer size (a power of two) and force the masked value to be + * materialized with a barrier so the read uses the bounded register. + */ + isz &= sizeof(sc->interp) - 1; + barrier_var(isz); + + id = bpf_get_current_pid_tgid(); + if (bpf_map_update_elem(&scratch_map, &id, &zero_scratch, BPF_ANY)) + goto out; + sc = bpf_map_lookup_elem(&scratch_map, &id); + if (!sc) + goto out_del; + + if (bpf_dynptr_read(sc->interp, isz, &dp, phdr.p_offset, 0)) + goto out_del; + if (sc->interp[isz - 1] != '\0') + goto out_del; + + /* Not "$ORIGIN/..." anymore? Then it is not ours anymore either. */ + if (sc->interp[0] != '$' || sc->interp[1] != 'O' || + sc->interp[2] != 'R' || sc->interp[3] != 'I' || + sc->interp[4] != 'G' || sc->interp[5] != 'I' || + sc->interp[6] != 'N' || sc->interp[7] != '/') + goto out_del; + + /* + * From here on resolution failures fail the exec instead of falling + * back to binfmt_elf, which would resolve the literal "$ORIGIN/..." + * relative to the caller's cwd. + */ + ret = -ENOENT; + len = bpf_path_d_path(&bprm->file->f_path, sc->path, sizeof(sc->path)); + if (len <= 0 || len > sizeof(sc->path)) + goto out_del; + /* Unreachable or unlinked ("... (deleted)") binaries can't resolve. */ + if (sc->path[0] != '/') + goto out_del; + + /* $ORIGIN = dirname of the binary. */ + slash = 0; + bpf_for(i, 1, len - 1) { + if (i >= sizeof(sc->path)) + break; + if (sc->path[i] == '/') + slash = i; + } + + /* Splice the suffix (leading '/' and NUL included) onto the dir. */ + sfx = isz - ORIGIN_LEN; + rsz = slash + sfx; + if (rsz > sizeof(sc->path)) { + ret = -ENAMETOOLONG; + goto out_del; + } + bpf_for(i, 0, sfx) { + __u32 s = ORIGIN_LEN + i, d = slash + i; + + if (s >= sizeof(sc->interp) || d >= sizeof(sc->path)) + break; + sc->path[d] = sc->interp[s]; + } + + ret = bpf_binprm_set_interp(bprm, sc->path, rsz); +out_del: + bpf_map_delete_elem(&scratch_map, &id); +out: + bpf_dynptr_file_discard(&dp); + return ret; +} + +SEC(".struct_ops.link") +struct binfmt_misc_ops nix_origin = { + .match = (void *)nix_origin_match, + .load = (void *)nix_origin_load, + .name = "nix_origin", +}; diff --git a/tools/testing/selftests/exec/transparent.bpf.c b/tools/testing/selftests/exec/transparent.bpf.c new file mode 100644 index 000000000000..7632019ebe69 --- /dev/null +++ b/tools/testing/selftests/exec/transparent.bpf.c @@ -0,0 +1,57 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * binfmt_misc_ops handler for the transparent-mode case: match a synthetic + * riscv ELF header and run the asserting interpreter transparently - the + * argument vector untouched, the binary in AT_EXECFD and mm->exe_file + * labeled with the binary. + */ +#include "vmlinux.h" +#include +#include + +char _license[] SEC("license") = "GPL"; + +#define EI_CLASS 4 +#define ELFCLASS64 2 +#define EM_RISCV 243 + +extern int bpf_binprm_set_interp(struct linux_binprm *bprm, const char *path, + size_t path__sz) __ksym; +extern int bpf_binprm_set_flags(struct linux_binprm *bprm, + enum bpf_binprm_flags flags) __ksym; + +SEC("struct_ops.s/match") +bool BPF_PROG(transparent_match, struct linux_binprm *bprm) +{ + __u16 machine; + + if (bprm->buf[0] != 0x7f || bprm->buf[1] != 'E' || + bprm->buf[2] != 'L' || bprm->buf[3] != 'F' || + bprm->buf[EI_CLASS] != ELFCLASS64) + return false; + + /* e_machine is a 16-bit little-endian field at offset 18. */ + machine = (__u8)bprm->buf[18] | ((__u16)(__u8)bprm->buf[19] << 8); + return machine == EM_RISCV; +} + +SEC("struct_ops.s/load") +int BPF_PROG(transparent_load, struct linux_binprm *bprm) +{ + char interp[] = "/tmp/binfmt_transparent_interp"; + int err; + + err = bpf_binprm_set_flags(bprm, BPF_BINPRM_TRANSPARENT); + if (err) + return err; + + /* @path__sz includes the terminating NUL; 0 commits the selection. */ + return bpf_binprm_set_interp(bprm, interp, sizeof(interp)); +} + +SEC(".struct_ops.link") +struct binfmt_misc_ops transparent = { + .match = (void *)transparent_match, + .load = (void *)transparent_load, + .name = "transparent", +}; diff --git a/tools/testing/selftests/filesystems/.gitignore b/tools/testing/selftests/filesystems/.gitignore index a78f894157de..9eb185fb2f9d 100644 --- a/tools/testing/selftests/filesystems/.gitignore +++ b/tools/testing/selftests/filesystems/.gitignore @@ -6,3 +6,4 @@ file_stressor anon_inode_test kernfs_test idmapped_tmpfile +ustat_test diff --git a/tools/testing/selftests/filesystems/Makefile b/tools/testing/selftests/filesystems/Makefile index a7ec2ba2dd83..03be337c1f35 100644 --- a/tools/testing/selftests/filesystems/Makefile +++ b/tools/testing/selftests/filesystems/Makefile @@ -1,7 +1,7 @@ # SPDX-License-Identifier: GPL-2.0 CFLAGS += $(KHDR_INCLUDES) -TEST_GEN_PROGS := devpts_pts file_stressor anon_inode_test kernfs_test fclog +TEST_GEN_PROGS := devpts_pts file_stressor anon_inode_test kernfs_test fclog ustat_test TEST_GEN_PROGS += idmapped_tmpfile TEST_GEN_PROGS_EXTENDED := dnotify_test diff --git a/tools/testing/selftests/filesystems/failfs/.gitignore b/tools/testing/selftests/filesystems/failfs/.gitignore new file mode 100644 index 000000000000..cd3b5d884d7e --- /dev/null +++ b/tools/testing/selftests/filesystems/failfs/.gitignore @@ -0,0 +1,2 @@ +# SPDX-License-Identifier: GPL-2.0-only +failfs_test diff --git a/tools/testing/selftests/filesystems/failfs/Makefile b/tools/testing/selftests/filesystems/failfs/Makefile new file mode 100644 index 000000000000..3c5d98b4fe72 --- /dev/null +++ b/tools/testing/selftests/filesystems/failfs/Makefile @@ -0,0 +1,5 @@ +# SPDX-License-Identifier: GPL-2.0 +CFLAGS += -Wall -O2 -g $(KHDR_INCLUDES) +TEST_GEN_PROGS := failfs_test + +include ../../lib.mk diff --git a/tools/testing/selftests/filesystems/failfs/failfs_test.c b/tools/testing/selftests/filesystems/failfs/failfs_test.c new file mode 100644 index 000000000000..29a3c294127e --- /dev/null +++ b/tools/testing/selftests/filesystems/failfs/failfs_test.c @@ -0,0 +1,585 @@ +// SPDX-License-Identifier: GPL-2.0 +#define _GNU_SOURCE +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "../../kselftest_harness.h" + +#ifndef __NR_fchroot +#define __NR_fchroot 472 +#endif + +#ifndef FD_PIDFS_ROOT +#define FD_PIDFS_ROOT -10002 +#endif + +#ifndef FD_NSFS_ROOT +#define FD_NSFS_ROOT -10003 +#endif + +#ifndef FD_FAILFS_ROOT +#define FD_FAILFS_ROOT -10004 +#endif + +#define NOBODY_UID 65534 + +/* Child sentinel exit code: the exec was blocked as expected. */ +#define FAILFS_EXEC_BLOCKED 99 + +/* Stack for the CLONE_FS helper in fchroot_sentinel_shared_fs_struct. */ +#define FAILFS_CLONE_STACK (64 * 1024) + +static int sys_fchroot(int fd, unsigned int flags) +{ + return syscall(__NR_fchroot, fd, flags); +} + +/* + * Raw syscall: glibc's getcwd() rejects the kernel's "(unreachable)" + * result and falls back to a generic implementation. + */ +static long sys_getcwd(char *buf, size_t size) +{ + return syscall(__NR_getcwd, buf, size); +} + +static int drop_to_nobody(void) +{ + return setresuid(NOBODY_UID, NOBODY_UID, NOBODY_UID); +} + +/* Parked CLONE_FS child; dies with its parent so it never leaks. */ +static int failfs_park(void *arg) +{ + pid_t parent = (pid_t)(long)arg; + + prctl(PR_SET_PDEATHSIG, SIGKILL); + /* The parent may have died before the death signal was armed. */ + if (getppid() != parent) + _exit(0); + pause(); + return 0; +} + +/* Is fd a dynamically linked ELF with an absolute PT_INTERP interpreter? */ +static int elf_has_absolute_interp(int fd) +{ + ElfW(Ehdr) ehdr; + ElfW(Phdr) phdr; + char interp; + int i; + + if (pread(fd, &ehdr, sizeof(ehdr), 0) != sizeof(ehdr)) + return 0; + if (memcmp(ehdr.e_ident, ELFMAG, SELFMAG) != 0) + return 0; + + for (i = 0; i < ehdr.e_phnum; i++) { + if (pread(fd, &phdr, sizeof(phdr), + ehdr.e_phoff + i * sizeof(phdr)) != sizeof(phdr)) + return 0; + if (phdr.p_type != PT_INTERP) + continue; + if (pread(fd, &interp, 1, phdr.p_offset) != 1) + return 0; + return interp == '/'; + } + + return 0; +} + +TEST(fchdir_sentinel) +{ + char buf[PATH_MAX]; + int fd; + + ASSERT_EQ(fchdir(FD_FAILFS_ROOT), 0); + + /* The working directory is unreachable from the process root. */ + ASSERT_GT(sys_getcwd(buf, sizeof(buf)), 0); + ASSERT_EQ(strncmp(buf, "(unreachable)", 13), 0); + + /* Every AT_FDCWD-relative lookup fails. */ + ASSERT_EQ(openat(AT_FDCWD, "foo", O_RDONLY), -1); + ASSERT_EQ(errno, EOPNOTSUPP); + ASSERT_EQ(openat(AT_FDCWD, ".", O_RDONLY), -1); + ASSERT_EQ(errno, EOPNOTSUPP); + ASSERT_EQ(openat(AT_FDCWD, "..", O_RDONLY), -1); + ASSERT_EQ(errno, EOPNOTSUPP); + ASSERT_EQ(openat(AT_FDCWD, "foo", O_WRONLY | O_CREAT, 0600), -1); + ASSERT_EQ(errno, EOPNOTSUPP); + + /* The cwd cannot be pinned by following /proc/self/cwd into it. */ + ASSERT_EQ(open("/proc/self/cwd", O_PATH), -1); + ASSERT_EQ(errno, EOPNOTSUPP); + + /* The root is untouched so absolute lookups keep working... */ + fd = open("/", O_RDONLY | O_DIRECTORY); + ASSERT_GE(fd, 0); + ASSERT_EQ(close(fd), 0); + + /* ... and the working directory can be recovered. */ + ASSERT_EQ(chdir("/"), 0); + ASSERT_GT(sys_getcwd(buf, sizeof(buf)), 0); + ASSERT_EQ(strcmp(buf, "/"), 0); +} + +TEST(fchdir_rejects_other_sentinels) +{ + ASSERT_EQ(fchdir(FD_PIDFS_ROOT), -1); + ASSERT_EQ(errno, EBADF); + ASSERT_EQ(fchdir(FD_NSFS_ROOT), -1); + ASSERT_EQ(errno, EBADF); + ASSERT_EQ(fchdir(-10009), -1); + ASSERT_EQ(errno, EBADF); +} + +TEST(fchroot_flags) +{ + int fd; + + ASSERT_EQ(sys_fchroot(FD_FAILFS_ROOT, 1), -1); + ASSERT_EQ(errno, EINVAL); + + fd = open("/", O_PATH | O_DIRECTORY); + ASSERT_GE(fd, 0); + ASSERT_EQ(sys_fchroot(fd, 1), -1); + ASSERT_EQ(errno, EINVAL); + ASSERT_EQ(close(fd), 0); +} + +TEST(fchroot_bad_fd) +{ + ASSERT_EQ(sys_fchroot(-1, 0), -1); + ASSERT_EQ(errno, EBADF); + + /* Only FD_FAILFS_ROOT is a valid sentinel. */ + ASSERT_EQ(sys_fchroot(FD_PIDFS_ROOT, 0), -1); + ASSERT_EQ(errno, EBADF); + ASSERT_EQ(sys_fchroot(FD_NSFS_ROOT, 0), -1); + ASSERT_EQ(errno, EBADF); +} + +TEST(fchroot_notdir) +{ + int fd; + + fd = open("/proc/self/status", O_RDONLY); + ASSERT_GE(fd, 0); + ASSERT_EQ(sys_fchroot(fd, 0), -1); + ASSERT_EQ(errno, ENOTDIR); + ASSERT_EQ(close(fd), 0); +} + +TEST(fchroot_realfd_requires_cap) +{ + int fd; + + if (geteuid() == 0) + ASSERT_EQ(drop_to_nobody(), 0); + + fd = open("/", O_PATH | O_DIRECTORY); + ASSERT_GE(fd, 0); + ASSERT_EQ(sys_fchroot(fd, 0), -1); + ASSERT_EQ(errno, EPERM); + ASSERT_EQ(close(fd), 0); +} + +TEST(fchroot_realfd) +{ + char template[] = "/tmp/failfs_test.XXXXXX"; + char path[PATH_MAX]; + struct stat st; + int tmpfd, dfd, fd; + + if (geteuid() != 0) + SKIP(return, "fchroot() with a regular fd requires CAP_SYS_CHROOT"); + + tmpfd = open("/tmp", O_PATH | O_DIRECTORY); + ASSERT_GE(tmpfd, 0); + + ASSERT_NE(mkdtemp(template), NULL); + snprintf(path, sizeof(path), "%s/canary", template); + fd = open(path, O_WRONLY | O_CREAT, 0600); + ASSERT_GE(fd, 0); + ASSERT_EQ(close(fd), 0); + + dfd = open(template, O_PATH | O_DIRECTORY); + ASSERT_GE(dfd, 0); + ASSERT_EQ(sys_fchroot(dfd, 0), 0); + ASSERT_EQ(close(dfd), 0); + + ASSERT_EQ(stat("/canary", &st), 0); + + /* Best-effort cleanup: dirfd-anchored I/O works with the new root. */ + snprintf(path, sizeof(path), "%s/canary", template + strlen("/tmp/")); + unlinkat(tmpfd, path, 0); + unlinkat(tmpfd, template + strlen("/tmp/"), AT_REMOVEDIR); +} + +TEST(fchroot_sentinel) +{ + char template[] = "/tmp/failfs_test.XXXXXX"; + struct stat realroot, st; + struct statfs sfs; + char buf[PATH_MAX]; + int procfd, tmpfd, dfd, fd; + struct { + struct file_handle handle; + unsigned char f_handle[MAX_HANDLE_SZ]; + } fh; + int mntid; + ssize_t ret; + + if (geteuid() != 0) + SKIP(return, "privileged fchroot(FD_FAILFS_ROOT) requires CAP_SYS_CHROOT"); + + ASSERT_EQ(stat("/", &realroot), 0); + procfd = open("/proc", O_PATH | O_DIRECTORY); + ASSERT_GE(procfd, 0); + tmpfd = open("/tmp", O_PATH | O_DIRECTORY); + ASSERT_GE(tmpfd, 0); + ASSERT_NE(mkdtemp(template), NULL); + dfd = open(template, O_RDONLY | O_DIRECTORY); + ASSERT_GE(dfd, 0); + + ASSERT_EQ(sys_fchroot(FD_FAILFS_ROOT, 0), 0); + + /* Absolute lookups fail. */ + ASSERT_EQ(open("/etc/passwd", O_RDONLY), -1); + ASSERT_EQ(errno, EOPNOTSUPP); + ASSERT_EQ(mkdir("/foo", 0700), -1); + ASSERT_EQ(errno, EOPNOTSUPP); + + /* + * The root cannot be referenced at all - not even an O_PATH open, + * which skips ->permission(), because it lands on the root as a + * jumped walk terminal that ->d_weak_revalidate() refuses. + */ + ASSERT_EQ(open("/", O_RDONLY | O_DIRECTORY), -1); + ASSERT_EQ(errno, EOPNOTSUPP); + ASSERT_EQ(open("/", O_PATH), -1); + ASSERT_EQ(errno, EOPNOTSUPP); + ASSERT_EQ(statfs("/", &sfs), -1); + ASSERT_EQ(errno, EOPNOTSUPP); + + /* + * It cannot be pinned by following /proc/self/root into it either + * (only the root is in failfs here, so self/cwd is still real). + */ + ASSERT_EQ(openat(procfd, "self/root", O_PATH), -1); + ASSERT_EQ(errno, EOPNOTSUPP); + + /* Nor encoded into a file handle. */ + fh.handle.handle_bytes = MAX_HANDLE_SZ; + ASSERT_EQ(name_to_handle_at(AT_FDCWD, "/", &fh.handle, &mntid, 0), -1); + ASSERT_EQ(errno, EOPNOTSUPP); + + /* The working directory is now unreachable from the root. */ + ASSERT_GT(sys_getcwd(buf, sizeof(buf)), 0); + ASSERT_EQ(strncmp(buf, "(unreachable)", 13), 0); + + /* Lookups anchored at real directories keep working. */ + fd = openat(AT_FDCWD, ".", O_RDONLY | O_DIRECTORY); + ASSERT_GE(fd, 0); + ASSERT_EQ(close(fd), 0); + fd = openat(dfd, "canary", O_WRONLY | O_CREAT, 0600); + ASSERT_GE(fd, 0); + ASSERT_EQ(write(fd, "x", 1), 1); + ASSERT_EQ(close(fd), 0); + fd = openat(dfd, "canary", O_RDONLY); + ASSERT_GE(fd, 0); + ASSERT_EQ(close(fd), 0); + + /* ".." walks clamp at the top of the mount tree, not at failfs. */ + fd = openat(AT_FDCWD, "../../../../../../../../../..", O_PATH); + ASSERT_GE(fd, 0); + ASSERT_EQ(fstat(fd, &st), 0); + ASSERT_EQ(st.st_dev, realroot.st_dev); + ASSERT_EQ(st.st_ino, realroot.st_ino); + ASSERT_EQ(close(fd), 0); + + /* readlink of the magic link still works: it does not follow. */ + ret = readlinkat(procfd, "self/root", buf, sizeof(buf) - 1); + ASSERT_GT(ret, 0); + buf[ret] = '\0'; + TH_LOG("/proc/self/root points to '%s'", buf); + /* d_path() names the failfs root synthetically, never as a real path. */ + ASSERT_EQ(strcmp(buf, "failfs:/"), 0); + + /* But following it into failfs is refused. */ + ASSERT_EQ(fstatat(procfd, "self/root", &st, 0), -1); + ASSERT_EQ(errno, EOPNOTSUPP); + + /* Best-effort cleanup via the pre-opened dirfds. */ + unlinkat(dfd, "canary", 0); + unlinkat(tmpfd, template + strlen("/tmp/"), AT_REMOVEDIR); +} + +TEST(fchroot_sentinel_absolute_symlink) +{ + char template[] = "/tmp/failfs_test.XXXXXX"; + int tmpfd, dfd, fd; + + if (geteuid() != 0) + SKIP(return, "privileged fchroot(FD_FAILFS_ROOT) requires CAP_SYS_CHROOT"); + + tmpfd = open("/tmp", O_PATH | O_DIRECTORY); + ASSERT_GE(tmpfd, 0); + ASSERT_NE(mkdtemp(template), NULL); + dfd = open(template, O_RDONLY | O_DIRECTORY); + ASSERT_GE(dfd, 0); + + fd = openat(dfd, "target", O_WRONLY | O_CREAT, 0600); + ASSERT_GE(fd, 0); + ASSERT_EQ(close(fd), 0); + ASSERT_EQ(symlinkat("target", dfd, "rel"), 0); + ASSERT_EQ(symlinkat("/etc", dfd, "abs"), 0); + + ASSERT_EQ(sys_fchroot(FD_FAILFS_ROOT, 0), 0); + + /* Relative symlinks keep resolving within the dirfd-anchored walk... */ + fd = openat(dfd, "rel", O_RDONLY); + ASSERT_GE(fd, 0); + ASSERT_EQ(close(fd), 0); + + /* ... absolute symlinks restart the walk at the failfs root. */ + ASSERT_EQ(openat(dfd, "abs", O_RDONLY), -1); + ASSERT_EQ(errno, EOPNOTSUPP); + + /* Best-effort cleanup via the pre-opened dirfds. */ + unlinkat(dfd, "abs", 0); + unlinkat(dfd, "rel", 0); + unlinkat(dfd, "target", 0); + unlinkat(tmpfd, template + strlen("/tmp/"), AT_REMOVEDIR); +} + +TEST(fchroot_sentinel_unprivileged) +{ + char buf[PATH_MAX]; + + if (geteuid() == 0) + ASSERT_EQ(drop_to_nobody(), 0); + + /* Without no_new_privs entering failfs is not allowed... */ + ASSERT_EQ(sys_fchroot(FD_FAILFS_ROOT, 0), -1); + ASSERT_EQ(errno, EPERM); + + /* ... with no_new_privs set it is allowed. */ + ASSERT_EQ(prctl(PR_SET_NO_NEW_PRIVS, 1, 0, 0, 0), 0); + ASSERT_EQ(sys_fchroot(FD_FAILFS_ROOT, 0), 0); + + ASSERT_EQ(open("/etc/passwd", O_RDONLY), -1); + ASSERT_EQ(errno, EOPNOTSUPP); + + /* The task counts as chrooted: no user namespaces anymore. */ + ASSERT_EQ(unshare(CLONE_NEWUSER), -1); + ASSERT_EQ(errno, EPERM); + + /* With both root and cwd in failfs getcwd() reports "/". */ + ASSERT_EQ(fchdir(FD_FAILFS_ROOT), 0); + ASSERT_GT(sys_getcwd(buf, sizeof(buf)), 0); + ASSERT_EQ(strcmp(buf, "/"), 0); +} + +TEST(fchroot_sentinel_rejected_when_chrooted) +{ + char template[] = "/tmp/failfs_test.XXXXXX"; + int tmpfd; + + if (geteuid() != 0) + SKIP(return, "chroot() requires CAP_SYS_CHROOT"); + + tmpfd = open("/tmp", O_PATH | O_DIRECTORY); + ASSERT_GE(tmpfd, 0); + ASSERT_NE(mkdtemp(template), NULL); + ASSERT_EQ(chroot(template), 0); + ASSERT_EQ(chdir("/"), 0); + + /* Remove the jail while still privileged; sticky /tmp blocks nobody. */ + unlinkat(tmpfd, template + strlen("/tmp/"), AT_REMOVEDIR); + + ASSERT_EQ(drop_to_nobody(), 0); + ASSERT_EQ(prctl(PR_SET_NO_NEW_PRIVS, 1, 0, 0, 0), 0); + + /* An unprivileged chrooted task must not lift its ".." barrier. */ + ASSERT_EQ(sys_fchroot(FD_FAILFS_ROOT, 0), -1); + ASSERT_EQ(errno, EPERM); +} + +TEST(fchroot_sentinel_shared_fs_struct) +{ + char stack[FAILFS_CLONE_STACK]; + pid_t pid; + + if (geteuid() == 0) + ASSERT_EQ(drop_to_nobody(), 0); + + /* A CLONE_FS sibling shares the fs_struct: bump fs->users to 2. */ + pid = clone(failfs_park, stack + sizeof(stack), CLONE_FS | SIGCHLD, + (void *)(long)getpid()); + ASSERT_GE(pid, 0); + + ASSERT_EQ(prctl(PR_SET_NO_NEW_PRIVS, 1, 0, 0, 0), 0); + + /* + * A sibling without no_new_privs could exec a setuid binary with + * the failfs root, so a shared fs_struct is refused even with + * no_new_privs set. + */ + ASSERT_EQ(sys_fchroot(FD_FAILFS_ROOT, 0), -1); + ASSERT_EQ(errno, EINVAL); + + ASSERT_EQ(kill(pid, SIGKILL), 0); + ASSERT_EQ(waitpid(pid, NULL, 0), pid); +} + +TEST(fchroot_sentinel_no_overmount) +{ + if (geteuid() != 0) + SKIP(return, "mounting requires privileges"); + + /* + * Contain the blast radius: if failfs ever regressed and "/" + * resolved to the real root, the tmpfs mount below must not touch + * the host. A private mount namespace keeps it local to this child. + */ + ASSERT_EQ(unshare(CLONE_NEWNS), 0); + ASSERT_EQ(mount(NULL, "/", NULL, MS_REC | MS_PRIVATE, NULL), 0); + + ASSERT_EQ(sys_fchroot(FD_FAILFS_ROOT, 0), 0); + + /* + * Nothing can be mounted on top of the failfs root. It cannot even + * be named as a mount target: resolving "/" is refused before the + * mount machinery (which, failfs being in no mount namespace, would + * reject it anyway) is ever reached. open_tree(OPEN_TREE_CLONE) is + * likewise moot since no fd to the root can be obtained. + */ + ASSERT_EQ(mount("none", "/", "tmpfs", 0, NULL), -1); + ASSERT_EQ(errno, EOPNOTSUPP); +} + +TEST(fchroot_sentinel_setns_escape) +{ + struct stat realroot, st; + int nsfd; + + if (geteuid() != 0) + SKIP(return, "setns() to a mount namespace requires privileges"); + + ASSERT_EQ(stat("/", &realroot), 0); + nsfd = open("/proc/self/ns/mnt", O_RDONLY); + ASSERT_GE(nsfd, 0); + + ASSERT_EQ(sys_fchroot(FD_FAILFS_ROOT, 0), 0); + ASSERT_EQ(open("/etc", O_PATH), -1); + ASSERT_EQ(errno, EOPNOTSUPP); + + /* A mount namespace fd is the key out: it resets root and cwd. */ + ASSERT_EQ(setns(nsfd, CLONE_NEWNS), 0); + ASSERT_EQ(close(nsfd), 0); + + ASSERT_EQ(stat("/", &st), 0); + ASSERT_EQ(st.st_dev, realroot.st_dev); + ASSERT_EQ(st.st_ino, realroot.st_ino); +} + +TEST(fchroot_sentinel_exec) +{ + pid_t pid; + int status; + + if (geteuid() != 0) + SKIP(return, "privileged fchroot(FD_FAILFS_ROOT) requires CAP_SYS_CHROOT"); + + ASSERT_EQ(sys_fchroot(FD_FAILFS_ROOT, 0), 0); + + /* + * Exec in a child: a wrongly successful exec would replace the test + * image and its exit code would not match the sentinel below. + */ + pid = fork(); + ASSERT_GE(pid, 0); + if (pid == 0) { + execl("/bin/true", "true", NULL); + _exit(errno == EOPNOTSUPP ? FAILFS_EXEC_BLOCKED : 1); + } + ASSERT_EQ(waitpid(pid, &status, 0), pid); + ASSERT_TRUE(WIFEXITED(status)); + ASSERT_EQ(WEXITSTATUS(status), FAILFS_EXEC_BLOCKED); +} + +TEST(fchroot_sentinel_exec_interpreter) +{ + static const char * const argv[] = { "failfs_test", NULL }; + static const char * const envp[] = { NULL }; + pid_t pid; + int status, exefd; + + if (geteuid() != 0) + SKIP(return, "privileged fchroot(FD_FAILFS_ROOT) requires CAP_SYS_CHROOT"); + + /* Exec ourselves: the one binary guaranteed to be around. */ + exefd = open("/proc/self/exe", O_RDONLY); + ASSERT_GE(exefd, 0); + if (!elf_has_absolute_interp(exefd)) + SKIP(return, "test binary has no absolute PT_INTERP interpreter"); + + ASSERT_EQ(sys_fchroot(FD_FAILFS_ROOT, 0), 0); + + /* + * The binary itself needs no path lookup - it is executed by fd - + * but loading it fails on opening the absolute PT_INTERP + * interpreter. Run it in a child so a wrongly successful exec does + * not replace the test image and masquerade as a pass. + */ + pid = fork(); + ASSERT_GE(pid, 0); + if (pid == 0) { + syscall(__NR_execveat, exefd, "", argv, envp, AT_EMPTY_PATH); + _exit(errno == EOPNOTSUPP ? FAILFS_EXEC_BLOCKED : 1); + } + ASSERT_EQ(waitpid(pid, &status, 0), pid); + ASSERT_TRUE(WIFEXITED(status)); + ASSERT_EQ(WEXITSTATUS(status), FAILFS_EXEC_BLOCKED); +} + +TEST(fchroot_sentinel_inherited) +{ + pid_t pid; + int status; + + if (geteuid() != 0) + SKIP(return, "privileged fchroot(FD_FAILFS_ROOT) requires CAP_SYS_CHROOT"); + + ASSERT_EQ(sys_fchroot(FD_FAILFS_ROOT, 0), 0); + + pid = fork(); + ASSERT_GE(pid, 0); + if (pid == 0) { + if (open("/etc", O_PATH) != -1 || errno != EOPNOTSUPP) + _exit(1); + _exit(0); + } + ASSERT_EQ(waitpid(pid, &status, 0), pid); + ASSERT_TRUE(WIFEXITED(status)); + ASSERT_EQ(WEXITSTATUS(status), 0); +} + +TEST_HARNESS_MAIN diff --git a/tools/testing/selftests/filesystems/mntns_cleanup/.gitignore b/tools/testing/selftests/filesystems/mntns_cleanup/.gitignore new file mode 100644 index 000000000000..493fbcf8d9ec --- /dev/null +++ b/tools/testing/selftests/filesystems/mntns_cleanup/.gitignore @@ -0,0 +1,2 @@ +# SPDX-License-Identifier: GPL-2.0-only +mntns_cleanup_test diff --git a/tools/testing/selftests/filesystems/mntns_cleanup/Makefile b/tools/testing/selftests/filesystems/mntns_cleanup/Makefile new file mode 100644 index 000000000000..0e09e7030a5c --- /dev/null +++ b/tools/testing/selftests/filesystems/mntns_cleanup/Makefile @@ -0,0 +1,6 @@ +# SPDX-License-Identifier: GPL-2.0 +TEST_GEN_PROGS := mntns_cleanup_test + +CFLAGS += -Wall -O2 -g $(KHDR_INCLUDES) + +include ../../lib.mk diff --git a/tools/testing/selftests/filesystems/mntns_cleanup/mntns_cleanup_test.c b/tools/testing/selftests/filesystems/mntns_cleanup/mntns_cleanup_test.c new file mode 100644 index 000000000000..5209712568b1 --- /dev/null +++ b/tools/testing/selftests/filesystems/mntns_cleanup/mntns_cleanup_test.c @@ -0,0 +1,58 @@ +// SPDX-License-Identifier: GPL-2.0 + +#define _GNU_SOURCE +#include +#include +#include +#include +#include +#include + +#include "../../kselftest_harness.h" + +FIXTURE(mntns_cleanup) { +}; + +FIXTURE_SETUP(mntns_cleanup) +{ + if (geteuid() != 0) + SKIP(return, "test requires CAP_SYS_ADMIN"); + + ASSERT_EQ(unshare(CLONE_NEWNS), 0); + ASSERT_EQ(mount("", "/", NULL, MS_REC | MS_PRIVATE, NULL), 0); + + rmdir("/mnt_dir"); + ASSERT_EQ(mkdir("/mnt_dir", 0755), 0); + ASSERT_EQ(mount("tmpfs", "/mnt_dir", "tmpfs", 0, NULL), 0); + ASSERT_EQ(mkdir("/mnt_dir/hidden", 0755), 0); + ASSERT_EQ(mkdir("/mnt_dir/hidden/secret", 0755), 0); + ASSERT_EQ(mount("tmpfs", "/mnt_dir/hidden", "tmpfs", 0, NULL), 0); +} + +FIXTURE_TEARDOWN(mntns_cleanup) +{ +} + +/* Mounts must stay connected when a mount namespace is cleaned up. */ +TEST_F(mntns_cleanup, keeps_mounts_connected) +{ + int fd, sfd, err; + + fd = open("/mnt_dir", O_PATH | O_DIRECTORY | O_CLOEXEC); + ASSERT_GE(fd, 0); + + /* Destroy the namespace; the fd keeps /mnt_dir alive. */ + ASSERT_EQ(unshare(CLONE_NEWNS), 0); + + sfd = openat(fd, "hidden/secret", O_RDONLY); + err = errno; + if (sfd >= 0) + close(sfd); + close(fd); + + ASSERT_LT(sfd, 0) + TH_LOG("mount namespace teardown revealed what the overmount covered"); + ASSERT_EQ(err, ENOENT); +} + +TEST_HARNESS_MAIN diff --git a/tools/testing/selftests/filesystems/overlayfs/.gitignore b/tools/testing/selftests/filesystems/overlayfs/.gitignore index e23a18c8b37f..077f7a128168 100644 --- a/tools/testing/selftests/filesystems/overlayfs/.gitignore +++ b/tools/testing/selftests/filesystems/overlayfs/.gitignore @@ -1,3 +1,4 @@ # SPDX-License-Identifier: GPL-2.0-only dev_in_maps set_layers_via_fds +idmapped_mounts diff --git a/tools/testing/selftests/filesystems/overlayfs/Makefile b/tools/testing/selftests/filesystems/overlayfs/Makefile index d3ad4a77db9b..b3185f684add 100644 --- a/tools/testing/selftests/filesystems/overlayfs/Makefile +++ b/tools/testing/selftests/filesystems/overlayfs/Makefile @@ -8,7 +8,9 @@ LOCAL_HDRS += ../wrappers.h log.h TEST_GEN_PROGS := dev_in_maps TEST_GEN_PROGS += set_layers_via_fds +TEST_GEN_PROGS += idmapped_mounts include ../../lib.mk $(OUTPUT)/set_layers_via_fds: ../utils.c +$(OUTPUT)/idmapped_mounts: ../utils.c diff --git a/tools/testing/selftests/filesystems/overlayfs/idmapped_mounts.c b/tools/testing/selftests/filesystems/overlayfs/idmapped_mounts.c new file mode 100644 index 000000000000..44a75839f4ed --- /dev/null +++ b/tools/testing/selftests/filesystems/overlayfs/idmapped_mounts.c @@ -0,0 +1,501 @@ +// SPDX-License-Identifier: GPL-2.0 +#define _GNU_SOURCE + +#include +#include +#include +#include +#include +#include +#include + +#include +#include + +#include "kselftest_harness.h" +#include "../wrappers.h" +#include "../utils.h" + +/* + * An idmapping that maps the mount-visible id range [0, ID_RANGE) onto the + * host/overlay-final id range [ID_HOST, ID_HOST + ID_RANGE). Through such an + * idmapped overlay mount, an overlay-final id of ID_HOST + n is reported as n, + * and an id of n requested through the mount is stored as ID_HOST + n. + */ +#define ID_NS 0 +#define ID_HOST 10000 +#define ID_RANGE 10000 + +/* + * For the composition test the lower layer's on-disk ids live in a + * separate range and are mapped by an idmapped lower layer onto the + * overlay-final range [ID_HOST, ID_HOST + ID_RANGE). + */ +#define LAYER_HOST 20000 + +#ifndef MOUNT_ATTR_IDMAP +#define MOUNT_ATTR_IDMAP 0x00100000 +#endif + +#ifndef __NR_mount_setattr +#define __NR_mount_setattr 442 +#endif + +static inline int sys_mount_setattr(int dfd, const char *path, + unsigned int flags, + struct mount_attr *attr, size_t size) +{ + return syscall(__NR_mount_setattr, dfd, path, flags, attr, size); +} + +static bool ovl_supported(void) +{ + int fd = sys_fsopen("overlay", 0); + + if (fd < 0) + return false; + close(fd); + return true; +} + +/* base/{l,u,w} owned by ID_HOST so they map to ID_NS through the idmap. */ +static int setup_layers(const char *base) +{ + static const char *sub[] = { "", "/l", "/u", "/w" }; + char path[PATH_MAX]; + + for (size_t i = 0; i < ARRAY_SIZE(sub); i++) { + snprintf(path, sizeof(path), "%s%s", base, sub[i]); + if (mkdir(path, 0755) && errno != EEXIST) + return -1; + if (i && chown(path, ID_HOST, ID_HOST)) + return -1; + } + return 0; +} + +static int ovl_mount(const char *base, bool nfs_export) +{ + char lower[PATH_MAX], upper[PATH_MAX], work[PATH_MAX]; + int fsfd, ovl; + + snprintf(lower, sizeof(lower), "%s/l", base); + snprintf(upper, sizeof(upper), "%s/u", base); + snprintf(work, sizeof(work), "%s/w", base); + + fsfd = sys_fsopen("overlay", 0); + if (fsfd < 0) + return -1; + + if (sys_fsconfig(fsfd, FSCONFIG_SET_STRING, "source", "test", 0) || + sys_fsconfig(fsfd, FSCONFIG_SET_STRING, "lowerdir", lower, 0) || + sys_fsconfig(fsfd, FSCONFIG_SET_STRING, "upperdir", upper, 0) || + sys_fsconfig(fsfd, FSCONFIG_SET_STRING, "workdir", work, 0)) + goto err; + if (nfs_export && + (sys_fsconfig(fsfd, FSCONFIG_SET_STRING, "index", "on", 0) || + sys_fsconfig(fsfd, FSCONFIG_SET_STRING, "nfs_export", "on", 0))) + goto err; + if (sys_fsconfig(fsfd, FSCONFIG_CMD_CREATE, NULL, NULL, 0)) + goto err; + + ovl = sys_fsmount(fsfd, 0, 0); + close(fsfd); + return ovl; +err: + close(fsfd); + return -1; +} + +/* Idmap the (still detached, not yet visible) overlay mount @mfd. */ +static int ovl_idmap(int mfd) +{ + struct mount_attr attr = { + .attr_set = MOUNT_ATTR_IDMAP, + }; + int ret, userns_fd; + + /* + * get_userns_fd(fs_id, mount_id, range): a file whose filesystem id + * is fs_id + n is shown through the idmapped mount as mount_id + n. + * Here the overlay-final (fs side) range is [ID_HOST, ..) and the + * caller-visible (mount side) range is [ID_NS, ..). + */ + userns_fd = get_userns_fd(ID_HOST, ID_NS, ID_RANGE); + if (userns_fd < 0) + return -1; + + attr.userns_fd = userns_fd; + ret = sys_mount_setattr(mfd, "", AT_EMPTY_PATH, &attr, sizeof(attr)); + close(userns_fd); + return ret; +} + +/* Clone @path into a detached, idmapped mount usable as an overlay layer. */ +static int idmapped_layer_fd(const char *path, int nsid, int hostid, int range) +{ + struct mount_attr attr = { + .attr_set = MOUNT_ATTR_IDMAP, + }; + int fd_tree, userns_fd; + + fd_tree = sys_open_tree(AT_FDCWD, path, + OPEN_TREE_CLONE | OPEN_TREE_CLOEXEC); + if (fd_tree < 0) + return -1; + userns_fd = get_userns_fd(nsid, hostid, range); + if (userns_fd < 0) { + close(fd_tree); + return -1; + } + attr.userns_fd = userns_fd; + if (sys_mount_setattr(fd_tree, "", AT_EMPTY_PATH, &attr, + sizeof(attr))) { + close(userns_fd); + close(fd_tree); + return -1; + } + close(userns_fd); + return fd_tree; +} + +/* Overlay with a layer passed by fd (idmapped) plus a plain upper/work. */ +static int ovl_mount_lower_fd(const char *upper, const char *work, int fd_lower) +{ + int fsfd, ovl; + + fsfd = sys_fsopen("overlay", 0); + if (fsfd < 0) + return -1; + + if (sys_fsconfig(fsfd, FSCONFIG_SET_STRING, "source", "test", 0) || + sys_fsconfig(fsfd, FSCONFIG_SET_STRING, "upperdir", upper, 0) || + sys_fsconfig(fsfd, FSCONFIG_SET_STRING, "workdir", work, 0) || + sys_fsconfig(fsfd, FSCONFIG_SET_FD, "lowerdir+", NULL, fd_lower) || + sys_fsconfig(fsfd, FSCONFIG_CMD_CREATE, NULL, NULL, 0)) + goto err; + + ovl = sys_fsmount(fsfd, 0, 0); + close(fsfd); + return ovl; +err: + close(fsfd); + return -1; +} + +/* + * Mount an overlay inside user namespace @u1 (so the overlay sb's s_user_ns is + * not the initial namespace) and idmap that overlay mount with @u2. Runs in a + * child that joins @u1; returns 0 on success. + */ +static int userns_overlay_child(int u1) +{ + struct mount_attr attr = { + .attr_set = MOUNT_ATTR_IDMAP, + }; + struct stat st; + int ovl, u2; + + /* Become root in the overlay sb's user namespace u1. */ + if (!switch_userns(u1, 0, 0, false)) + return fprintf(stderr, "userns: switch_userns: %m\n"), -1; + if (unshare(CLONE_NEWNS) || + sys_mount(NULL, "/", NULL, MS_SLAVE | MS_REC, NULL)) + return fprintf(stderr, "userns: unshare/slave: %m\n"), -1; + if (sys_mount("tmpfs", "/tmp", "tmpfs", 0, NULL)) + return fprintf(stderr, "userns: mount tmpfs: %m\n"), -1; + if (setup_layers("/tmp/ovl")) + return fprintf(stderr, "userns: setup_layers: %m\n"), -1; + if (mknod("/tmp/ovl/l/file", S_IFREG | 0644, 0) || + chown("/tmp/ovl/l/file", ID_HOST + 5, ID_HOST + 5)) + return fprintf(stderr, "userns: lower file: %m\n"), -1; + + ovl = ovl_mount("/tmp/ovl", false); + if (ovl < 0) + return fprintf(stderr, "userns: ovl_mount: %m\n"), -1; + + /* + * mount_setattr() requires CAP_SYS_ADMIN over the idmap user + * namespace, so it must be a child of u1. Create it now, from + * inside u1. + */ + u2 = get_userns_fd(ID_HOST, ID_NS, ID_RANGE); + if (u2 < 0) + return fprintf(stderr, "userns: get_userns_fd: %m\n"), -1; + attr.userns_fd = u2; + if (sys_mount_setattr(ovl, "", AT_EMPTY_PATH, &attr, sizeof(attr))) + return fprintf(stderr, "userns: mount_setattr: %m\n"), -1; + close(u2); + + if (fstatat(ovl, "file", &st, 0)) + return fprintf(stderr, "userns: fstatat: %m\n"), -1; + if (st.st_uid != ID_NS + 5 || st.st_gid != ID_NS + 5) { + fprintf(stderr, "userns: got %u:%u expected %u:%u\n", + st.st_uid, st.st_gid, ID_NS + 5, ID_NS + 5); + return -1; + } + return 0; +} + +FIXTURE(idmapped_overlay) { + char base[64]; +}; + +FIXTURE_SETUP(idmapped_overlay) +{ + /* Private mount namespace so test mounts need no cleanup. */ + ASSERT_EQ(unshare(CLONE_NEWNS), 0); + ASSERT_EQ(sys_mount(NULL, "/", NULL, MS_SLAVE | MS_REC, NULL), 0); + + /* tmpfs for the layers so we can chown them to arbitrary ids. */ + ASSERT_EQ(sys_mount("tmpfs", "/tmp", "tmpfs", 0, NULL), 0); + + snprintf(self->base, sizeof(self->base), "/tmp/ovl"); + ASSERT_EQ(setup_layers(self->base), 0); +} + +FIXTURE_TEARDOWN(idmapped_overlay) +{ +} + +/* A file owned by ID_HOST + 5 is reported as ID_NS + 5 through the idmap. */ +TEST_F(idmapped_overlay, getattr) +{ + char path[PATH_MAX]; + struct stat st; + int ovl; + + if (!ovl_supported()) + SKIP(return, "overlayfs not supported"); + + snprintf(path, sizeof(path), "%s/l/file", self->base); + ASSERT_EQ(mknod(path, S_IFREG | 0644, 0), 0); + ASSERT_EQ(chown(path, ID_HOST + 5, ID_HOST + 5), 0); + + ovl = ovl_mount(self->base, false); + ASSERT_GE(ovl, 0); + ASSERT_EQ(ovl_idmap(ovl), 0); + + ASSERT_EQ(fstatat(ovl, "file", &st, 0), 0); + EXPECT_EQ(st.st_uid, ID_NS + 5); + EXPECT_EQ(st.st_gid, ID_NS + 5); + + EXPECT_EQ(close(ovl), 0); +} + +/* + * Every creation path initializes the new owner through the mount idmap: + * created as caller id ID_NS, stored on the upper layer as overlay-final + * ID_HOST. Covers ovl_create() (regular file), ovl_mkdir(), ovl_mknod() + * and ovl_symlink() (which share ovl_create_object()), plus the separate + * ovl_tmpfile() path. + */ +TEST_F(idmapped_overlay, create) +{ + static const char *names[] = { "reg", "dir", "fifo", "lnk" }; + char path[PATH_MAX]; + struct stat st; + int ovl, fd; + + if (!ovl_supported()) + SKIP(return, "overlayfs not supported"); + + ovl = ovl_mount(self->base, false); + ASSERT_GE(ovl, 0); + ASSERT_EQ(ovl_idmap(ovl), 0); + + /* One object per creation operation, all as caller id ID_NS. */ + fd = openat(ovl, "reg", O_CREAT | O_WRONLY | O_EXCL, 0644); + ASSERT_GE(fd, 0); + EXPECT_EQ(close(fd), 0); + ASSERT_EQ(mkdirat(ovl, "dir", 0755), 0); + ASSERT_EQ(mknodat(ovl, "fifo", S_IFIFO | 0644, 0), 0); + ASSERT_EQ(symlinkat("target", ovl, "lnk"), 0); + + for (size_t i = 0; i < ARRAY_SIZE(names); i++) { + /* Reported as ID_NS through the idmapped mount ... */ + ASSERT_EQ(fstatat(ovl, names[i], &st, AT_SYMLINK_NOFOLLOW), 0); + EXPECT_EQ(st.st_uid, ID_NS); + EXPECT_EQ(st.st_gid, ID_NS); + /* ... and stored as ID_HOST on the upper layer. */ + snprintf(path, sizeof(path), "%s/u/%s", self->base, names[i]); + ASSERT_EQ(lstat(path, &st), 0); + EXPECT_EQ(st.st_uid, ID_HOST); + EXPECT_EQ(st.st_gid, ID_HOST); + } + + /* O_TMPFILE goes through the separate ovl_tmpfile() path. */ + fd = openat(ovl, ".", O_TMPFILE | O_WRONLY, 0644); + ASSERT_GE(fd, 0); + /* Inside the mount: caller id ID_NS. */ + ASSERT_EQ(fstat(fd, &st), 0); + EXPECT_EQ(st.st_uid, ID_NS); + EXPECT_EQ(st.st_gid, ID_NS); + /* Link it in so the upper backing file can be inspected too. */ + ASSERT_EQ(linkat(fd, "", ovl, "tmp", AT_EMPTY_PATH), 0); + EXPECT_EQ(close(fd), 0); + snprintf(path, sizeof(path), "%s/u/tmp", self->base); + ASSERT_EQ(lstat(path, &st), 0); + EXPECT_EQ(st.st_uid, ID_HOST); + EXPECT_EQ(st.st_gid, ID_HOST); + + EXPECT_EQ(close(ovl), 0); +} + +/* chown through the idmapped mount round-trips: ID_NS + 5 <-> ID_HOST + 5. */ +TEST_F(idmapped_overlay, chown) +{ + char path[PATH_MAX]; + struct stat st; + int ovl, fd; + + if (!ovl_supported()) + SKIP(return, "overlayfs not supported"); + + ovl = ovl_mount(self->base, false); + ASSERT_GE(ovl, 0); + ASSERT_EQ(ovl_idmap(ovl), 0); + + fd = openat(ovl, "f", O_CREAT | O_WRONLY | O_EXCL, 0644); + ASSERT_GE(fd, 0); + EXPECT_EQ(close(fd), 0); + + ASSERT_EQ(fchownat(ovl, "f", ID_NS + 5, ID_NS + 5, 0), 0); + + ASSERT_EQ(fstatat(ovl, "f", &st, 0), 0); + EXPECT_EQ(st.st_uid, ID_NS + 5); + EXPECT_EQ(st.st_gid, ID_NS + 5); + + snprintf(path, sizeof(path), "%s/u/f", self->base); + ASSERT_EQ(stat(path, &st), 0); + EXPECT_EQ(st.st_uid, ID_HOST + 5); + EXPECT_EQ(st.st_gid, ID_HOST + 5); + + EXPECT_EQ(close(ovl), 0); +} + +/* + * Composition: an idmapped lower layer underneath an idmapped overlay mount. + * An on-disk id is mapped by the layer idmap into the overlay-final range and + * then by the mount idmap into the caller's range: + * + * on-disk LAYER_HOST+7 --layer--> ID_HOST+7 --mount--> ID_NS+7 + */ +TEST_F(idmapped_overlay, composition) +{ + char lower[PATH_MAX], upper[PATH_MAX], work[PATH_MAX], path[PATH_MAX]; + struct stat st; + int ovl, fd_lower; + + if (!ovl_supported()) + SKIP(return, "overlayfs not supported"); + + snprintf(lower, sizeof(lower), "%s/l", self->base); + snprintf(upper, sizeof(upper), "%s/u", self->base); + snprintf(work, sizeof(work), "%s/w", self->base); + + /* Put the lower layer's ids in the on-disk [LAYER_HOST, ..) range. */ + ASSERT_EQ(chown(lower, LAYER_HOST, LAYER_HOST), 0); + snprintf(path, sizeof(path), "%s/l/file", self->base); + ASSERT_EQ(mknod(path, S_IFREG | 0644, 0), 0); + ASSERT_EQ(chown(path, LAYER_HOST + 7, LAYER_HOST + 7), 0); + + /* Idmapped lower: on-disk LAYER_HOST <-> overlay-final ID_HOST. */ + fd_lower = idmapped_layer_fd(lower, LAYER_HOST, ID_HOST, ID_RANGE); + ASSERT_GE(fd_lower, 0); + + ovl = ovl_mount_lower_fd(upper, work, fd_lower); + ASSERT_GE(ovl, 0); + EXPECT_EQ(close(fd_lower), 0); + + /* Idmap the overlay mount: overlay-final ID_HOST <-> caller ID_NS. */ + ASSERT_EQ(ovl_idmap(ovl), 0); + + ASSERT_EQ(fstatat(ovl, "file", &st, 0), 0); + EXPECT_EQ(st.st_uid, ID_NS + 7); + EXPECT_EQ(st.st_gid, ID_NS + 7); + + EXPECT_EQ(close(ovl), 0); +} + +/* An idmapped overlay mount whose sb lives inside a user namespace. */ +TEST_F(idmapped_overlay, userns) +{ + int u1; + pid_t pid; + + if (!ovl_supported()) + SKIP(return, "overlayfs not supported"); + + /* u1 backs the overlay sb: identity-mapped, but not the init ns. */ + u1 = get_userns_fd(0, 0, 65536); + if (u1 < 0) + SKIP(return, "user namespaces not available"); + + pid = fork(); + ASSERT_GE(pid, 0); + if (pid == 0) { + int ret = userns_overlay_child(u1); + + _exit(ret ? EXIT_FAILURE : EXIT_SUCCESS); + } + EXPECT_EQ(wait_for_pid(pid), 0); + + EXPECT_EQ(close(u1), 0); +} + +/* + * An nfs_export overlay can be idmapped, and decodable file handles round-trip + * through the idmapped mount with correctly mapped ownership. Overlay file + * handles encode object identity, not ownership, so the mount idmap does not + * affect them; it only maps the owner reported once a handle is reopened. + */ +TEST_F(idmapped_overlay, nfs_export_handles) +{ + char path[PATH_MAX], mnt[128]; + union { + struct file_handle fh; + char buf[sizeof(struct file_handle) + MAX_HANDLE_SZ]; + } fhu; + struct file_handle *fh = &fhu.fh; + struct stat st; + int ovl, mfd, fd, mount_id; + + if (!ovl_supported()) + SKIP(return, "overlayfs not supported"); + + snprintf(path, sizeof(path), "%s/l/file", self->base); + ASSERT_EQ(mknod(path, S_IFREG | 0644, 0), 0); + ASSERT_EQ(chown(path, ID_HOST + 7, ID_HOST + 7), 0); + + /* nfs_export=on gives decodable overlay file handles. */ + ovl = ovl_mount(self->base, true); + if (ovl < 0) + SKIP(return, "overlayfs nfs_export not supported"); + ASSERT_EQ(ovl_idmap(ovl), 0); + + /* Attach the idmapped mount so handles can be resolved against it. */ + snprintf(mnt, sizeof(mnt), "%s/mnt", self->base); + ASSERT_EQ(mkdir(mnt, 0755), 0); + ASSERT_EQ(sys_move_mount(ovl, "", AT_FDCWD, mnt, + MOVE_MOUNT_F_EMPTY_PATH), 0); + + snprintf(path, sizeof(path), "%s/file", mnt); + fh->handle_bytes = MAX_HANDLE_SZ; + ASSERT_EQ(name_to_handle_at(AT_FDCWD, path, fh, &mount_id, 0), 0); + + mfd = open(mnt, O_RDONLY | O_DIRECTORY); + ASSERT_GE(mfd, 0); + fd = open_by_handle_at(mfd, fh, O_RDONLY); + EXPECT_EQ(close(mfd), 0); + ASSERT_GE(fd, 0); + + ASSERT_EQ(fstat(fd, &st), 0); + EXPECT_EQ(st.st_uid, ID_NS + 7); + EXPECT_EQ(st.st_gid, ID_NS + 7); + + EXPECT_EQ(close(fd), 0); + EXPECT_EQ(close(ovl), 0); +} + +TEST_HARNESS_MAIN diff --git a/tools/testing/selftests/filesystems/overlayfs/set_layers_via_fds.c b/tools/testing/selftests/filesystems/overlayfs/set_layers_via_fds.c index 3c0b93183348..7a293544233d 100644 --- a/tools/testing/selftests/filesystems/overlayfs/set_layers_via_fds.c +++ b/tools/testing/selftests/filesystems/overlayfs/set_layers_via_fds.c @@ -624,7 +624,7 @@ TEST_F(set_layers_via_fds, set_layers_via_detached_mount_fds) ASSERT_EQ(sys_move_mount(fd_tmpfs, "", -EBADF, "/set_layers_via_fds_tmpfs", MOVE_MOUNT_F_EMPTY_PATH), 0); - fd_tmp = open_tree(fd_tmpfs, "u", OPEN_TREE_CLONE | OPEN_TREE_CLOEXEC); + fd_tmp = sys_open_tree(fd_tmpfs, "u", OPEN_TREE_CLONE | OPEN_TREE_CLOEXEC); ASSERT_GE(fd_tmp, 0); layer_fds[0] = openat(fd_tmp, "upper", O_CLOEXEC | O_DIRECTORY | O_PATH); @@ -633,25 +633,25 @@ TEST_F(set_layers_via_fds, set_layers_via_detached_mount_fds) layer_fds[1] = openat(fd_tmp, "work", O_CLOEXEC | O_DIRECTORY | O_PATH); ASSERT_GE(layer_fds[1], 0); - layer_fds[2] = open_tree(fd_tmpfs, "l1", OPEN_TREE_CLONE | OPEN_TREE_CLOEXEC); + layer_fds[2] = sys_open_tree(fd_tmpfs, "l1", OPEN_TREE_CLONE | OPEN_TREE_CLOEXEC); ASSERT_GE(layer_fds[2], 0); - layer_fds[3] = open_tree(fd_tmpfs, "l2", OPEN_TREE_CLONE | OPEN_TREE_CLOEXEC); + layer_fds[3] = sys_open_tree(fd_tmpfs, "l2", OPEN_TREE_CLONE | OPEN_TREE_CLOEXEC); ASSERT_GE(layer_fds[3], 0); - layer_fds[4] = open_tree(fd_tmpfs, "l3", OPEN_TREE_CLONE | OPEN_TREE_CLOEXEC); + layer_fds[4] = sys_open_tree(fd_tmpfs, "l3", OPEN_TREE_CLONE | OPEN_TREE_CLOEXEC); ASSERT_GE(layer_fds[4], 0); - layer_fds[5] = open_tree(fd_tmpfs, "l4", OPEN_TREE_CLONE | OPEN_TREE_CLOEXEC); + layer_fds[5] = sys_open_tree(fd_tmpfs, "l4", OPEN_TREE_CLONE | OPEN_TREE_CLOEXEC); ASSERT_GE(layer_fds[5], 0); - layer_fds[6] = open_tree(fd_tmpfs, "d1", OPEN_TREE_CLONE | OPEN_TREE_CLOEXEC); + layer_fds[6] = sys_open_tree(fd_tmpfs, "d1", OPEN_TREE_CLONE | OPEN_TREE_CLOEXEC); ASSERT_GE(layer_fds[6], 0); - layer_fds[7] = open_tree(fd_tmpfs, "d2", OPEN_TREE_CLONE | OPEN_TREE_CLOEXEC); + layer_fds[7] = sys_open_tree(fd_tmpfs, "d2", OPEN_TREE_CLONE | OPEN_TREE_CLOEXEC); ASSERT_GE(layer_fds[7], 0); - layer_fds[8] = open_tree(fd_tmpfs, "d3", OPEN_TREE_CLONE | OPEN_TREE_CLOEXEC); + layer_fds[8] = sys_open_tree(fd_tmpfs, "d3", OPEN_TREE_CLONE | OPEN_TREE_CLOEXEC); ASSERT_GE(layer_fds[8], 0); ASSERT_EQ(close(fd_tmpfs), 0); diff --git a/tools/testing/selftests/filesystems/statmount/statmount_test.c b/tools/testing/selftests/filesystems/statmount/statmount_test.c index 8dc018d47a93..60c2c544db6a 100644 --- a/tools/testing/selftests/filesystems/statmount/statmount_test.c +++ b/tools/testing/selftests/filesystems/statmount/statmount_test.c @@ -82,6 +82,9 @@ static void cleanup_namespace(void) { int ret; + if (f_mountinfo) + fclose(f_mountinfo); + ret = fchdir(orig_root); if (ret == -1) ksft_perror("fchdir to original root"); @@ -515,7 +518,7 @@ static void test_statmount_mnt_opts(void) return; } - ksft_test_result_fail("didnt't find mount entry\n"); + ksft_test_result_fail("didn't find mount entry\n"); free(sm); free(line); } diff --git a/tools/testing/selftests/filesystems/ustat_test.c b/tools/testing/selftests/filesystems/ustat_test.c new file mode 100644 index 000000000000..d429fd18d779 --- /dev/null +++ b/tools/testing/selftests/filesystems/ustat_test.c @@ -0,0 +1,135 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Test ustat(2): looking up superblocks by device number. + * + * ustat() resolves a device number to a mounted superblock via + * user_get_super(). Check that the device number of a mounted tmpfs (an + * anonymous device) resolves, that it stops resolving once the filesystem + * is unmounted and that bogus device numbers report EINVAL. + */ +#define _GNU_SOURCE +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "../kselftest_harness.h" + +/* struct ustat is not exported through UAPI, mirror include/linux/types.h. */ +struct ustat_buf { + int f_tfree; + unsigned long f_tinode; + char f_fname[6]; + char f_fpack[6]; + /* slack in case an architecture lays the struct out differently */ + char pad[64]; +}; + +#ifdef __NR_ustat + +/* + * The kernel decodes @dev with new_decode_dev(), which matches the low 32 + * bits of the st_dev encoding stat(2) returns for any major below 4096. + */ +static int sys_ustat(unsigned int dev, struct ustat_buf *buf) +{ + return syscall(__NR_ustat, dev, buf); +} + +static int write_string(const char *path, const char *string) +{ + ssize_t len = strlen(string); + int fd; + + fd = open(path, O_WRONLY); + if (fd < 0) + return -1; + if (write(fd, string, len) != len) { + close(fd); + return -1; + } + return close(fd); +} + +/* Enter namespaces in which mounting a tmpfs instance is allowed. */ +static int setup_namespaces(void) +{ + uid_t uid = getuid(); + gid_t gid = getgid(); + char map[64]; + + if (unshare(CLONE_NEWNS | (uid ? CLONE_NEWUSER : 0))) + return -1; + + if (uid) { + if (write_string("/proc/self/setgroups", "deny")) + return -1; + snprintf(map, sizeof(map), "0 %d 1", uid); + if (write_string("/proc/self/uid_map", map)) + return -1; + snprintf(map, sizeof(map), "0 %d 1", gid); + if (write_string("/proc/self/gid_map", map)) + return -1; + } + + return mount(NULL, "/", NULL, MS_REC | MS_PRIVATE, NULL); +} + +TEST(resolves_mounted_superblock) +{ + char dir[] = "/tmp/ustat_test.XXXXXX"; + struct ustat_buf ub; + struct stat st; + + ASSERT_NE(NULL, mkdtemp(dir)); + + if (setup_namespaces()) { + rmdir(dir); + SKIP(return, "cannot set up namespaces: %s", strerror(errno)); + } + + ASSERT_EQ(0, mount("ustat_test", dir, "tmpfs", 0, NULL)); + ASSERT_EQ(0, stat(dir, &st)); + + memset(&ub, 0xff, sizeof(ub)); + ASSERT_EQ(0, sys_ustat(st.st_dev, &ub)) + TH_LOG("ustat(%u): %s", (unsigned int)st.st_dev, + strerror(errno)); + + ASSERT_EQ(0, umount(dir)); + + /* The unmount removed the superblock, the device is gone. */ + ASSERT_EQ(-1, sys_ustat(st.st_dev, &ub)); + ASSERT_EQ(EINVAL, errno); + + rmdir(dir); +} + +TEST(bogus_device_numbers) +{ + struct ustat_buf ub; + + ASSERT_EQ(-1, sys_ustat(0, &ub)); + ASSERT_EQ(EINVAL, errno); + + /* major 4095, minor 1048575: nothing plausible lives there */ + ASSERT_EQ(-1, sys_ustat((0xfffu << 8) | 0xffu | (0xfff00u << 12), &ub)); + ASSERT_EQ(EINVAL, errno); +} + +#else /* !__NR_ustat */ + +TEST(unsupported) +{ + SKIP(return, "ustat(2) is not available on this architecture"); +} + +#endif /* __NR_ustat */ + +TEST_HARNESS_MAIN diff --git a/tools/testing/selftests/proc/proc-pidns.c b/tools/testing/selftests/proc/proc-pidns.c index 25b9a2933c45..6f7c10fe97b3 100644 --- a/tools/testing/selftests/proc/proc-pidns.c +++ b/tools/testing/selftests/proc/proc-pidns.c @@ -6,6 +6,7 @@ #include #include +#include #include #include #include