100 Commits
Author SHA1 Message Date
Al Viro 40a1926022 fix the breakage in close_fd_get_file() calling conventions change
It used to grab an extra reference to struct file rather than
just transferring to caller the one it had removed from descriptor
table.  New variant doesn't, and callers need to be adjusted.

Reported-and-tested-by: syzbot+47dd250f527cb7bebf24@syzkaller.appspotmail.com
Fixes: 6319194ec5 ("Unify the primitives for file descriptor closing")
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2022-06-05 15:03:03 -04:00
Al Viro 52dba645ca get rid of dead code in legitimize_root()
Combination of LOOKUP_IS_SCOPED and NULL nd->root.mnt is impossible
after successful path_init().  All places where ->root.mnt might
become NULL do that only if LOOKUP_IS_SCOPED is not there and
path_init() itself can return success without setting nd->root
only if ND_ROOT_PRESET had been set (in which case nd->root
had been set by caller and never changed) or if the name had
been a relative one *and* none of the bits in LOOKUP_IS_SCOPED
had been present.

Since all calls of legitimize_root() must be downstream of successful
path_init(), the check for !nd->root.mnt && (nd->flags & LOOKUP_IS_SCOPED)
is pure paranoia.

FWIW, it had been discussed (and agreed upon) with Aleksa back when
scoped lookups had been merged; looks like that had fallen through the
cracks back then.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2022-05-19 23:26:29 -04:00
Al Viro e5ca024e16 fs/namei.c:reserve_stack(): tidy up the call of try_to_unlazy()
!foo() != 0 is a strange way to spell !foo(); fallout from
"fs: make unlazy_walk() error handling consistent"...

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2022-05-19 23:26:29 -04:00
Al Viro 70f8d9c575 move mount-related externs from fs.h to mount.h
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2022-05-19 23:25:48 -04:00
Al Viro 279b192c23 blob_to_mnt(): kern_unmount() is needed to undo kern_mount()
plain mntput() won't do.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2022-05-19 23:25:47 -04:00
Al Viro f6957b7191 m->mnt_root->d_inode->i_sb is a weird way to spell m->mnt_sb...
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2022-05-19 23:25:45 -04:00
Al Viro 59df85d5fb linux/mount.h: trim includes
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2022-05-19 23:25:18 -04:00
Al Viro a5f85d7834 uninline may_mount() and don't opencode it in fspick(2)/fsopen(2)
It's done once per (mount-related) syscall and there's no point
whatsoever making it inline.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2022-05-19 23:25:10 -04:00
Al Viro fb4554c223 Fix double fget() in vhost_net_set_backend()
Descriptor table is a shared resource; two fget() on the same descriptor
may return different struct file references.  get_tap_ptr_ring() is
called after we'd found (and pinned) the socket we'll be using and it
tries to find the private tun/tap data structures associated with it.
Redoing the lookup by the same file descriptor we'd used to get the
socket is racy - we need to same struct file.

Thanks to Jason for spotting a braino in the original variant of patch -
I'd missed the use of fd == -1 for disabling backend, and in that case
we can end up with sock == NULL and sock != oldsock.

Cc: stable@kernel.org
Acked-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2022-05-18 12:33:51 -04:00
Al Viro a91714312e percpu_ref_init(): clean ->percpu_count_ref on failure
That way percpu_ref_exit() is safe after failing percpu_ref_init().
At least one user (cgroup_create()) had a double-free that way;
there might be other similar bugs.  Easier to fix in percpu_ref_init(),
rather than playing whack-a-mole in sloppy users...

Usual symptoms look like a messed refcounting in one of subsystems
that use percpu allocations (might be percpu-refcount, might be
something else).  Having refcounts for two different objects share
memory is Not Nice(tm)...

Reported-by: syzbot+5b1e53987f858500ec00@syzkaller.appspotmail.com
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2022-05-18 02:20:17 -04:00
Al Viro 6319194ec5 Unify the primitives for file descriptor closing
Currently we have 3 primitives for removing an opened file from descriptor
table - pick_file(), __close_fd_get_file() and close_fd_get_file().  Their
calling conventions are rather odd and there's a code duplication for no
good reason.  They can be unified -

1) have __range_close() cap max_fd in the very beginning; that way
we don't need separate way for pick_file() to report being past the end
of descriptor table.

2) make {__,}close_fd_get_file() return file (or NULL) directly, rather
than returning it via struct file ** argument.  Don't bother with
(bogus) return value - nobody wants that -ENOENT.

3) make pick_file() return NULL on unopened descriptor - the only caller
that used to care about the distinction between descriptor past the end
of descriptor table and finding NULL in descriptor table doesn't give
a damn after (1).

4) lift ->files_lock out of pick_file()

That actually simplifies the callers, as well as the primitives themselves.
Code duplication is also gone...

Reviewed-by: Christian Brauner (Microsoft) <brauner@kernel.org>
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2022-05-14 18:49:01 -04:00
Al Viro 4329490a78 io_uring_enter(): don't leave f.flags uninitialized
simplifies logics on cleanup, as well...

Reviewed-by: Jens Axboe <axboe@kernel.dk>
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2022-05-12 17:07:05 -04:00
Al Viro e257039f0f mount_setattr(): clean the control flow and calling conventions
separate the "cleanup" and "apply" codepaths (they have almost no overlap),
fold the "cleanup" into "prepare" (which eliminates the need of ->revert)
and make loops more idiomatic.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2022-03-15 19:17:13 -04:00
Al Viro 124f75f864 clean overflow checks in count_mounts() a bit
Wraparound checks in there are redundant (x + y < x and
x + y < y are equivalent when x and y are both unsigned int).

IMO more straightforward code would be better here...

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2022-02-13 22:42:30 -05:00
Al Viro 6692531df6 uml/x86: use x86 load_unaligned_zeropad()
allows, among other things, to drop !DCACHE_WORD_ACCESS mess in
x86 csum-partial_64.c

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2022-01-30 21:26:39 -05:00
Al Viro 0c9dceb9bb asm/user.h: killed unused macros
Some of them used to be used by libbfd for a.out coredump handling.
Seeing that
	* libbfd has their copies anyway
	* we don't export them into userland headers
	* we don't support a.out coredumps anymore
let's bury the definitions.  They never had in-kernel
users anyway...

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2022-01-30 21:17:00 -05:00
Al Viro 1e2d84644d constify struct path argument of finish_automount()/do_add_mount()
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2022-01-30 21:16:48 -05:00
Al Viro 4c1f795773 um: header debriding - sigio.h
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Richard Weinberger <richard@nod.at>
2021-12-21 21:31:36 +01:00
Al Viro 021fdaef80 um: header debriding - os.h
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Richard Weinberger <richard@nod.at>
2021-12-21 21:31:36 +01:00
Al Viro b31ef6d89d um: header debriding - net_*.h
externs dead since before the initial merge

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Richard Weinberger <richard@nod.at>
2021-12-21 21:31:35 +01:00
Al Viro bb1a2c4e2d um: header debriding - mem_user.h
get_vm(), add_iomem(), phys_offset() dead since 2004;
init_mem_user() and setup_memory() - since before the initial merge.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Richard Weinberger <richard@nod.at>
2021-12-21 21:31:35 +01:00
Al Viro ed4b1cc590 um: header debriding - activate_ipi()
... had been dead for 15 years.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Richard Weinberger <richard@nod.at>
2021-12-21 21:31:35 +01:00
Al Viro 8e5d7cf347 um: common-offsets.h debriding...
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Richard Weinberger <richard@nod.at>
2021-12-21 21:31:35 +01:00
Al Viro 2610ed63ea um, x86: bury crypto_tfm_ctx_offset
unused since 2011

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Richard Weinberger <richard@nod.at>
2021-12-21 21:31:35 +01:00
Al Viro 21cba62bea um: unexport handle_page_fault()
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Richard Weinberger <richard@nod.at>
2021-12-21 21:31:35 +01:00
Al Viro 7f5f156dae um: remove a dangling extern of syscall_trace()
the function had been gone since 2012...

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Richard Weinberger <richard@nod.at>
2021-12-21 21:31:35 +01:00
Al Viro 6605a44866 um: kill unused cpu()
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Richard Weinberger <richard@nod.at>
2021-12-21 21:31:35 +01:00
Al Viro 2098e213dd uml/i386: missing include in barrier.h
we need cpufeatures.h there

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Richard Weinberger <richard@nod.at>
2021-12-21 21:31:35 +01:00
Al Viro dbba7f704a um: stop polluting the namespace with registers.h contents
Only one extern in there is needed in processor-generic.h, and it's
not needed anywhere else.  So move it over there and get rid of
the include in processor-generic.h, adding includes of registers.h
to the few files that need the declarations in it.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Richard Weinberger <richard@nod.at>
2021-12-21 21:31:35 +01:00
Al Viro 5f174ec3c1 logic_io instance of iounmap() needs volatile on argument
... same as the rest of implementations

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Richard Weinberger <richard@nod.at>
2021-12-21 21:31:08 +01:00
Al Viro 577ade59b9 um: move amd64 variant of mmap(2) to arch/x86/um/syscalls_64.c
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Richard Weinberger <richard@nod.at>
2021-12-21 21:30:44 +01:00
Al Viro 8f5c84f367 uml: trim unused junk from arch/x86/um/sys_call_table_*.c
a bunch of detritus there - definitions that are never expanded or
checked.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Richard Weinberger <richard@nod.at>
2021-12-21 21:30:44 +01:00
Al Viro fbd63c08cd csky: don't let sigreturn play with priveleged bits of status register
csky restore_sigcontext() blindly overwrites regs->sr with the value
it finds in sigcontext.  Attacker can store whatever they want in there,
which includes things like S-bit.  Userland shouldn't be able to set
that, or anything other than C flag (bit 0).

Do the same thing other architectures with protected bits in flags
register do - preserve everything that shouldn't be settable in
user mode, picking the rest from the value saved is sigcontext.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Guo Ren <guoren@kernel.org>
Cc: stable@vger.kernel.org
2021-10-16 07:20:12 +08:00
Al Viro 0d20abde98 m68k: Leave stack mangling to asm wrapper of sigreturn()
sigreturn has to deal with an unpleasant problem - exception stack frames
have different sizes, depending upon the exception (and processor model, as
well) and variable-sized part of exception frame may contain information
needed for instruction restart.  So when signal handler terminates and calls
sigreturn to resume the execution at the place where we'd been when we caught
the signal, it has to rearrange the frame at the bottom of kernel stack.
Worse, it might need to open a gap in the kernel stack, shifting pt_regs
towards lower addresses.

Doing that from C is insane - we'd need to shift stack frames (return addresses,
local variables, etc.) of C call chain, right under the nose of compiler and
hope it won't fall apart horribly.  What had been actually done is only slightly
less insane - an inline asm in mangle_kernel_stack() moved the stuff around,
then reset stack pointer and jumped to label in asm glue.

However, we can avoid all that mess if the asm wrapper we have to use anyway
would reserve some space on the stack between switch_stack and the C stack
frame of do_{rt_,}sigreturn().   Then C part can simply memmove() pt_regs +
switch_stack, memcpy() the variable part of exception frame into the opened
gap - all of that without inline asm, buggering C call chain, magical jumps
to asm labels, etc.

Asm wrapper would need to know where the moved switch_stack has ended up -
it might have been shifted into the gap we'd reserved before do_rt_sigreturn()
call.  That's where it needs to set the stack pointer to.  So let the C part
return just that and be done with that.

While we are at it, the call of berr_040cleanup() we need to do when
returning via 68040 bus error exception frame can be moved into C part
as well.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Tested-by: Michael Schmitz <schmitzmic@gmail.com>
Reviewed-by: Michael Schmitz <schmitzmic@gmail.com>
Tested-by: Finn Thain <fthain@linux-m68k.org>
Link: https://lore.kernel.org/r/YP2dTQPm1wGPWFgD@zeniv-ca.linux.org.uk
Signed-off-by: Geert Uytterhoeven <geert@linux-m68k.org>
2021-09-24 13:35:03 +02:00
Al Viro 50e43a5733 m68k: Update ->thread.esp0 before calling syscall_trace() in ret_from_signal
We get there when sigreturn has performed obscene acts on kernel stack;
in particular, the location of pt_regs has shifted.  We are about to call
syscall_trace(), which might stop for tracer.  If that happens, we'd better
have task_pt_regs() returning correct result...

Fucked-up-by: Al Viro <viro@zeniv.linux.org.uk>
Fixes: bd6f56a75b ("m68k: Missing syscall_trace() on sigreturn")
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Tested-by: Michael Schmitz <schmitzmic@gmail.com>
Reviewed-by: Michael Schmitz <schmitzmic@gmail.com>
Tested-by: Finn Thain <fthain@linux-m68k.org>
Link: https://lore.kernel.org/r/YP2dMWeV1LkHiOpr@zeniv-ca.linux.org.uk
Signed-off-by: Geert Uytterhoeven <geert@linux-m68k.org>
2021-09-24 13:35:01 +02:00
Al Viro 4bb0bd81ce m68k: Handle arrivals of multiple signals correctly
When we have several pending signals, have entered with the kernel
with large exception frame *and* have already built at least one
sigframe, regs->stkadj is going to be non-zero and regs->format/sr/pc
are going to be junk - the real values are in shifted exception stack
frame we'd built when putting together the first sigframe.

If that happens, subsequent sigframes are going to be garbage.
Not hard to fix - just need to find the "adjusted" frame first
and look for format/vector/sr/pc in it.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Tested-by: Michael Schmitz <schmitzmic@gmail.com>
Reviewed-by: Michael Schmitz <schmitzmic@gmail.com>
Tested-by: Finn Thain <fthain@linux-m68k.org>
Link: https://lore.kernel.org/r/YP2dBIAPTaVvHiZ6@zeniv-ca.linux.org.uk
Signed-off-by: Geert Uytterhoeven <geert@linux-m68k.org>
2021-09-24 13:34:46 +02:00
Al Viro ca42bc4b7b sh: fix trivial misannotations
Trivial misannotations in
* get_user() (__gu_addr is a userland pointer there)
* ip_fast_csum() (sum is __wsum, not unsigned int)
* csum_and_copy_to_user() (destination is void *, not const void * -
mea culpa)
* __clear_user() (to is a userland pointer)
* several places in kernel/traps_32.c (regs->pc is a userland pointer
when regs is a userland pt_regs)
* math-emu/math.c: READ() and WRITE() casts of address should be to
userland pointer.

No changes in code generation and those take care of the majority of
noise from sparse on sh builds.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Tested-by: John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de>
Signed-off-by: Rich Felker <dalias@libc.org>
2021-09-17 13:59:56 -04:00
Al Viro ea47ab1116 putname(): IS_ERR_OR_NULL() is wrong here
Mixing NULL and ERR_PTR() just in case is a Bad Idea(tm).  For
struct filename the former is wrong - failures are reported
as ERR_PTR(...), not as NULL.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-09-07 16:14:05 -04:00
Al Viro c5f563f9e9 rename __filename_parentat() to filename_parentat()
... in separate commit, to avoid noise in previous one

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-09-07 15:57:42 -04:00
Al Viro 97c885d585 x86: signal: don't do sas_ss_reset() until we are certain that sigframe won't be abandoned
Currently we handle SS_AUTODISARM as soon as we have stored the altstack
settings into sigframe - that's the point when we have set the things up
for eventual sigreturn to restore the old settings.  And if we manage to
set the sigframe up (we are not done with that yet), everything's fine.
However, in case of failure we end up with sigframe-to-be abandoned and
SIGSEGV force-delivered.  And in that case we end up with inconsistent
rules - late failures have altstack reset, early ones do not.

It's trivial to get consistent behaviour - just handle SS_AUTODISARM once
we have set the sigframe up and are committed to entering the handler,
i.e.  in signal_delivered().

Link: https://lore.kernel.org/lkml/20200404170604.GN23230@ZenIV.linux.org.uk/
Link: https://github.com/ClangBuiltLinux/linux/issues/876
Link: https://lkml.kernel.org/r/20210422230846.1756380-1-ndesaulniers@google.com
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
Acked-by: Oleg Nesterov <oleg@redhat.com>
Tested-by: Nathan Chancellor <natechancellor@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2021-07-01 11:06:06 -07:00
Al Viro be752283a2 __unix_find_socket_byname(): don't pass hash and type separately
We only care about exclusive or of those, so pass that directly.
Makes life simpler for callers as well...

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: David S. Miller <davem@davemloft.net>
2021-06-21 12:28:49 -07:00
Al Viro c0c3b8d380 unix_bind_bsd(): unlink if we fail after successful mknod
We can do that more or less safely, since the parent is
held locked all along.  Yes, somebody might observe the
object via dcache, only to have it disappear afterwards,
but there's really no good way to prevent that.  It won't
race with other bind(2) or attempts to move the sucker
elsewhere, or put something else in its place - locked
parent prevents that.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: David S. Miller <davem@davemloft.net>
2021-06-21 12:28:49 -07:00
Al Viro 56c1731b28 unix_bind_bsd(): move done_path_create() call after dealing with ->bindlock
Final preparations for doing unlink on failure past the successful
mknod.  We can't hold ->bindlock over ->mknod() or ->unlink(), since
either might do sb_start_write() (e.g. on overlayfs).  However, we
can do it while holding filesystem and VFS locks - doing
	kern_path_create()
	vfs_mknod()
	grab ->bindlock
	if u->addr had been set
		drop ->bindlock
		done_path_create
		return -EINVAL
	else
		assign the address to socket
		drop ->bindlock
		done_path_create
		return 0
would be deadlock-free.  Here we massage unix_bind_bsd() to that
form.  We are still doing equivalent transformations.

Next commit will *not* be an equivalent transformation - it will
add a call of vfs_unlink() before done_path_create() in "alread bound"
case.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: David S. Miller <davem@davemloft.net>
2021-06-21 12:28:49 -07:00
Al Viro 71e6be6f7d fold unix_mknod() into unix_bind_bsd()
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: David S. Miller <davem@davemloft.net>
2021-06-21 12:28:49 -07:00
Al Viro fa42d910a3 unix_bind(): take BSD and abstract address cases into new helpers
unix_bind_bsd() and unix_bind_abstract() respectively.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: David S. Miller <davem@davemloft.net>
2021-06-21 12:28:49 -07:00
Al Viro aee5151705 unix_bind(): separate BSD and abstract cases
We do get some duplication that way, but it's minor compared to
parts that are different.  What we get is an ability to change
locking in BSD case without making failure exits very hard to
follow.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: David S. Miller <davem@davemloft.net>
2021-06-21 12:28:49 -07:00
Al Viro c34d458251 unix_bind(): allocate addr earlier
makes it easier to massage; we do pay for that by extra work
(kmalloc+memcpy+kfree) in some error cases, but those are not
on the hot paths anyway.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: David S. Miller <davem@davemloft.net>
2021-06-21 12:28:49 -07:00
Al Viro 185ab886d3 af_unix: take address assignment/hash insertion into a new helper
Duplicated logics in all bind variants (autobind, bind-to-path,
bind-to-abstract) gets taken into a common helper.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: David S. Miller <davem@davemloft.net>
2021-06-21 12:28:49 -07:00
Al Viro d99cf13f14 selinux: kill 'flags' argument in avc_has_perm_flags() and avc_audit()
... along with avc_has_perm_flags() itself, since now it's identical
to avc_has_perm() (as pointed out by Paul Moore)

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
[PM: add "selinux:" prefix to subj and tweak for length]
Signed-off-by: Paul Moore <paul@paul-moore.com>
2021-06-11 13:11:45 -04:00
Al Viro b17ec22fb3 selinux: slow_avc_audit has become non-blocking
dump_common_audit_data() is safe to use under rcu_read_lock() now;
no need for AVC_NONBLOCKING and games around it

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Paul Moore <paul@paul-moore.com>
2021-06-11 13:05:18 -04:00
Al Viro 6852df1266 csum_and_copy_to_pipe_iter(): leave handling of csum_state to caller
... since all the logics is already there for use by iovec/kvec/etc.
cases.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-06-10 11:45:25 -04:00
Al Viro 2a510a744b clean up copy_mc_pipe_to_iter()
... and we don't need kmap_atomic() there - kmap_local_page() is fine.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-06-10 11:45:24 -04:00
Al Viro 893839fd57 pipe_zero(): we don't need no stinkin' kmap_atomic()...
FWIW, memcpy_to_page() itself almost certainly ought to
use kmap_local_page()...

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-06-10 11:45:24 -04:00
Al Viro 2495bdcc86 iov_iter: clean csum_and_copy_...() primitives up a bit
1) kmap_atomic() is not needed here, kmap_local_page() is enough.
2) No need to make sum = csum_block_add(sum, next, off); conditional
upon next != 0 - adding 0 is a no-op as far as csum_block_add()
is concerned.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-06-10 11:45:23 -04:00
Al Viro 55ca375c5d copy_page_from_iter(): don't need kmap_atomic() for kvec/bvec cases
kmap_local_page() is enough.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-06-10 11:45:22 -04:00
Al Viro c1d4d6a9ae copy_page_to_iter(): don't bother with kmap_atomic() for bvec/kvec cases
kmap_local_page() is enough there.  Moreover, we can use _copy_to_iter()
for actual copying in those cases - no useful extra checks on the
address we are copying from in that call.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-06-10 11:45:22 -04:00
Al Viro 4b179e9a9c iterate_xarray(): only of the first iteration we might get offset != 0
recalculating offset on each iteration is pointless - on all subsequent
passes through the loop it will be zero anyway.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-06-10 11:45:21 -04:00
Al Viro a6e4ec7bfd pull handling of ->iov_offset into iterate_{iovec,bvec,xarray}
fewer arguments (by one, but still...) for iterate_...() macros

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-06-10 11:45:20 -04:00
Al Viro 7baa509900 iov_iter: make iterator callbacks use base and len instead of iovec
Iterator macros used to provide the arguments for step callbacks in
a structure matching the flavour - iovec for ITER_IOVEC, kvec for
ITER_KVEC and bio_vec for ITER_BVEC.  That already broke down for
ITER_XARRAY (bio_vec there); now that we are using kvec callback
for bvec and xarray cases, we are always passing a pointer + length
(void __user * + size_t for ITER_IOVEC callback, void * + size_t
for everything else).

Note that the original reason for bio_vec (page + offset + len) in
case of ITER_BVEC used to be that we did *not* want to kmap a
page when all we wanted was e.g. to find the alignment of its
subrange.  Now all such users are gone and the ones that are left
want the page mapped anyway for actually copying the data.

So in all cases we have pointer + length, and there's no good
reason for keeping those in struct iovec or struct kvec - we
can just pass them to callback separately.

Again, less boilerplate in callbacks...

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-06-10 11:45:20 -04:00
Al Viro 622838f3fd iov_iter: make the amount already copied available to iterator callbacks
Making iterator macros keep track of the amount of data copied is pretty
easy and it has several benefits:
	1) we no longer need the mess like (from += v.iov_len) - v.iov_len
in the callbacks - initial value + total amount copied so far would do
just fine.
	2) less obviously, we no longer need to remember the initial amount
of data we wanted to copy; the loops in iterator macros are along the lines
of
	wanted = bytes;
	while (bytes) {
		copy some
		bytes -= copied
		if short copy
			break
	}
	bytes = wanted - bytes;
Replacement is
	offs = 0;
	while (bytes) {
		copy some
		offs += copied
		bytes -= copied
		if short copy
			break
	}
	bytes = offs;
That wouldn't be a win per se, but unlike the initial value of bytes, the amount
copied so far *is* useful in callbacks.
	3) in some cases (csum_and_copy_..._iter()) we already had offs manually
maintained by the callbacks.  With that change we can drop that.

	Less boilerplate and more readable code...

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-06-10 11:45:19 -04:00
Al Viro 21b56c8477 iov_iter: get rid of separate bvec and xarray callbacks
After the previous commit we have
	* xarray and bvec callbacks idential in all cases
	* both equivalent to kvec callback wrapped into
kmap_local_page()/kunmap_local() pair.

So we can pass only two (iovec and kvec) callbacks to
iterate_and_advance() and let iterate_{bvec,xarray} wrap
it into kmap_local_page()/kunmap_local_page().

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-06-10 11:45:18 -04:00
Al Viro 1b4fb5ffd7 iov_iter: teach iterate_{bvec,xarray}() about possible short copies
... and now we finally can sort out the mess in _copy_mc_to_iter().
Provide a variant of iterate_and_advance() that does *NOT* ignore
the return values of bvec, xarray and kvec callbacks, use that in
_copy_mc_to_iter().  That gets rid of magic in those callbacks -
we used to need it so we'd get at least the right return value in
case of failure halfway through.

As a bonus, now iterator is advanced by the amount actually copied
for all flavours.  That's what the callers expect and it used to do that
correctly in iovec and xarray cases.  However, in kvec and bvec cases
the iterator had not been advanced on such failures, breaking the users.
Fixed now...

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-06-10 11:45:18 -04:00
Al Viro 7491a2bf64 iterate_bvec(): expand bvec.h macro forest, massage a bit
... incidentally, using pointer instead of index in an array
(the only change here) trims half-kilobyte of .text...

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-06-10 11:45:17 -04:00
Al Viro 5c67aa90cd iov_iter: unify iterate_iovec and iterate_kvec
The differences between iterate_iovec and iterate_kvec are minor:
	* kvec callback is treated as if it returned 0
	* initialization of __p is with i->iov and i->kvec resp.
which is trivially dealt with.

No code generation changes - compiler is quite capable of turning
	left = ((void)(STEP), 0);
	__v.iov_len -= left;
(with no accesses to left downstream) and
	(void)(STEP);
into the same code.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-06-10 11:45:16 -04:00
Al Viro 7a1bcb5d25 iov_iter: massage iterate_iovec and iterate_kvec to logics similar to iterate_bvec
Premature optimization is the root of all evil...  Trying
to unroll the first pass through the loop makes it harder
to follow and not just for readers - compiler ends up
generating worse code than it would on a "non-optimized"
loop.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-06-10 11:45:16 -04:00
Al Viro f5da83545f iterate_and_advance(): get rid of magic in case when n is 0
iov_iter_advance() needs to do some non-trivial work when it's given
0 as argument (skip all empty iovecs, mostly).  We used to implement
it via iterate_and_advance(); we no longer do so and for all other
users of iterate_and_advance() zero length is a no-op.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-06-10 11:45:15 -04:00
Al Viro 594e450b3f csum_and_copy_to_iter(): massage into form closer to csum_and_copy_from_iter()
Namely, have off counted starting from 0 rather than from csstate->off.
To compensate we need to shift the initial value (csstate->sum) (rotate
by 8 bits, as usual for csum) and do the same after we are finished adding
the pieces up.

What we get out of that is a bit more redundancy in our variables - from
is always equal to addr + off, which will be useful several commits down
the road.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-06-10 11:45:14 -04:00
Al Viro f0b65f39ac iov_iter: replace iov_iter_copy_from_user_atomic() with iterator-advancing variant
Replacement is called copy_page_from_iter_atomic(); unlike the old primitive the
callers do *not* need to do iov_iter_advance() after it.  In case when they end
up consuming less than they'd been given they need to do iov_iter_revert() on
everything they had not consumed.  That, however, needs to be done only on slow
paths.

All in-tree callers converted.  And that kills the last user of iterate_all_kinds()

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-06-10 11:45:14 -04:00
Al Viro e4f8df8679 [xarray] iov_iter_npages(): just use DIV_ROUND_UP()
Compiler is capable of recognizing division by power of 2 and turning
it into shifts.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-06-10 11:45:13 -04:00
Al Viro 66531c65aa iov_iter_npages(): don't bother with iterate_all_kinds()
note that in bvec case pages can be compound ones - we can't just assume
that each segment is covered by one (sub)page

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-06-10 11:45:12 -04:00
Al Viro 3d671ca62a get rid of iterate_all_kinds() in iov_iter_get_pages()/iov_iter_get_pages_alloc()
Here iterate_all_kinds() is used just to find the first (non-empty, in
case of iovec) segment.  Which can be easily done explicitly.
Note that in bvec case we now can get more than PAGE_SIZE worth of them,
in case when we have a compound page in bvec and a range that crosses
a subpage boundary.  Older behaviour had been to stop on that boundary;
we used to get the right first page (for_each_bvec() took care of that),
but that was all we'd got.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-06-10 11:45:12 -04:00
Al Viro 610c7a7154 iov_iter_gap_alignment(): get rid of iterate_all_kinds()
For one thing, it's only used for iovec (and makes sense only for those).
For another, here we don't care about iov_offset, since the beginning of
the first segment and the end of the last one are ignored.  So it makes
a lot more sense to just walk through the iovec array...

We need to deal with the case of truncated iov_iter, but unlike the
situation with iov_iter_alignment() we don't care where the last
segment ends - just which segment is the last one.

[fixed a braino spotted by Qian Cai <quic_qiancai@quicinc.com>]

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-06-10 11:45:11 -04:00
Al Viro 9221d2e37b iov_iter_alignment(): don't bother with iterate_all_kinds()
It's easier to go over the array manually.  We need to watch out
for truncated iov_iter, though - iovec array might cover more
than i->count.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-06-10 11:45:10 -04:00
Al Viro 8409a0d261 sanitize iov_iter_fault_in_readable()
1) constify iov_iter argument; we are not advancing it in this primitive.

2) cap the amount requested by the amount of data in iov_iter.  All
existing callers should've been safe, but the check is really cheap and
doing it here makes for easier analysis, as well as more consistent
semantics among the primitives.

3) don't bother with iterate_iovec().  Explicit loop is not any harder
to follow, and we get rid of standalone iterate_iovec() users - it's
only used by iterate_and_advance() and (soon to be gone) iterate_all_kinds().

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-06-10 11:45:10 -04:00
Al Viro 185ac4d436 iov_iter: optimize iov_iter_advance() for iovec and kvec
We can do better than generic iterate_and_advance() for this one;
inspired by bvec_iter_advance() (and massaged into that form by
equivalent transformations).

[fixed a braino caught by kernel test robot <oliver.sang@intel.com>]

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-06-10 11:45:09 -04:00
Al Viro 8cd54c1c84 iov_iter: separate direction from flavour
Instead of having them mixed in iter->type, use separate ->iter_type
and ->data_source (u8 and bool resp.)  And don't bother with (pseudo-)
bitmap for the former - microoptimizations from being able to check
if the flavour is one of two values are not worth the confusion for
optimizer.  It can't prove that we never get e.g. ITER_IOVEC | ITER_PIPE,
so we end up with extra headache.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-06-10 11:45:08 -04:00
Al Viro 556351c1c0 iov_iter_advance(): don't modify ->iov_offset for ITER_DISCARD
the field is not used for that flavour

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-06-10 11:45:08 -04:00
Al Viro 28f38db7ed iov_iter: reorder handling of flavours in primitives
iovec is the most common one; test it first and test explicitly,
rather than "not anything else".  Replace all flavour checks with
use of iov_iter_is_...() helpers.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-06-10 11:45:07 -04:00
Al Viro 4b6c132b7d iov_iter: switch ..._full() variants of primitives to use of iov_iter_revert()
Use corresponding plain variants, revert on short copy.  That's the way it
should've been done from the very beginning, except that we didn't have
iov_iter_revert() back then...

[fixed another braino caught by Qian Cai <quic_qiancai@quicinc.com>]

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-06-10 11:44:23 -04:00
Al Viro 3b3fc051cd iov_iter_advance(): use consistent semantics for move past the end
asking to advance by more than we have left in the iov_iter should
move to the very end; it should *not* leave negative i->count and
it should not spew into syslog, etc. - it's a legitimate operation.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-06-03 10:36:52 -04:00
Al Viro 0e8f0d6740 [xarray] iov_iter_fault_in_readable() should do nothing in xarray case
... and actually should just check it's given an iovec-backed iterator
in the first place.

Cc: stable@vger.kernel.org
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-06-03 10:36:51 -04:00
Al Viro a506abc7b6 copy_page_to_iter(): fix ITER_DISCARD case
we need to advance the iterator...

Cc: stable@vger.kernel.org
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-06-03 10:36:51 -04:00
Al Viro 08aa647960 teach copy_page_to_iter() to handle compound pages
In situation when copy_page_to_iter() got a compound page the current
code would only work on systems with no CONFIG_HIGHMEM.  It *is* the majority
of real-world setups, or we would've drown in bug reports by now.  Still needs
fixing.

	Current variant works for solitary page; rename that to
__copy_page_to_iter() and turn the handling of compound pages into a loop over
subpages.

Cc: stable@vger.kernel.org
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-06-03 10:36:50 -04:00
Al Viro 8959a23924 fuse_fill_write_pages(): don't bother with iov_iter_single_seg_count()
another rudiment of fault-in originally having been limited to the
first segment, same as in generic_perform_write() and friends.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-06-03 10:34:55 -04:00
Al Viro bc1bb416bb generic_perform_write()/iomap_write_actor(): saner logics for short copy
if we run into a short copy and ->write_end() refuses to advance at all,
use the amount we'd managed to copy for the next iteration to handle.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-06-02 17:50:44 -04:00
Al Viro 9067931236 ntfs_copy_from_user_iter(): don't bother with copying iov_iter
Advance the original, let the caller revert if it needs to.
Don't mess with iov_iter_single_seg_count() in the caller -
if we got a (non-zero) short copy, use the amount actually
copied for the next pass, limit it to "up to the end
of page" if nothing got copied at all.

Originally fault-in only read the first iovec; back then it used
to make sense to limit to the just one iovec for the pass after
short copy.  These days it's no long true.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-06-02 17:50:38 -04:00
Al Viro e4b2755318 getcwd(2): clean up error handling
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-05-18 20:15:58 -04:00
Al Viro cf4febc1ad d_path: prepend_path() is unlikely to return non-zero
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-05-18 20:15:58 -04:00
Al Viro 008673ff74 d_path: prepend_path(): lift the inner loop into a new helper
... and leave the rename_lock/mount_lock handling in prepend_path()
itself

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-05-18 20:15:57 -04:00
Al Viro 2dac0ad175 d_path: prepend_path(): lift resetting b in case when we'd return 3 out of loop
preparation to extracting the loop into helper (next commit)

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-05-18 20:15:56 -04:00
Al Viro 7c0d552fd5 d_path: prepend_path(): get rid of vfsmnt
it's kept equal to &mnt->mnt all along.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-05-18 20:15:56 -04:00
Al Viro ad08ae5865 d_path: introduce struct prepend_buffer
We've a lot of places where we have pairs of form (pointer to end
of buffer, amount of space left in front of that).  These sit in pairs of
variables located next to each other and usually passed by reference.
Turn those into instances of new type (struct prepend_buffer) and pass
reference to the pair instead of pairs of references to its fields.

Declared and initialized by DECLARE_BUFFER(name, buf, buflen).

extract_string(prepend_buffer) returns the buffer contents if
no overflow has happened, ERR_PTR(ENAMETOOLONG) otherwise.
All places where we used to have that boilerplate converted to use
of that helper.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-05-18 20:14:53 -04:00
Al Viro 95b55c42f6 d_path: make prepend_name() boolean
It returns only 0 or -ENAMETOOLONG and both callers only check if
the result is negative.  Might as well return true on success and
false on failure...

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-05-18 20:08:12 -04:00
Al Viro 01a4428ee7 d_path: lift -ENAMETOOLONG handling into callers of prepend_path()
The only negative value ever returned by prepend_path() is -ENAMETOOLONG
and callers can recognize that situation (overflow) by looking at the
sign of buflen.  Lift that into the callers; we already have the
same logics (buf if buflen is non-negative, ERR_PTR(-ENAMETOOLONG) otherwise)
in several places and that'll become a new primitive several commits down
the road.

Make prepend_path() return 0 instead of -ENAMETOOLONG.  That makes for
saner calling conventions (0/1/2/3/-ENAMETOOLONG is obnoxious) and
callers actually get simpler, especially once the aforementioned
primitive gets added.

In prepend_path() itself we switch prepending the / (in case of
empty path) to use of prepend() - no need to open-code that, compiler
will do the right thing.  It's exactly the same logics as in
__dentry_path().

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-05-18 20:08:12 -04:00
Al Viro d8548232ea d_path: don't bother with return value of prepend()
Only simple_dname() checks it, and there we can simply do those
calls and check for overflow (by looking of negative buflen)
in the end.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-05-18 20:08:11 -04:00
Al Viro a0378fb9b3 getcwd(2): saner logics around prepend_path() call
The only negative value that might get returned by prepend_path() is
-ENAMETOOLONG, and that happens only on overflow.  The same goes for
prepend_unreachable().  Overflow is detectable by observing negative
buflen, so we can simplify the control flow around the prepend_path()
call.  Expand prepend_unreachable(), while we are at it - that's the
only caller.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-05-18 20:08:11 -04:00
Al Viro 9024348f53 d_path: get rid of path_with_deleted()
expand in the sole caller; transform the initial prepends similar to
what we'd done in dentry_path() (prepend_path() will fail the right
way if we call it with negative buflen, same as __dentry_path() does).

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-05-18 20:08:10 -04:00
Al Viro 3acca04326 d_path: regularize handling of root dentry in __dentry_path()
All path-forming primitives boil down to sequence of prepend_name()
on dentries encountered along the way toward root.  Each time we prepend
/ + dentry name to the buffer.  Normally that does exactly what we want,
but there's a corner case when we don't call prepend_name() at all (in case
of __dentry_path() that happens if we are given root dentry).  We obviously
want to end up with "/", rather than "", so this corner case needs to be
handled.

__dentry_path() used to manually put '/' in the end of buffer before
doing anything else, to be overwritten by the first call of prepend_name()
if one happens and to be left in place if we don't call prepend_name() at
all.  That required manually checking that we had space in the buffer
(prepend_name() and prepend() take care of such checks themselves) and lead
to clumsy keeping track of return value.

A better approach is to check if the main loop has added anything
into the buffer and prepend "/" if it hasn't.  A side benefit of using prepend()
is that it does the right thing if we'd already run out of buffer, making
the overflow-handling logics simpler.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-05-18 20:06:36 -04:00
Al Viro 3a291c974c d_path: saner calling conventions for __dentry_path()
1) lift NUL-termination into the callers
2) pass pointer to the end of buffer instead of that to beginning.

(1) allows to simplify dentry_path() - we don't need to play silly
games with restoring the leading / of "//deleted" after __dentry_path()
would've overwritten it with NUL.

We also do not need to check if (either) prepend() in there fails -
if the buffer is not large enough, we'll end with negative buflen
after prepend() and __dentry_path() will return the right value
(ERR_PTR(-ENAMETOOLONG)) just fine.

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-05-17 21:13:57 -04:00
Al Viro dfe5087675 d_path: "\0" is {0,0}, not {0}
Single-element array consisting of one NUL is spelled ""...

Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2021-05-17 21:11:45 -04:00