Pull more Kbuild updates from Masahiro Yamada:
- fix build rules in binderfs sample
- fix build errors when Kbuild recurses to the top Makefile
- covert '---help---' in Kconfig to 'help'
* tag 'kbuild-v5.8-2' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild:
treewide: replace '---help---' in Kconfig files with 'help'
kbuild: fix broken builds because of GZIP,BZIP2,LZOP variables
samples: binderfs: really compile this sample and fix build issues
Pull notification queue from David Howells:
"This adds a general notification queue concept and adds an event
source for keys/keyrings, such as linking and unlinking keys and
changing their attributes.
Thanks to Debarshi Ray, we do have a pull request to use this to fix a
problem with gnome-online-accounts - as mentioned last time:
https://gitlab.gnome.org/GNOME/gnome-online-accounts/merge_requests/47
Without this, g-o-a has to constantly poll a keyring-based kerberos
cache to find out if kinit has changed anything.
[ There are other notification pending: mount/sb fsinfo notifications
for libmount that Karel Zak and Ian Kent have been working on, and
Christian Brauner would like to use them in lxc, but let's see how
this one works first ]
LSM hooks are included:
- A set of hooks are provided that allow an LSM to rule on whether or
not a watch may be set. Each of these hooks takes a different
"watched object" parameter, so they're not really shareable. The
LSM should use current's credentials. [Wanted by SELinux & Smack]
- A hook is provided to allow an LSM to rule on whether or not a
particular message may be posted to a particular queue. This is
given the credentials from the event generator (which may be the
system) and the watch setter. [Wanted by Smack]
I've provided SELinux and Smack with implementations of some of these
hooks.
WHY
===
Key/keyring notifications are desirable because if you have your
kerberos tickets in a file/directory, your Gnome desktop will monitor
that using something like fanotify and tell you if your credentials
cache changes.
However, we also have the ability to cache your kerberos tickets in
the session, user or persistent keyring so that it isn't left around
on disk across a reboot or logout. Keyrings, however, cannot currently
be monitored asynchronously, so the desktop has to poll for it - not
so good on a laptop. This facility will allow the desktop to avoid the
need to poll.
DESIGN DECISIONS
================
- The notification queue is built on top of a standard pipe. Messages
are effectively spliced in. The pipe is opened with a special flag:
pipe2(fds, O_NOTIFICATION_PIPE);
The special flag has the same value as O_EXCL (which doesn't seem
like it will ever be applicable in this context)[?]. It is given up
front to make it a lot easier to prohibit splice&co from accessing
the pipe.
[?] Should this be done some other way? I'd rather not use up a new
O_* flag if I can avoid it - should I add a pipe3() system call
instead?
The pipe is then configured::
ioctl(fds[1], IOC_WATCH_QUEUE_SET_SIZE, queue_depth);
ioctl(fds[1], IOC_WATCH_QUEUE_SET_FILTER, &filter);
Messages are then read out of the pipe using read().
- It should be possible to allow write() to insert data into the
notification pipes too, but this is currently disabled as the
kernel has to be able to insert messages into the pipe *without*
holding pipe->mutex and the code to make this work needs careful
auditing.
- sendfile(), splice() and vmsplice() are disabled on notification
pipes because of the pipe->mutex issue and also because they
sometimes want to revert what they just did - but one or more
notification messages might've been interleaved in the ring.
- The kernel inserts messages with the wait queue spinlock held. This
means that pipe_read() and pipe_write() have to take the spinlock
to update the queue pointers.
- Records in the buffer are binary, typed and have a length so that
they can be of varying size.
This allows multiple heterogeneous sources to share a common
buffer; there are 16 million types available, of which I've used
just a few, so there is scope for others to be used. Tags may be
specified when a watchpoint is created to help distinguish the
sources.
- Records are filterable as types have up to 256 subtypes that can be
individually filtered. Other filtration is also available.
- Notification pipes don't interfere with each other; each may be
bound to a different set of watches. Any particular notification
will be copied to all the queues that are currently watching for it
- and only those that are watching for it.
- When recording a notification, the kernel will not sleep, but will
rather mark a queue as having lost a message if there's
insufficient space. read() will fabricate a loss notification
message at an appropriate point later.
- The notification pipe is created and then watchpoints are attached
to it, using one of:
keyctl_watch_key(KEY_SPEC_SESSION_KEYRING, fds[1], 0x01);
watch_mount(AT_FDCWD, "/", 0, fd, 0x02);
watch_sb(AT_FDCWD, "/mnt", 0, fd, 0x03);
where in both cases, fd indicates the queue and the number after is
a tag between 0 and 255.
- Watches are removed if either the notification pipe is destroyed or
the watched object is destroyed. In the latter case, a message will
be generated indicating the enforced watch removal.
Things I want to avoid:
- Introducing features that make the core VFS dependent on the
network stack or networking namespaces (ie. usage of netlink).
- Dumping all this stuff into dmesg and having a daemon that sits
there parsing the output and distributing it as this then puts the
responsibility for security into userspace and makes handling
namespaces tricky. Further, dmesg might not exist or might be
inaccessible inside a container.
- Letting users see events they shouldn't be able to see.
TESTING AND MANPAGES
====================
- The keyutils tree has a pipe-watch branch that has keyctl commands
for making use of notifications. Proposed manual pages can also be
found on this branch, though a couple of them really need to go to
the main manpages repository instead.
If the kernel supports the watching of keys, then running "make
test" on that branch will cause the testing infrastructure to spawn
a monitoring process on the side that monitors a notifications pipe
for all the key/keyring changes induced by the tests and they'll
all be checked off to make sure they happened.
https://git.kernel.org/pub/scm/linux/kernel/git/dhowells/keyutils.git/log/?h=pipe-watch
- A test program is provided (samples/watch_queue/watch_test) that
can be used to monitor for keyrings, mount and superblock events.
Information on the notifications is simply logged to stdout"
* tag 'notifications-20200601' of git://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs:
smack: Implement the watch_key and post_notification hooks
selinux: Implement the watch_key security hook
keys: Make the KEY_NEED_* perms an enum rather than a mask
pipe: Add notification lossage handling
pipe: Allow buffers to be marked read-whole-or-error for notifications
Add sample notification program
watch_queue: Add a key/keyring notification facility
security: Add hooks to rule on setting a watch
pipe: Add general notification queue support
pipe: Add O_NOTIFICATION_PIPE
security: Add a hook for the point of notification insertion
uapi: General notification queue definitions
Even after commit c624adc9cb ("samples: fix binderfs sample"), this
sample is never compiled.
'hostprogs' teaches Kbuild that this is a host program, but not enough
to order to compile it. You must add it to 'always-y' to really compile
it.
Since this sample has never been compiled in upstream, various issues
are left unnoticed.
[1] compilers without <linux/android/binderfs.h> are still widely used
<linux/android/binderfs.h> is only available since commit c13295ad21
("binderfs: rename header to binderfs.h"), i.e., Linux 5.0
If your compiler is based on UAPI headers older than Linux 5.0, you
will see the following error:
samples/binderfs/binderfs_example.c:16:10: fatal error: linux/android/binderfs.h: No such file or directory
#include <linux/android/binderfs.h>
^~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
You cannot rely on compilers having such a new header.
The common approach is to install UAPI headers of this kernel into
usr/include, and then add it to the header search path.
I added 'depends on HEADERS_INSTALL' in Kconfig, and '-I usr/include'
compiler flag in Makefile.
[2] compile the sample for target architecture
Because headers_install works for the target architecture, only the
native compiler was able to build sample code that requires
'-I usr/include'.
Commit 7f3a59db27 ("kbuild: add infrastructure to build userspace
programs") added the new syntax 'userprogs' to compile user-space
programs for the target architecture.
Use it, and then 'ifndef CROSS_COMPILE' will go away.
I added 'depends on CC_CAN_LINK' because $(CC) is not necessarily
capable of linking user-space programs.
[3] use subdir-y to descend into samples/binderfs
Since this directory does not contain any kernel-space code, it has no
point in generating built-in.a or modules.order.
Replace obj-$(CONFIG_...) with subdir-$(CONFIG_...).
[4] -Wunused-variable warning
If I compile this, I see the following warning.
samples/binderfs/binderfs_example.c: In function 'main':
samples/binderfs/binderfs_example.c:21:9: warning: unused variable 'len' [-Wunused-variable]
21 | size_t len;
| ^~~
I removed the unused 'len'.
[5] CONFIG_ANDROID_BINDERFS is not required
Since this is a user-space standalone program, it is independent of
the kernel configuration.
Remove 'depends on ANDROID_BINDERFS'.
Fixes: 9762dc1432 ("samples: add binderfs sample program")
Fixes: c624adc9cb ("samples: fix binderfs sample")
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Acked-by: Christian Brauner <christian.brauner@ubuntu.com>
The sample program is run like:
./samples/watch_queue/watch_test
and watches "/" for mount changes and the current session keyring for key
changes:
# keyctl add user a a @s
1035096409
# keyctl unlink 1035096409 @s
producing:
# ./watch_test
read() = 16
NOTIFY[000]: ty=000001 sy=02 i=00000110
KEY 2ffc2e5d change=2[linked] aux=1035096409
read() = 16
NOTIFY[000]: ty=000001 sy=02 i=00000110
KEY 2ffc2e5d change=3[unlinked] aux=1035096409
Other events may be produced, such as with a failing disk:
read() = 22
NOTIFY[000]: ty=000003 sy=02 i=00000416
USB 3-7.7 dev-reset e=0 r=0
read() = 24
NOTIFY[000]: ty=000002 sy=06 i=00000418
BLOCK 00800050 e=6[critical medium] s=64000ef8
This corresponds to:
blk_update_request: critical medium error, dev sdf, sector 1677725432 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
in dmesg.
Signed-off-by: David Howells <dhowells@redhat.com>
Kbuild now supports the 'userprogs' syntax to compile userspace
programs for the same architecture as the kernel.
Add the entry to samples/Makefile to put this into the build bot
coverage.
I also added the CONFIG option guarded by 'depends on CC_CAN_LINK'
because $(CC) may not provide libc.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Acked-by: Sam Ravnborg <sam@ravnborg.org>
Kbuild now supports the 'userprogs' syntax to compile userspace
programs for the same architecture as the kernel.
Add the entry to samples/Makefile to put this into the build bot
coverage.
I also added the CONFIG option guarded by 'depends on CC_CAN_LINK'
because $(CC) may not provide libc.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Acked-by: Sam Ravnborg <sam@ravnborg.org>
Kbuild now supports the 'userprogs' syntax to compile userspace
programs for the same architecture as the kernel.
Add the entry to samples/Makefile to put this into the build bot
coverage.
I also added the CONFIG option guarded by 'depends on CC_CAN_LINK'
because $(CC) may not provide libc.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Acked-by: Miguel Ojeda <miguel.ojeda.sandonis@gmail.com>
Acked-by: Sam Ravnborg <sam@ravnborg.org>
This userspace program includes UAPI headers exported to usr/include/.
'make headers' always works for the target architecture (i.e. the same
architecture as the kernel), so the sample program should be built for
the target as well. Kbuild now supports 'userprogs' for that.
Add the entry to samples/Makefile to put this into the build bot
coverage.
I also added the CONFIG option guarded by 'depends on CC_CAN_LINK'
because $(CC) may not provide libc.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Acked-by: Sam Ravnborg <sam@ravnborg.org>
This reverts commit 8728497895.
samples/ contains only sub-directories.
Because OBJECT_FILES_NON_STANDARD does not work recursively,
this line has no effect.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Acked-by: Josh Poimboeuf <jpoimboe@redhat.com>
Acked-by: Sam Ravnborg <sam@ravnborg.org>
Pull tracing updates from Steven Rostedt:
"New tracing features:
- New PERMANENT flag to ftrace_ops when attaching a callback to a
function.
As /proc/sys/kernel/ftrace_enabled when set to zero will disable
all attached callbacks in ftrace, this has a detrimental impact on
live kernel tracing, as it disables all that it patched. If a
ftrace_ops is registered to ftrace with the PERMANENT flag set, it
will prevent ftrace_enabled from being disabled, and if
ftrace_enabled is already disabled, it will prevent a ftrace_ops
with PREMANENT flag set from being registered.
- New register_ftrace_direct().
As eBPF would like to register its own trampolines to be called by
the ftrace nop locations directly, without going through the ftrace
trampoline, this function has been added. This allows for eBPF
trampolines to live along side of ftrace, perf, kprobe and live
patching. It also utilizes the ftrace enabled_functions file that
keeps track of functions that have been modified in the kernel, to
allow for security auditing.
- Allow for kernel internal use of ftrace instances.
Subsystems in the kernel can now create and destroy their own
tracing instances which allows them to have their own tracing
buffer, and be able to record events without worrying about other
users from writing over their data.
- New seq_buf_hex_dump() that lets users use the hex_dump() in their
seq_buf usage.
- Notifications now added to tracing_max_latency to allow user space
to know when a new max latency is hit by one of the latency
tracers.
- Wider spread use of generic compare operations for use of bsearch
and friends.
- More synthetic event fields may be defined (32 up from 16)
- Use of xarray for architectures with sparse system calls, for the
system call trace events.
This along with small clean ups and fixes"
* tag 'trace-v5.5' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace: (51 commits)
tracing: Enable syscall optimization for MIPS
tracing: Use xarray for syscall trace events
tracing: Sample module to demonstrate kernel access to Ftrace instances.
tracing: Adding new functions for kernel access to Ftrace instances
tracing: Fix Kconfig indentation
ring-buffer: Fix typos in function ring_buffer_producer
ftrace: Use BIT() macro
ftrace: Return ENOTSUPP when DYNAMIC_FTRACE_WITH_DIRECT_CALLS is not configured
ftrace: Rename ftrace_graph_stub to ftrace_stub_graph
ftrace: Add a helper function to modify_ftrace_direct() to allow arch optimization
ftrace: Add helper find_direct_entry() to consolidate code
ftrace: Add another check for match in register_ftrace_direct()
ftrace: Fix accounting bug with direct->count in register_ftrace_direct()
ftrace/selftests: Fix spelling mistake "wakeing" -> "waking"
tracing: Increase SYNTH_FIELDS_MAX for synthetic_events
ftrace/samples: Add a sample module that implements modify_ftrace_direct()
ftrace: Add modify_ftrace_direct()
tracing: Add missing "inline" in stub function of latency_fsnotify()
tracing: Remove stray tab in TRACE_EVAL_MAP_FILE's help text
tracing: Use seq_buf_hex_dump() to dump buffers
...
Add a sample module that shows a simple use case for
regsiter_ftrace_direct(), and how to use it.
Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
Commit 5318321d36 ("samples: disable CONFIG_SAMPLES for UML") used
a big hammer to fix the build errors under the samples/ directory.
Only some samples actually include uapi headers from usr/include.
Introduce CONFIG_HEADERS_INSTALL since 'depends on HEADERS_INSTALL' is
clearer than 'depends on !UML'. If this option is enabled, uapi headers
are installed before starting directory descending.
I added 'depends on HEADERS_INSTALL' to per-sample CONFIG options.
This allows UML to compile some samples.
$ make ARCH=um allmodconfig samples/
[ snip ]
CC [M] samples/configfs/configfs_sample.o
CC [M] samples/kfifo/bytestream-example.o
CC [M] samples/kfifo/dma-example.o
CC [M] samples/kfifo/inttype-example.o
CC [M] samples/kfifo/record-example.o
CC [M] samples/kobject/kobject-example.o
CC [M] samples/kobject/kset-example.o
CC [M] samples/trace_events/trace-events-sample.o
CC [M] samples/trace_printk/trace-printk.o
AR samples/vfio-mdev/built-in.a
AR samples/built-in.a
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Do not descend to sub-directories when unneeded.
I used subdir-$(CONFIG_...) for hidraw, seccomp, and vfs because
they only contain host programs.
While we are here, let's add SPDX License tag, and sort the directories
alphabetically.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Pull Kbuild updates from Masahiro Yamada:
- allow users to invoke 'make' out of the source tree
- refactor scripts/mkmakefile
- deprecate KBUILD_SRC, which was used to track the source tree
location for O= build.
- fix recordmcount.pl in case objdump output is localized
- turn unresolved symbols in external modules to errors from warnings
by default; pass KBUILD_MODPOST_WARN=1 to get them back to warnings
- generate modules.builtin.modinfo to collect .modinfo data from
built-in modules
- misc Makefile cleanups
* tag 'kbuild-v5.2' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: (21 commits)
.gitignore: add more all*.config patterns
moduleparam: Save information about built-in modules in separate file
Remove MODULE_ALIAS() calls that take undefined macro
.gitignore: add leading and trailing slashes to generated directories
scripts/tags.sh: fix direct execution of scripts/tags.sh
scripts: override locale from environment when running recordmcount.pl
samples: kobject: allow CONFIG_SAMPLE_KOBJECT to become y
samples: seccomp: turn CONFIG_SAMPLE_SECCOMP into a bool option
kbuild: move Documentation to vmlinux-alldirs
kbuild: move samples/ to KBUILD_VMLINUX_OBJS
modpost: make KBUILD_MODPOST_WARN also configurable for external modules
kbuild: check arch/$(SRCARCH)/include/generated before out-of-tree build
kbuild: remove unneeded dependency for include/config/kernel.release
memory: squash drivers/memory/Makefile.asm-offsets
kbuild: use $(srctree) instead of KBUILD_SRC to check out-of-tree build
kbuild: mkmakefile: generate a simple wrapper of top Makefile
kbuild: mkmakefile: do not check the generated Makefile marker
kbuild: allow Kbuild to start from any directory
kbuild: pass $(MAKECMDGOALS) to sub-make as is
kbuild: fix warning "overriding recipe for target 'Makefile'"
...
Pull mount ABI updates from Al Viro:
"The syscalls themselves, finally.
That's not all there is to that stuff, but switching individual
filesystems to new methods is fortunately independent from everything
else, so e.g. NFS series can go through NFS tree, etc.
As those conversions get done, we'll be finally able to get rid of a
bunch of duplication in fs/super.c introduced in the beginning of the
entire thing. I expect that to be finished in the next window..."
* 'work.mount-syscalls' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs:
vfs: Add a sample program for the new mount API
vfs: syscall: Add fspick() to select a superblock for reconfiguration
vfs: syscall: Add fsmount() to create a mount for a superblock
vfs: syscall: Add fsconfig() for configuring and managing a context
vfs: Implement logging through fs_context
vfs: syscall: Add fsopen() to prepare for superblock creation
Make anon_inodes unconditional
teach move_mount(2) to work with OPEN_TREE_CLONE
vfs: syscall: Add move_mount(2) to move mounts around
vfs: syscall: Add open_tree(2) to reference or clone a mount
This is a sample program showing userspace how to get race-free access
to process metadata from a pidfd. It is rather easy to do and userspace
can actually simply reuse code that currently parses a process's status
file in procfs.
The program can easily be extended into a generic helper suitable for
inclusion in a libc to make it even easier for userspace to gain metadata
access.
Since this came up in a discussion because this API is going to be used
in various service managers: A lot of programs will have a whitelist
seccomp filter that returns <some-errno> for all new syscalls. This
means that programs might get confused if CLONE_PIDFD works but the
later pidfd_send_signal() syscall doesn't. Hence, here's a ahead of
time check that pidfd_send_signal() is supported:
bool pidfd_send_signal_supported()
{
int procfd = open("/proc/self", O_DIRECTORY | O_RDONLY | O_CLOEXEC);
if (procfd < 0)
return false;
/*
* A process is always allowed to signal itself so
* pidfd_send_signal() should never fail this test. If it does
* it must mean it is not available, blocked by an LSM, seccomp,
* or other.
*/
return pidfd_send_signal(procfd, 0, NULL, 0) == 0;
}
Signed-off-by: Christian Brauner <christian@brauner.io>
Co-developed-by: Jann Horn <jannh@google.com>
Signed-off-by: Jann Horn <jannh@google.com>
Reviewed-by: Oleg Nesterov <oleg@redhat.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: "Eric W. Biederman" <ebiederm@xmission.com>
Cc: Kees Cook <keescook@chromium.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: David Howells <dhowells@redhat.com>
Cc: "Michael Kerrisk (man-pages)" <mtk.manpages@gmail.com>
Cc: Andy Lutomirsky <luto@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Aleksa Sarai <cyphar@cyphar.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Handle samples/ like the other top-level directories to simplify
the Makefile.
Include include/config/auto.conf earlier to evaluate
drivers-$(CONFIG_SAMPLES).
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Add a sample program to demonstrate fsopen/fsmount/move_mount to mount
something.
To make it compile on all arches, irrespective of whether or not syscall
numbers are assigned, define the syscall number to -1 if it isn't to cause
the kernel to return -ENOSYS.
Signed-off-by: David Howells <dhowells@redhat.com>
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
This adds a simple sample program mounting binderfs and adding, then
removing a binder device. Hopefully, it will be helpful to users who want
to know how binderfs is supposed to be used.
Signed-off-by: Christian Brauner <christian.brauner@ubuntu.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Pull remoteproc updates from Bjorn Andersson:
- add support for generating coredumps for remoteprocs using
devcoredump
- add the Qualcomm sysmon driver for intra-remoteproc crash handling
- a number of fixes in Qualcomm and IMX drivers
* tag 'rproc-v4.17' of git://github.com/andersson/remoteproc:
remoteproc: fix null pointer dereference on glink only platforms
soc: qcom: qmi: add CONFIG_NET dependency
remoteproc: imx_rproc: Slightly simplify code in 'imx_rproc_probe()'
remoteproc: imx_rproc: Re-use existing error handling path in 'imx_rproc_probe()'
remoteproc: imx_rproc: Fix an error handling path in 'imx_rproc_probe()'
samples: Introduce Qualcomm QMI sample client
remoteproc: qcom: Introduce sysmon
remoteproc: Pass type of shutdown to subdev remove
remoteproc: qcom: Register segments for core dump
soc: qcom: mdt-loader: Return relocation base
remoteproc: Rename "load_rsc_table" to "parse_fw"
remoteproc: Add remote processor coredump support
remoteproc: Remove null character write of shared mem
The Analog Devices Blackfin port was added in 2007 and was rather
active for a while, but all work on it has come to a standstill
over time, as Analog have changed their product line-up.
Aaron Wu confirmed that the architecture port is no longer relevant,
and multiple people suggested removing blackfin independently because
of some of its oddities like a non-working SMP port, and the amount of
duplication between the chip variants, which cause extra work when
doing cross-architecture changes.
Link: https://docs.blackfin.uclinux.org/
Acked-by: Aaron Wu <Aaron.Wu@analog.com>
Acked-by: Bryan Wu <cooloney@gmail.com>
Cc: Steven Miao <realmz6@gmail.com>
Cc: Mike Frysinger <vapier@chromium.org>
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
