Merge branch 'for-mingo' of git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu into locking/kcsan

Pull the KCSAN subsystem from Paul E. McKenney:

   "This pull request contains base kernel concurrency sanitizer
    (KCSAN) enablement for x86, courtesy of Marco Elver.  KCSAN is a
    sampling watchpoint-based data-race detector, and is documented in
    Documentation/dev-tools/kcsan.rst.  KCSAN was announced in September,
    and much feedback has since been incorporated:

      http://lkml.kernel.org/r/CANpmjNPJ_bHjfLZCAPV23AXFfiPiyXXqqu72n6TgWzb2Gnu1eA@mail.gmail.com

    The data races located thus far have resulted in a number of fixes:

      https://github.com/google/ktsan/wiki/KCSAN#upstream-fixes-of-data-races-found-by-kcsan

    Additional information may be found here:

      https://lore.kernel.org/lkml/20191114180303.66955-1-elver@google.com/
   "

Signed-off-by: Ingo Molnar <mingo@kernel.org>
This commit is contained in:
Ingo Molnar
2019-11-19 19:56:28 +01:00
45 changed files with 2623 additions and 206 deletions
+1
View File
@@ -21,6 +21,7 @@ whole; patches welcome!
kasan
ubsan
kmemleak
kcsan
gdb-kernel-debugging
kgdb
kselftest
+256
View File
@@ -0,0 +1,256 @@
The Kernel Concurrency Sanitizer (KCSAN)
========================================
Overview
--------
*Kernel Concurrency Sanitizer (KCSAN)* is a dynamic data race detector for
kernel space. KCSAN is a sampling watchpoint-based data race detector. Key
priorities in KCSAN's design are lack of false positives, scalability, and
simplicity. More details can be found in `Implementation Details`_.
KCSAN uses compile-time instrumentation to instrument memory accesses. KCSAN is
supported in both GCC and Clang. With GCC it requires version 7.3.0 or later.
With Clang it requires version 7.0.0 or later.
Usage
-----
To enable KCSAN configure kernel with::
CONFIG_KCSAN = y
KCSAN provides several other configuration options to customize behaviour (see
their respective help text for more info).
Error reports
~~~~~~~~~~~~~
A typical data race report looks like this::
==================================================================
BUG: KCSAN: data-race in generic_permission / kernfs_refresh_inode
write to 0xffff8fee4c40700c of 4 bytes by task 175 on cpu 4:
kernfs_refresh_inode+0x70/0x170
kernfs_iop_permission+0x4f/0x90
inode_permission+0x190/0x200
link_path_walk.part.0+0x503/0x8e0
path_lookupat.isra.0+0x69/0x4d0
filename_lookup+0x136/0x280
user_path_at_empty+0x47/0x60
vfs_statx+0x9b/0x130
__do_sys_newlstat+0x50/0xb0
__x64_sys_newlstat+0x37/0x50
do_syscall_64+0x85/0x260
entry_SYSCALL_64_after_hwframe+0x44/0xa9
read to 0xffff8fee4c40700c of 4 bytes by task 166 on cpu 6:
generic_permission+0x5b/0x2a0
kernfs_iop_permission+0x66/0x90
inode_permission+0x190/0x200
link_path_walk.part.0+0x503/0x8e0
path_lookupat.isra.0+0x69/0x4d0
filename_lookup+0x136/0x280
user_path_at_empty+0x47/0x60
do_faccessat+0x11a/0x390
__x64_sys_access+0x3c/0x50
do_syscall_64+0x85/0x260
entry_SYSCALL_64_after_hwframe+0x44/0xa9
Reported by Kernel Concurrency Sanitizer on:
CPU: 6 PID: 166 Comm: systemd-journal Not tainted 5.3.0-rc7+ #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014
==================================================================
The header of the report provides a short summary of the functions involved in
the race. It is followed by the access types and stack traces of the 2 threads
involved in the data race.
The other less common type of data race report looks like this::
==================================================================
BUG: KCSAN: data-race in e1000_clean_rx_irq+0x551/0xb10
race at unknown origin, with read to 0xffff933db8a2ae6c of 1 bytes by interrupt on cpu 0:
e1000_clean_rx_irq+0x551/0xb10
e1000_clean+0x533/0xda0
net_rx_action+0x329/0x900
__do_softirq+0xdb/0x2db
irq_exit+0x9b/0xa0
do_IRQ+0x9c/0xf0
ret_from_intr+0x0/0x18
default_idle+0x3f/0x220
arch_cpu_idle+0x21/0x30
do_idle+0x1df/0x230
cpu_startup_entry+0x14/0x20
rest_init+0xc5/0xcb
arch_call_rest_init+0x13/0x2b
start_kernel+0x6db/0x700
Reported by Kernel Concurrency Sanitizer on:
CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.3.0-rc7+ #2
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014
==================================================================
This report is generated where it was not possible to determine the other
racing thread, but a race was inferred due to the data value of the watched
memory location having changed. These can occur either due to missing
instrumentation or e.g. DMA accesses.
Selective analysis
~~~~~~~~~~~~~~~~~~
To disable KCSAN data race detection for an entire subsystem, add to the
respective ``Makefile``::
KCSAN_SANITIZE := n
To disable KCSAN on a per-file basis, add to the ``Makefile``::
KCSAN_SANITIZE_file.o := n
KCSAN also understands the ``data_race(expr)`` annotation, which tells KCSAN
that any data races due to accesses in ``expr`` should be ignored and resulting
behaviour when encountering a data race is deemed safe.
debugfs
~~~~~~~
* The file ``/sys/kernel/debug/kcsan`` can be read to get stats.
* KCSAN can be turned on or off by writing ``on`` or ``off`` to
``/sys/kernel/debug/kcsan``.
* Writing ``!some_func_name`` to ``/sys/kernel/debug/kcsan`` adds
``some_func_name`` to the report filter list, which (by default) blacklists
reporting data races where either one of the top stackframes are a function
in the list.
* Writing either ``blacklist`` or ``whitelist`` to ``/sys/kernel/debug/kcsan``
changes the report filtering behaviour. For example, the blacklist feature
can be used to silence frequently occurring data races; the whitelist feature
can help with reproduction and testing of fixes.
Data Races
----------
Informally, two operations *conflict* if they access the same memory location,
and at least one of them is a write operation. In an execution, two memory
operations from different threads form a **data race** if they *conflict*, at
least one of them is a *plain access* (non-atomic), and they are *unordered* in
the "happens-before" order according to the `LKMM
<../../tools/memory-model/Documentation/explanation.txt>`_.
Relationship with the Linux Kernel Memory Model (LKMM)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The LKMM defines the propagation and ordering rules of various memory
operations, which gives developers the ability to reason about concurrent code.
Ultimately this allows to determine the possible executions of concurrent code,
and if that code is free from data races.
KCSAN is aware of *atomic* accesses (``READ_ONCE``, ``WRITE_ONCE``,
``atomic_*``, etc.), but is oblivious of any ordering guarantees. In other
words, KCSAN assumes that as long as a plain access is not observed to race
with another conflicting access, memory operations are correctly ordered.
This means that KCSAN will not report *potential* data races due to missing
memory ordering. If, however, missing memory ordering (that is observable with
a particular compiler and architecture) leads to an observable data race (e.g.
entering a critical section erroneously), KCSAN would report the resulting
data race.
Race conditions vs. data races
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Race conditions are logic bugs, where unexpected interleaving of racing
concurrent operations result in an erroneous state.
Data races on the other hand are defined at the *memory model/language level*.
Many data races are also harmful race conditions, which a tool like KCSAN
reports! However, not all data races are race conditions and vice-versa.
KCSAN's intent is to report data races according to the LKMM. A data race
detector can only work at the memory model/language level.
Deeper analysis, to find high-level race conditions only, requires conveying
the intended kernel logic to a tool. This requires (1) the developer writing a
specification or model of their code, and then (2) the tool verifying that the
implementation matches. This has been done for small bits of code using model
checkers and other formal methods, but does not scale to the level of what can
be covered with a dynamic analysis based data race detector such as KCSAN.
For reasons outlined in this `article <https://lwn.net/Articles/793253/>`_,
data races can be much more subtle, but can cause no less harm than high-level
race conditions.
Implementation Details
----------------------
The general approach is inspired by `DataCollider
<http://usenix.org/legacy/events/osdi10/tech/full_papers/Erickson.pdf>`_.
Unlike DataCollider, KCSAN does not use hardware watchpoints, but instead
relies on compiler instrumentation. Watchpoints are implemented using an
efficient encoding that stores access type, size, and address in a long; the
benefits of using "soft watchpoints" are portability and greater flexibility in
limiting which accesses trigger a watchpoint.
More specifically, KCSAN requires instrumenting plain (unmarked, non-atomic)
memory operations; for each instrumented plain access:
1. Check if a matching watchpoint exists; if yes, and at least one access is a
write, then we encountered a racing access.
2. Periodically, if no matching watchpoint exists, set up a watchpoint and
stall for a small delay.
3. Also check the data value before the delay, and re-check the data value
after delay; if the values mismatch, we infer a race of unknown origin.
To detect data races between plain and atomic memory operations, KCSAN also
annotates atomic accesses, but only to check if a watchpoint exists
(``kcsan_check_atomic_*``); i.e. KCSAN never sets up a watchpoint on atomic
accesses.
Key Properties
~~~~~~~~~~~~~~
1. **Memory Overhead:** The current implementation uses a small array of longs
to encode watchpoint information, which is negligible.
2. **Performance Overhead:** KCSAN's runtime aims to be minimal, using an
efficient watchpoint encoding that does not require acquiring any shared
locks in the fast-path. For kernel boot on a system with 8 CPUs:
- 5.0x slow-down with the default KCSAN config;
- 2.8x slow-down from runtime fast-path overhead only (set very large
``KCSAN_SKIP_WATCH`` and unset ``KCSAN_SKIP_WATCH_RANDOMIZE``).
3. **Annotation Overheads:** Minimal annotations are required outside the KCSAN
runtime. As a result, maintenance overheads are minimal as the kernel
evolves.
4. **Detects Racy Writes from Devices:** Due to checking data values upon
setting up watchpoints, racy writes from devices can also be detected.
5. **Memory Ordering:** KCSAN is *not* explicitly aware of the LKMM's ordering
rules; this may result in missed data races (false negatives).
6. **Analysis Accuracy:** For observed executions, due to using a sampling
strategy, the analysis is *unsound* (false negatives possible), but aims to
be complete (no false positives).
Alternatives Considered
-----------------------
An alternative data race detection approach for the kernel can be found in
`Kernel Thread Sanitizer (KTSAN) <https://github.com/google/ktsan/wiki>`_.
KTSAN is a happens-before data race detector, which explicitly establishes the
happens-before order between memory operations, which can then be used to
determine data races as defined in `Data Races`_. To build a correct
happens-before relation, KTSAN must be aware of all ordering rules of the LKMM
and synchronization primitives. Unfortunately, any omission leads to false
positives, which is especially important in the context of the kernel which
includes numerous custom synchronization mechanisms. Furthermore, KTSAN's
implementation requires metadata for each memory location (shadow memory);
currently, for each page, KTSAN requires 4 pages of shadow memory.
+11
View File
@@ -8850,6 +8850,17 @@ F: Documentation/kbuild/kconfig*
F: scripts/kconfig/
F: scripts/Kconfig.include
KCSAN
M: Marco Elver <elver@google.com>
R: Dmitry Vyukov <dvyukov@google.com>
L: kasan-dev@googlegroups.com
S: Maintained
F: Documentation/dev-tools/kcsan.rst
F: include/linux/kcsan*.h
F: kernel/kcsan/
F: lib/Kconfig.kcsan
F: scripts/Makefile.kcsan
KDUMP
M: Dave Young <dyoung@redhat.com>
M: Baoquan He <bhe@redhat.com>
+2 -1
View File
@@ -478,7 +478,7 @@ export KBUILD_HOSTCXXFLAGS KBUILD_HOSTLDFLAGS KBUILD_HOSTLDLIBS LDFLAGS_MODULE
export KBUILD_CPPFLAGS NOSTDINC_FLAGS LINUXINCLUDE OBJCOPYFLAGS KBUILD_LDFLAGS
export KBUILD_CFLAGS CFLAGS_KERNEL CFLAGS_MODULE
export CFLAGS_KASAN CFLAGS_KASAN_NOSANITIZE CFLAGS_UBSAN
export CFLAGS_KASAN CFLAGS_KASAN_NOSANITIZE CFLAGS_UBSAN CFLAGS_KCSAN
export KBUILD_AFLAGS AFLAGS_KERNEL AFLAGS_MODULE
export KBUILD_AFLAGS_MODULE KBUILD_CFLAGS_MODULE KBUILD_LDFLAGS_MODULE
export KBUILD_AFLAGS_KERNEL KBUILD_CFLAGS_KERNEL
@@ -900,6 +900,7 @@ endif
include scripts/Makefile.kasan
include scripts/Makefile.extrawarn
include scripts/Makefile.ubsan
include scripts/Makefile.kcsan
# Add user supplied CPPFLAGS, AFLAGS and CFLAGS as the last assignments
KBUILD_CPPFLAGS += $(KCPPFLAGS)
+1
View File
@@ -226,6 +226,7 @@ config X86
select VIRT_TO_BUS
select X86_FEATURE_NAMES if PROC_FS
select PROC_PID_ARCH_STATUS if PROC_FS
select HAVE_ARCH_KCSAN if X86_64
config INSTRUCTION_DECODER
def_bool y
+2
View File
@@ -9,7 +9,9 @@
# Changed by many, many contributors over the years.
#
# Sanitizer runtimes are unavailable and cannot be linked for early boot code.
KASAN_SANITIZE := n
KCSAN_SANITIZE := n
OBJECT_FILES_NON_STANDARD := y
# Kernel does not boot with kcov instrumentation here.
+2
View File
@@ -17,7 +17,9 @@
# (see scripts/Makefile.lib size_append)
# compressed vmlinux.bin.all + u32 size of vmlinux.bin.all
# Sanitizer runtimes are unavailable and cannot be linked for early boot code.
KASAN_SANITIZE := n
KCSAN_SANITIZE := n
OBJECT_FILES_NON_STANDARD := y
# Prevents link failures: __sanitizer_cov_trace_pc() is not linked in.
+3
View File
@@ -10,8 +10,11 @@ ARCH_REL_TYPE_ABS += R_386_GLOB_DAT|R_386_JMP_SLOT|R_386_RELATIVE
include $(srctree)/lib/vdso/Makefile
KBUILD_CFLAGS += $(DISABLE_LTO)
# Sanitizer runtimes are unavailable and cannot be linked here.
KASAN_SANITIZE := n
UBSAN_SANITIZE := n
KCSAN_SANITIZE := n
OBJECT_FILES_NON_STANDARD := y
# Prevents link failures: __sanitizer_cov_trace_pc() is not linked in.
+5 -1
View File
@@ -201,8 +201,12 @@ arch_test_and_change_bit(long nr, volatile unsigned long *addr)
return GEN_BINARY_RMWcc(LOCK_PREFIX __ASM_SIZE(btc), *addr, c, "Ir", nr);
}
static __always_inline bool constant_test_bit(long nr, const volatile unsigned long *addr)
static __no_kcsan_or_inline bool constant_test_bit(long nr, const volatile unsigned long *addr)
{
/*
* Because this is a plain access, we need to disable KCSAN here to
* avoid double instrumentation via instrumented bitops.
*/
return ((1UL << (nr & (BITS_PER_LONG-1))) &
(addr[nr >> _BITOPS_LONG_SHIFT])) != 0;
}
+4
View File
@@ -28,6 +28,10 @@ KASAN_SANITIZE_dumpstack_$(BITS).o := n
KASAN_SANITIZE_stacktrace.o := n
KASAN_SANITIZE_paravirt.o := n
# With some compiler versions the generated code results in boot hangs, caused
# by several compilation units. To be safe, disable all instrumentation.
KCSAN_SANITIZE := n
OBJECT_FILES_NON_STANDARD_relocate_kernel_$(BITS).o := y
OBJECT_FILES_NON_STANDARD_test_nx.o := y
OBJECT_FILES_NON_STANDARD_paravirt_patch.o := y
+3
View File
@@ -13,6 +13,9 @@ endif
KCOV_INSTRUMENT_common.o := n
KCOV_INSTRUMENT_perf_event.o := n
# As above, instrumenting secondary CPU boot code causes boot hangs.
KCSAN_SANITIZE_common.o := n
# Make sure load_percpu_segment has no stackprotector
nostackp := $(call cc-option, -fno-stack-protector)
CFLAGS_common.o := $(nostackp)
+4
View File
@@ -6,10 +6,14 @@
# Produces uninteresting flaky coverage.
KCOV_INSTRUMENT_delay.o := n
# KCSAN uses udelay for introducing watchpoint delay; avoid recursion.
KCSAN_SANITIZE_delay.o := n
# Early boot use of cmdline; don't instrument it
ifdef CONFIG_AMD_MEM_ENCRYPT
KCOV_INSTRUMENT_cmdline.o := n
KASAN_SANITIZE_cmdline.o := n
KCSAN_SANITIZE_cmdline.o := n
ifdef CONFIG_FUNCTION_TRACER
CFLAGS_REMOVE_cmdline.o = -pg
+4
View File
@@ -7,6 +7,10 @@ KCOV_INSTRUMENT_mem_encrypt_identity.o := n
KASAN_SANITIZE_mem_encrypt.o := n
KASAN_SANITIZE_mem_encrypt_identity.o := n
# Disable KCSAN entirely, because otherwise we get warnings that some functions
# reference __initdata sections.
KCSAN_SANITIZE := n
ifdef CONFIG_FUNCTION_TRACER
CFLAGS_REMOVE_mem_encrypt.o = -pg
CFLAGS_REMOVE_mem_encrypt_identity.o = -pg
+2
View File
@@ -17,7 +17,9 @@ CFLAGS_sha256.o := -D__DISABLE_EXPORTS
LDFLAGS_purgatory.ro := -e purgatory_start -r --no-undefined -nostdlib -z nodefaultlib
targets += purgatory.ro
# Sanitizer runtimes are unavailable and cannot be linked here.
KASAN_SANITIZE := n
KCSAN_SANITIZE := n
KCOV_INSTRUMENT := n
# These are adjustments to the compiler flags used for objects that
+3
View File
@@ -6,7 +6,10 @@
# for more details.
#
#
# Sanitizer runtimes are unavailable and cannot be linked here.
KASAN_SANITIZE := n
KCSAN_SANITIZE := n
OBJECT_FILES_NON_STANDARD := y
subdir- := rm
+3
View File
@@ -6,7 +6,10 @@
# for more details.
#
#
# Sanitizer runtimes are unavailable and cannot be linked here.
KASAN_SANITIZE := n
KCSAN_SANITIZE := n
OBJECT_FILES_NON_STANDARD := y
# Prevents link failures: __sanitizer_cov_trace_pc() is not linked in.
+2
View File
@@ -31,7 +31,9 @@ KBUILD_CFLAGS := $(cflags-y) -DDISABLE_BRANCH_PROFILING \
-D__DISABLE_EXPORTS
GCOV_PROFILE := n
# Sanitizer runtimes are unavailable and cannot be linked here.
KASAN_SANITIZE := n
KCSAN_SANITIZE := n
UBSAN_SANITIZE := n
OBJECT_FILES_NON_STANDARD := y
File diff suppressed because it is too large Load Diff
+9
View File
@@ -24,6 +24,15 @@
#define __no_sanitize_address
#endif
#if __has_feature(thread_sanitizer)
/* emulate gcc's __SANITIZE_THREAD__ flag */
#define __SANITIZE_THREAD__
#define __no_sanitize_thread \
__attribute__((no_sanitize("thread")))
#else
#define __no_sanitize_thread
#endif
/*
* Not all versions of clang implement the the type-generic versions
* of the builtin overflow checkers. Fortunately, clang implements
+7
View File
@@ -145,6 +145,13 @@
#define __no_sanitize_address
#endif
#if defined(__SANITIZE_THREAD__) && __has_attribute(__no_sanitize_thread__)
#define __no_sanitize_thread \
__attribute__((__noinline__)) __attribute__((no_sanitize_thread))
#else
#define __no_sanitize_thread
#endif
#if GCC_VERSION >= 50100
#define COMPILER_HAS_GENERIC_BUILTIN_OVERFLOW 1
#endif

Some files were not shown because too many files have changed in this diff Show More