Make coccicheck checked for the existence of DEBUG_FILE on each semantic
patch, and bailed if it already existed. This meant that DEBUG_FILE was
useless for checking more than one semantic patch at a time. Now the check
is moved to the start of make coccicheck, and the 2> is changed to a 2>> to
append to the file on each semantic patch. Furthermore, the spatch command
that is run for each semantic patch is also added to the DEBUG_FILE, to
make clear what each stdout trace corresponds to.
Signed-off-by: Julia Lawall <Julia.Lawall@lip6.fr>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
This semantic patch detects duplicate arrays declared using BQ27XXX_DATA
within a single structure. It is currently specific to the file
drivers/power/supply/bq27xxx_battery.c. Nevertheless, having the script in
the kernel will allow others to check their code if the data structures
change in the future.
Signed-off-by: Julia Lawall <Julia.Lawall@lip6.fr>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
The spec file always passes %{?_smp_mflags}, but we have two
problems here.
[1] "make -jN rpm-pkg" emits the following warning message:
make[2]: warning: -jN forced in submake: disabling jobserver mode.
[2] We can not specify the number of jobs that run in parallel.
Whether we give -jN or not from the top Makefile, the spec file
always passes ${?_smp_mflags} to the build commands.
${?_smp_mflags} will be useful when we run rpmbuild by hand. When we
invoke it from Makefile, -jN is propagated down to submake; it should
not be overridden because we want to respect the number of jobs given
by the user. Set _smp_mflags to empty string in this case.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
If build fails during (bin)rpm-pkg, the spec file is not cleaned by
anyone until the next successful build of the package.
We do not have to immediately delete the spec file in case somebody
may want to take a look at it. Instead, make them ignored by git,
and cleaned up by make mrproper.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
If "make rpm-pkg" or "make binrpm-pkg" is run with -j[jobs] option,
the following warning message is displayed.
warning: jobserver unavailable: using -j1. Add '+' to parent make rule.
Follow the suggestion.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
$RPM_BUILD_ROOT must be escaped to prevent shell from expanding it
when generating the spec file.
%{build_root} is more readable than \$RPM_BUILD_ROOT.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
When CONFIG_MODULES is disabled, make rpm-pkg / binrpm-pkg fails
with the following message:
The present kernel configuration has modules disabled.
Type 'make config' and enable loadable module support.
Then build a kernel with module support enabled.
Do not install modules in the case. Also, omit the devel package.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
The repeat of echo is unreadable. The here-document is a well-known
device for such scripts. One difficulty is we have a bunch of PREBUILT
conditionals that would split the here-document.
My idea is to add "$S" annotatation to lines only for the source package
spec file, then post-process it by sed. I hope it will make our life
easier than repeat of "cat <<EOF ..."
I confirmed this commit still produced the same (bin)kernel.spec as
before.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Signal masks are false positives, we already check for SigBlk and SigCgt
but we missed SigIgn.
Add SigIgn to false positive check.
Signed-off-by: Tobin C. Harding <me@tobin.cc>
Currently script can stall if we read certain files (like
/proc/kmsg). While we have a mechanism to skip these files once they are
discovered it would be nice to not stall on as yet undiscovered files of
this kind.
Set a timer before each file is parsed, warn user if timer expires.
Suggested-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Tobin C. Harding <me@tobin.cc>
Currently script is targeted at x86_64. We can support other
architectures by using the correct regular expressions for each
architecture.
Add the infrastructure to support multiple architectures. Add support
for ppc64.
Signed-off-by: Tobin C. Harding <me@tobin.cc>
Currently script just dumps all results found. Potentially, this risks
losing single results among multiple duplicate results. We need some
way of restricting duplicates to assist users of the script. It would
also be nice if we got a report instead of raw results.
Duplicates can be defined in various ways, instead of trying to find a
single perfect solution we can present the user with various options to
display the output. Doing so will typically lead to users wanting to
view the output multiple times. Currently we scan the kernel each time,
this is slow and unnecessary. We can expedite the process by writing the
results to file for subsequent viewing.
Add command line options to enable summary reporting, including options
to write to and read from file.
Signed-off-by: Tobin C. Harding <me@tobin.cc>
There are a couple more files that cause the script to stall.
/sys/firmware/devicetree and its symlink /proc/device-tree, reported by
Michael Ellerman.
usbmon should be skipped were ever it appears. Reported by Kees Cook
Add files to be excluded from parsing.
Signed-off-by: Tobin C. Harding <me@tobin.cc>
Currently script accepts files to skip. This was added to make running
the script faster (for repeat runs). We can remove this functionality in
preparation for adding sub commands (scan and format) to the script.
Remove command line options.
Signed-off-by: Tobin C. Harding <me@tobin.cc>
debug_arrays is not called. Also, %seen hash is not used. We should
remove unused code.
Remove dead code.
Signed-off-by: Tobin C. Harding <me@tobin.cc>
Pull x86 core updates from Ingo Molnar:
"Note that in this cycle most of the x86 topics interacted at a level
that caused them to be merged into tip:x86/asm - but this should be a
temporary phenomenon, hopefully we'll back to the usual patterns in
the next merge window.
The main changes in this cycle were:
Hardware enablement:
- Add support for the Intel UMIP (User Mode Instruction Prevention)
CPU feature. This is a security feature that disables certain
instructions such as SGDT, SLDT, SIDT, SMSW and STR. (Ricardo Neri)
[ Note that this is disabled by default for now, there are some
smaller enhancements in the pipeline that I'll follow up with in
the next 1-2 days, which allows this to be enabled by default.]
- Add support for the AMD SEV (Secure Encrypted Virtualization) CPU
feature, on top of SME (Secure Memory Encryption) support that was
added in v4.14. (Tom Lendacky, Brijesh Singh)
- Enable new SSE/AVX/AVX512 CPU features: AVX512_VBMI2, GFNI, VAES,
VPCLMULQDQ, AVX512_VNNI, AVX512_BITALG. (Gayatri Kammela)
Other changes:
- A big series of entry code simplifications and enhancements (Andy
Lutomirski)
- Make the ORC unwinder default on x86 and various objtool
enhancements. (Josh Poimboeuf)
- 5-level paging enhancements (Kirill A. Shutemov)
- Micro-optimize the entry code a bit (Borislav Petkov)
- Improve the handling of interdependent CPU features in the early
FPU init code (Andi Kleen)
- Build system enhancements (Changbin Du, Masahiro Yamada)
- ... plus misc enhancements, fixes and cleanups"
* 'x86-asm-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (118 commits)
x86/build: Make the boot image generation less verbose
selftests/x86: Add tests for the STR and SLDT instructions
selftests/x86: Add tests for User-Mode Instruction Prevention
x86/traps: Fix up general protection faults caused by UMIP
x86/umip: Enable User-Mode Instruction Prevention at runtime
x86/umip: Force a page fault when unable to copy emulated result to user
x86/umip: Add emulation code for UMIP instructions
x86/cpufeature: Add User-Mode Instruction Prevention definitions
x86/insn-eval: Add support to resolve 16-bit address encodings
x86/insn-eval: Handle 32-bit address encodings in virtual-8086 mode
x86/insn-eval: Add wrapper function for 32 and 64-bit addresses
x86/insn-eval: Add support to resolve 32-bit address encodings
x86/insn-eval: Compute linear address in several utility functions
resource: Fix resource_size.cocci warnings
X86/KVM: Clear encryption attribute when SEV is active
X86/KVM: Decrypt shared per-cpu variables when SEV is active
percpu: Introduce DEFINE_PER_CPU_DECRYPTED
x86: Add support for changing memory encryption attribute in early boot
x86/io: Unroll string I/O when SEV is active
x86/boot: Add early boot support when running with SEV active
...
Pull core locking updates from Ingo Molnar:
"The main changes in this cycle are:
- Another attempt at enabling cross-release lockdep dependency
tracking (automatically part of CONFIG_PROVE_LOCKING=y), this time
with better performance and fewer false positives. (Byungchul Park)
- Introduce lockdep_assert_irqs_enabled()/disabled() and convert
open-coded equivalents to lockdep variants. (Frederic Weisbecker)
- Add down_read_killable() and use it in the VFS's iterate_dir()
method. (Kirill Tkhai)
- Convert remaining uses of ACCESS_ONCE() to
READ_ONCE()/WRITE_ONCE(). Most of the conversion was Coccinelle
driven. (Mark Rutland, Paul E. McKenney)
- Get rid of lockless_dereference(), by strengthening Alpha atomics,
strengthening READ_ONCE() with smp_read_barrier_depends() and thus
being able to convert users of lockless_dereference() to
READ_ONCE(). (Will Deacon)
- Various micro-optimizations:
- better PV qspinlocks (Waiman Long),
- better x86 barriers (Michael S. Tsirkin)
- better x86 refcounts (Kees Cook)
- ... plus other fixes and enhancements. (Borislav Petkov, Juergen
Gross, Miguel Bernal Marin)"
* 'locking-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (70 commits)
locking/x86: Use LOCK ADD for smp_mb() instead of MFENCE
rcu: Use lockdep to assert IRQs are disabled/enabled
netpoll: Use lockdep to assert IRQs are disabled/enabled
timers/posix-cpu-timers: Use lockdep to assert IRQs are disabled/enabled
sched/clock, sched/cputime: Use lockdep to assert IRQs are disabled/enabled
irq_work: Use lockdep to assert IRQs are disabled/enabled
irq/timings: Use lockdep to assert IRQs are disabled/enabled
perf/core: Use lockdep to assert IRQs are disabled/enabled
x86: Use lockdep to assert IRQs are disabled/enabled
smp/core: Use lockdep to assert IRQs are disabled/enabled
timers/hrtimer: Use lockdep to assert IRQs are disabled/enabled
timers/nohz: Use lockdep to assert IRQs are disabled/enabled
workqueue: Use lockdep to assert IRQs are disabled/enabled
irq/softirqs: Use lockdep to assert IRQs are disabled/enabled
locking/lockdep: Add IRQs disabled/enabled assertion APIs: lockdep_assert_irqs_enabled()/disabled()
locking/pvqspinlock: Implement hybrid PV queued/unfair locks
locking/rwlocks: Fix comments
x86/paravirt: Set up the virt_spin_lock_key after static keys get initialized
block, locking/lockdep: Assign a lock_class per gendisk used for wait_for_completion()
workqueue: Remove now redundant lock acquisitions wrt. workqueue flushes
...
Pull documentation updates from Jonathan Corbet:
"A relatively calm cycle for the docs tree again.
- The old driver statement has been added to the kernel docs.
- We have a couple of new helper scripts. find-unused-docs.sh from
Sayli Karnic will point out kerneldoc comments that are not actually
used in the documentation. Jani Nikula's
documentation-file-ref-check finds references to non-existing files.
- A new ftrace document from Steve Rostedt.
- Vinod Koul converted the dmaengine docs to RST
Beyond that, it's mostly simple fixes.
This set reaches outside of Documentation/ a bit more than most. In
all cases, the changes are to comment docs, mostly from Randy, in
places where there didn't seem to be anybody better to take them"
* tag 'docs-4.15' of git://git.lwn.net/linux: (52 commits)
documentation: fb: update list of available compiled-in fonts
MAINTAINERS: update DMAengine documentation location
dmaengine: doc: ReSTize pxa_dma doc
dmaengine: doc: ReSTize dmatest doc
dmaengine: doc: ReSTize client API doc
dmaengine: doc: ReSTize provider doc
dmaengine: doc: Add ReST style dmaengine document
ftrace/docs: Add documentation on how to use ftrace from within the kernel
bug-hunting.rst: Fix an example and a typo in a Sphinx tag
scripts: Add a script to find unused documentation
samples: Convert timers to use timer_setup()
documentation: kernel-api: add more info on bitmap functions
Documentation: fix selftests related file refs
Documentation: fix ref to power basic-pm-debugging
Documentation: fix ref to trace stm content
Documentation: fix ref to coccinelle content
Documentation: fix ref to workqueue content
Documentation: fix ref to sphinx/kerneldoc.py
Documentation: fix locking rt-mutex doc refs
docs: dev-tools: correct Coccinelle version number
...
I was not seeing my linker flags getting added when using ld-option when
cross compiling with Clang. Upon investigation, this seems to be due to
a difference in how GCC vs Clang handle cross compilation.
GCC is configured at build time to support one backend, that is implicit
when compiling. Clang is explicit via the use of `-target <triple>` and
ships with all supported backends by default.
GNU Make feature test macros that compile then link will always fail
when cross compiling with Clang unless Clang's triple is passed along to
the compiler. For example:
$ clang -x c /dev/null -c -o temp.o
$ aarch64-linux-android/bin/ld -E temp.o
aarch64-linux-android/bin/ld:
unknown architecture of input file `temp.o' is incompatible with
aarch64 output
aarch64-linux-android/bin/ld:
warning: cannot find entry symbol _start; defaulting to
0000000000400078
$ echo $?
1
$ clang -target aarch64-linux-android- -x c /dev/null -c -o temp.o
$ aarch64-linux-android/bin/ld -E temp.o
aarch64-linux-android/bin/ld:
warning: cannot find entry symbol _start; defaulting to 00000000004002e4
$ echo $?
0
This causes conditional checks that invoke $(CC) without the target
triple, then $(LD) on the result, to always fail.
Suggested-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
Reviewed-by: Matthias Kaehlcke <mka@chromium.org>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
The cache files are only cleaned away by "make clean". If you continue
incremental builds, the cache files will grow up little by little.
It is not a big deal in general use cases because compiler flags do not
change quite often.
However, if you do build-test for various architectures, compilers, and
kernel configurations, you will end up with huge cache files soon.
When the cache file exceeds 1000 lines, shrink it down to 500 by "tail".
The Least Recently Added lines are cut. (not Least Recently Used)
I hope it will work well enough.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Douglas Anderson <dianders@chromium.org>
While timing a "no-op" build of the kernel (incrementally building the
kernel even though nothing changed) in the Chrome OS build system I
found that it was much slower than I expected.
Digging into things a bit, I found that quite a bit of the time was
spent invoking the C compiler even though we weren't actually building
anything. Currently in the Chrome OS build system the C compiler is
called through a number of wrappers (one of which is written in
python!) and can take upwards of 100 ms to invoke even if we're not
doing anything difficult, so these invocations of the compiler were
taking a lot of time. Worse the invocations couldn't seem to take
advantage of the multiple cores on my system.
Certainly it seems like we could make the compiler invocations in the
Chrome OS build system faster, but only to a point. Inherently
invoking a program as big as a C compiler is a fairly heavy
operation. Thus even if we can speed the compiler calls it made sense
to track down what was happening.
It turned out that all the compiler invocations were coming from
usages like this in the kernel's Makefile:
KBUILD_CFLAGS += $(call cc-option,-fno-delete-null-pointer-checks,)
Due to the way cc-option and similar statements work the above
contains an implicit call to the C compiler. ...and due to the fact
that we're storing the result in KBUILD_CFLAGS, a simply expanded
variable, the call will happen every time the Makefile is parsed, even
if there are no users of KBUILD_CFLAGS.
Rather than redoing this computation every time, it makes a lot of
sense to cache the result of all of the Makefile's compiler calls just
like we do when we compile a ".c" file to a ".o" file. Conceptually
this is quite a simple idea. ...and since the calls to invoke the
compiler and similar tools are centrally located in the Kbuild.include
file this doesn't even need to be super invasive.
Implementing the cache in a simple-to-use and efficient way is not
quite as simple as it first sounds, though. To get maximum speed we
really want the cache in a format that make can natively understand
and make doesn't really have an ability to load/parse files. ...but
make _can_ import other Makefiles, so the solution is to store the
cache in Makefile format. This requires coming up with a valid/unique
Makefile variable name for each value to be cached, but that's
solvable with some cleverness.
After this change, we'll automatically create a ".cache.mk" file that
will contain our cached variables. We'll load this on each invocation
of make and will avoid recomputing anything that's already in our
cache. The cache is stored in a format that it shouldn't need any
invalidation since anything that might change should affect the "key"
and any old cached value won't be used.
Signed-off-by: Douglas Anderson <dianders@chromium.org>
Tested-by: Ingo Molnar <mingo@kernel.org>
Tested-by: Guenter Roeck <linux@roeck-us.net>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
