Coccinelle supports reading .cocciconfig, the order of precedence for
variables for .cocciconfig is as follows:
o Your current user's home directory is processed first
o Your directory from which spatch is called is processed next
o The directory provided with the --dir option is processed last, if used
Since coccicheck runs through make, it naturally runs from the kernel
proper dir, as such the second rule above would be implied for picking up a
.cocciconfig when using 'make coccicheck'.
'make coccicheck' also supports using M= targets.If you do not supply
any M= target, it is assumed you want to target the entire kernel.
The kernel coccicheck script has:
if [ "$KBUILD_EXTMOD" = "" ] ; then
OPTIONS="--dir $srctree $COCCIINCLUDE"
else
OPTIONS="--dir $KBUILD_EXTMOD $COCCIINCLUDE"
fi
KBUILD_EXTMOD is set when an explicit target with M= is used. For both cases
the spatch --dir argument is used, as such third rule applies when
whether M= is used or not, and when M= is used the target directory can
have its own .cocciconfig file. When M= is not passed as an argument to
coccicheck the target directory is the same as the directory from where
spatch was called.
If not using the kernel's coccicheck target, keep the above precedence order
logic of .cocciconfig reading. If using the kernel's coccicheck target,
override any of the kernel's .coccicheck's settings using SPFLAGS.
We help Coccinelle when used against Linux with a set of sensible defaults
options for Linux with our own Linux .cocciconfig. This hints to coccinelle
git can be used for 'git grep' queries over coccigrep. A timeout of 200
seconds should suffice for now.
The options picked up by coccinelle when reading a .cocciconfig do not appear
as arguments to spatch processes running on your system, to confirm what
options will be used by Coccinelle run:
spatch --print-options-only
You can override with your own preferred index option by using SPFLAGS.
Coccinelle supports both glimpse and idutils. Glimpse had historically
provided the best performance, however recent benchmarks reveal idutils
is performing just as well. Due to some recent fixes however you however
will need at least coccinelle >= 1.0.6 if using idutils.
Coccinelle carries a script scripts/idutils_index.sh which creates the
idutils database with as follows:
mkid -i C --output .id-utils.index
If using just "--use-idutils" coccinelle expects your idutils database to be
on the top level of the kernel as a file named ".id-utils.index". If you do
not use this you can symlink your database file to it, or you can specify the
database file following the "--use-idutils" argument. Examples:
make SPFLAGS=--use-idutils coccicheck
This assumes you have $srctree/.id-utils.index, where $srctree is
the top level of the kernel.
make SPFLAGS="--use-idutils /full-path/to/ID" coccicheck
Here you specify the full path of the idutils ID database. Using
.cocciconfig is possible, however given the order of precedence followed
by Coccinelle, and since the kernel now carries its own .cocciconfig,
you will need to use SPFLAGS to use idutils if desired.
v4:
o Recommend upgrade for using idutils with coccinelle due to some
recent fixes.
o Refer to using --print-options-only for testing what options are
picked up by .cocciconfig reading.
o Expand commit log considerably explaining *why* .cocconfig from
two precedence rules are used when using coccicheck, and how to
properly override these if needed.
o Expand Documentation/coccinelle.txt
v3: Expand commit log a bit more
Signed-off-by: Luis R. Rodriguez <mcgrof@kernel.org>
Acked-by: Julia Lawall <julia.lawall@lip6.fr>
Signed-off-by: Michal Marek <mmarek@suse.com>
When debugging (using --profile or --show-trying) you want to
avoid supressing output, use --quiet instead. While at it, extend
documentation for SPFLAGS use.
For instance one can use:
$ export COCCI=scripts/coccinelle/misc/irqf_oneshot.cocci
$ make coccicheck DEBUG_FILE="poo.err" MODE=report SPFLAGS="--profile --show-trying" M=./drivers/mfd/arizona-irq.c
Expand Documentation/coccinelle.txt as well.
v4: expand Documentation/coccinelle.txt
v3: rebased, resolve conflicts, expand Documentation/coccinelle.txt
v2: use egrep instead of the *"=--option"* check, this doesn't work for
disjunctions.
Signed-off-by: Luis R. Rodriguez <mcgrof@kernel.org>
Acked-by: Julia Lawall <julia.lawall@lip6.fr>
Signed-off-by: Michal Marek <mmarek@suse.com>
Enable to capture stderr via a DEBUG_FILE variable passed to
coccicheck. You can now do:
$ rm -f cocci.err
$ export COCCI=scripts/coccinelle/free/kfree.cocci
$ make coccicheck MODE=report DEBUG_FILE=cocci.err
...
$ cat cocci.err
This will be come more useful once we add support to
use more things which would go into stderr, such as
profiling. That will be done separately in another
commit.
Expand Documentation/coccinelle.txt with details.
Signed-off-by: Luis R. Rodriguez <mcgrof@kernel.org>
Acked-by: Nicolas Palix <nicolas.palix@imag.fr>
Signed-off-by: Michal Marek <mmarek@suse.com>
Coccinelle has had parmap support since 1.0.2, this means
it supports --jobs, enabling built-in multithreaded functionality,
instead of needing one to script it out. Just look for --jobs
in the help output to determine if this is supported and use it
only if your number of processors detected is > 1.
If parmap is enabled also enable the load balancing to be dynamic, so
that if a thread finishes early we keep feeding it.
stderr is currently sent to /dev/null, addressing a way to capture
that will be addressed next.
If --jobs is not supported we fallback to the old mechanism.
We expect to deprecate the old mechanism as soon as we can get
confirmation all users are ready.
While at it propagate back into the shell script any coccinelle error
code. When used in serialized mode where all cocci files are run this
also stops processing if an error has occured. This lets us handle some
errors in coccinelle cocci files and if they bail out we should inspect
the errors. This will be more useful later to help annotate coccinelle
version dependency requirements. This will let you run only SmPL files
that your system supports.
Extend Documentation/coccinelle.txt as well.
As a small example, prior to this change, on an 8-core system:
Before:
$ export COCCI=scripts/coccinelle/free/kfree.cocci
$ time make coccicheck MODE=report
...
real 29m14.912s
user 103m1.796s
sys 0m4.464s
After:
real 16m22.435s
user 128m30.060s
sys 0m2.712s
v4:
o expand Documentation/coccinelle.txt to reflect parmap support info
o update commit log to reflect what we actually do now with stderr
o split out DEBUG_FILE use into another patch
o detect number of CPUs and if its 1 then skip parmap support,
note that if you still support parmap, but have 1 CPU you will
also go through the new branches, so the old complex multithreaded process
is skipped as well.
v3:
o move USE_JOBS to avoid being overriden
v2:
o redirect coccinelle stderr to /dev/null by default and
only if DEBUG_FILE is used do we pass it to a file
o fix typo of paramap/parmap
Signed-off-by: Luis R. Rodriguez <mcgrof@kernel.org>
Acked-by: Nicolas Palix <nicolas.palix@imag.fr>
Signed-off-by: Michal Marek <mmarek@suse.com>
SPFLAGS is set early, it means that any heuristics done on
coccicheck cannot be overridden currently. Move SPFLAGS
after OPTIONS and set this at the end. This lets you override
any heuristics as coccinelle treats conflicts by only listening
to the last option that makes sense.
v3: this patch was added in the v3 series
v4: Update Documentation/coccinelle.txt explaining how
SPFLAGS works as well.
Signed-off-by: Luis R. Rodriguez <mcgrof@kernel.org>
Acked-by: Nicolas Palix <nicolas.palix@imag.fr>
Signed-off-by: Michal Marek <mmarek@suse.com>
This has no functional changes. This is being done
to enable us to later use spatch binary for some
flag checking for certain features early on.
Signed-off-by: Luis R. Rodriguez <mcgrof@kernel.org>
Acked-by: Nicolas Palix <nicolas.palix@imag.fr>
Signed-off-by: Michal Marek <mmarek@suse.com>
On May 4th, Bjørn Mork provided patch 697bbc7b83 ("builddeb: include
objtool binary in headers package"). However, that one only works if
$srctree=$objtree, because the objtool binaries are not written to the
srctree, but to the objtree.
Signed-off-by: Wilfried Klaebe <linux-kernel@lebenslange-mailadresse.de>
Fixes: 697bbc7b83 ("builddeb: include objtool binary in headers package")
Signed-off-by: Michal Marek <mmarek@suse.com>
krealloc() must not be used against devm_*() allocated
memory regions:
- if a bigger memory is to be allocated, krealloc() and
__krealloc() could return a different pointer than the
one given to them, creating a memory region which is not
managed, thus it will not be automatically released on
device removal.
- if a bigger memory is to be allocated, krealloc() could
kfree() the managed memory region which is passed to it.
The old pointer is left registered as a resource for the
device. On device removal, this dangling pointer will be
used and an unrelated memory region could be released.
- if the requested size is equal to 0, krealloc() can also
just behave like kfree(). Here too, the old pointer is
kept associated with the device. On device removal, this
invalid pointer will be used and an unrelated memory
region could be released.
For all these reasons, krealloc() must not be used on a
pointer returned by devm_*() functions.
Cc: Tejun Heo <tj@kernel.org>
Cc: Pekka Enberg <penberg@cs.helsinki.fi>
Acked-by: Julia Lawall <julia.lawall@lip6.fr>
Signed-off-by: Yann Droneaud <ydroneaud@opteya.com>
Signed-off-by: Michal Marek <mmarek@suse.com>
Since commit 3ef0e5ba46 ('slab: introduce kzfree()'),
kfree() is no more the only function to be considered:
kzfree() should be recognized too.
In particular, kzfree() must not be called on memory
allocated through devm_*() functions.
Cc: Johannes Weiner <hannes@cmpxchg.org>
Acked-by: Julia Lawall <julia.lawall@lip6.fr>
Signed-off-by: Yann Droneaud <ydroneaud@opteya.com>
Signed-off-by: Michal Marek <mmarek@suse.com>
Documentation/coccinelle.txt suggests using the SPFLAGS
make variable to pass additional options to spatch.
Reorder the way SPFLAGS is added to FLAGS, to allow
for options in the SPFLAGS to override the default
--very-quiet option.
Similarly, rearrage the FLAGS for org or report mode.
This allows for overriding of the default --no-show-diff
option through SPFLAGS.
Signed-off-by: Deepa Dinamani <deepa.kernel@gmail.com>
Cc: Gilles Muller <Gilles.Muller@lip6.fr>
Acked-by: Nicolas Palix <nicolas.palix@imag.fr>
Acked-by: Julia Lawall <Julia.Lawall@lip6.fr>
Signed-off-by: Michal Marek <mmarek@suse.com>
Add new rules to detect the cases where sizeof is used in
function calls as a argument.
Also, for the patch mode third rule should behave same as
second rule with arguments reversed. So, change that as well.
Signed-off-by: Vaishali Thakkar <vaishali.thakkar@oracle.com>
Acked-by: Julia Lawall <julia.lawall@lip6.fr>
Signed-off-by: Michal Marek <mmarek@suse.com>
The original name was simply hash_string(), but that conflicted with a
function with that name in drivers/base/power/trace.c, and I decided
that calling it "hashlen_" was better anyway.
But you have to do it in two places.
[ This caused build errors for architectures that don't define
CONFIG_DCACHE_WORD_ACCESS - Linus ]
Signed-off-by: George Spelvin <linux@sciencehorizons.net>
Reported-by: Guenter Roeck <linux@roeck-us.net>
Fixes: fcfd2fbf22 ("fs/namei.c: Add hashlen_string() function")
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
The HPFS filesystem used generic_show_options to produce string that is
displayed in /proc/mounts. However, there is a problem that the options
may disappear after remount. If we mount the filesystem with option1
and then remount it with option2, /proc/mounts should show both option1
and option2, however it only shows option2 because the whole option
string is replaced with replace_mount_options in hpfs_remount_fs.
To fix this bug, implement the hpfs_show_options function that prints
options that are currently selected.
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Cc: stable@vger.kernel.org
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Commit c8f33d0bec ("affs: kstrdup() memory handling") checks if the
kstrdup function returns NULL due to out-of-memory condition.
However, if we are remounting a filesystem with no change to
filesystem-specific options, the parameter data is NULL. In this case,
kstrdup returns NULL (because it was passed NULL parameter), although no
out of memory condition exists. The mount syscall then fails with
ENOMEM.
This patch fixes the bug. We fail with ENOMEM only if data is non-NULL.
The patch also changes the call to replace_mount_options - if we didn't
pass any filesystem-specific options, we don't call
replace_mount_options (thus we don't erase existing reported options).
Fixes: c8f33d0bec ("affs: kstrdup() memory handling")
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Cc: stable@vger.kernel.org # v4.1+
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Commit ce657611ba ("hpfs: kstrdup() out of memory handling") checks if
the kstrdup function returns NULL due to out-of-memory condition.
However, if we are remounting a filesystem with no change to
filesystem-specific options, the parameter data is NULL. In this case,
kstrdup returns NULL (because it was passed NULL parameter), although no
out of memory condition exists. The mount syscall then fails with
ENOMEM.
This patch fixes the bug. We fail with ENOMEM only if data is non-NULL.
The patch also changes the call to replace_mount_options - if we didn't
pass any filesystem-specific options, we don't call
replace_mount_options (thus we don't erase existing reported options).
Fixes: ce657611ba ("hpfs: kstrdup() out of memory handling")
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Cc: stable@vger.kernel.org
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Various builds (such as i386:allmodconfig) fail with
fs/binfmt_aout.c:133:2: error: expected identifier or '(' before 'return'
fs/binfmt_aout.c:134:1: error: expected identifier or '(' before '}' token
[ Oops. My bad, I had stupidly thought that "allmodconfig" covered this
on x86-64 too, but it obviously doesn't. Egg on my face. - Linus ]
Fixes: 5d22fc25d4 ("mm: remove more IS_ERR_VALUE abuses")
Signed-off-by: Guenter Roeck <linux@roeck-us.net>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Pull string hash improvements from George Spelvin:
"This series does several related things:
- Makes the dcache hash (fs/namei.c) useful for general kernel use.
(Thanks to Bruce for noticing the zero-length corner case)
- Converts the string hashes in <linux/sunrpc/svcauth.h> to use the
above.
- Avoids 64-bit multiplies in hash_64() on 32-bit platforms. Two
32-bit multiplies will do well enough.
- Rids the world of the bad hash multipliers in hash_32.
This finishes the job started in commit 689de1d6ca ("Minimal
fix-up of bad hashing behavior of hash_64()")
The vast majority of Linux architectures have hardware support for
32x32-bit multiply and so derive no benefit from "simplified"
multipliers.
The few processors that do not (68000, h8/300 and some models of
Microblaze) have arch-specific implementations added. Those
patches are last in the series.
- Overhauls the dcache hash mixing.
The patch in commit 0fed3ac866 ("namei: Improve hash mixing if
CONFIG_DCACHE_WORD_ACCESS") was an off-the-cuff suggestion.
Replaced with a much more careful design that's simultaneously
faster and better. (My own invention, as there was noting suitable
in the literature I could find. Comments welcome!)
- Modify the hash_name() loop to skip the initial HASH_MIX(). This
would let us salt the hash if we ever wanted to.
- Sort out partial_name_hash().
The hash function is declared as using a long state, even though
it's truncated to 32 bits at the end and the extra internal state
contributes nothing to the result. And some callers do odd things:
- fs/hfs/string.c only allocates 32 bits of state
- fs/hfsplus/unicode.c uses it to hash 16-bit unicode symbols not bytes
- Modify bytemask_from_count to handle inputs of 1..sizeof(long)
rather than 0..sizeof(long)-1. This would simplify users other
than full_name_hash"
Special thanks to Bruce Fields for testing and finding bugs in v1. (I
learned some humbling lessons about "obviously correct" code.)
On the arch-specific front, the m68k assembly has been tested in a
standalone test harness, I've been in contact with the Microblaze
maintainers who mostly don't care, as the hardware multiplier is never
omitted in real-world applications, and I haven't heard anything from
the H8/300 world"
* 'hash' of git://ftp.sciencehorizons.net/linux:
h8300: Add <asm/hash.h>
microblaze: Add <asm/hash.h>
m68k: Add <asm/hash.h>
<linux/hash.h>: Add support for architecture-specific functions
fs/namei.c: Improve dcache hash function
Eliminate bad hash multipliers from hash_32() and hash_64()
Change hash_64() return value to 32 bits
<linux/sunrpc/svcauth.h>: Define hash_str() in terms of hashlen_string()
fs/namei.c: Add hashlen_string() function
Pull out string hash to <linux/stringhash.h>
This will improve the performance of hash_32() and hash_64(), but due
to complete lack of multi-bit shift instructions on H8, performance will
still be bad in surrounding code.
Designing H8-specific hash algorithms to work around that is a separate
project. (But if the maintainers would like to get in touch...)
Signed-off-by: George Spelvin <linux@sciencehorizons.net>
Cc: Yoshinori Sato <ysato@users.sourceforge.jp>
Cc: uclinux-h8-devel@lists.sourceforge.jp
Microblaze is an FPGA soft core that can be configured various ways.
If it is configured without a multiplier, the standard __hash_32()
will require a call to __mulsi3, which is a slow software loop.
Instead, use a shift-and-add sequence for the constant multiply.
GCC knows how to do this, but it's not as clever as some.
Signed-off-by: George Spelvin <linux@sciencehorizons.net>
Cc: Alistair Francis <alistair.francis@xilinx.com>
Cc: Michal Simek <michal.simek@xilinx.com>
This is just the infrastructure; there are no users yet.
This is modelled on CONFIG_ARCH_RANDOM; a CONFIG_ symbol declares
the existence of <asm/hash.h>.
That file may define its own versions of various functions, and define
HAVE_* symbols (no CONFIG_ prefix!) to suppress the generic ones.
Included is a self-test (in lib/test_hash.c) that verifies the basics.
It is NOT in general required that the arch-specific functions compute
the same thing as the generic, but if a HAVE_* symbol is defined with
the value 1, then equality is tested.
Signed-off-by: George Spelvin <linux@sciencehorizons.net>
Cc: Geert Uytterhoeven <geert@linux-m68k.org>
Cc: Greg Ungerer <gerg@linux-m68k.org>
Cc: Andreas Schwab <schwab@linux-m68k.org>
Cc: Philippe De Muyter <phdm@macq.eu>
Cc: linux-m68k@lists.linux-m68k.org
Cc: Alistair Francis <alistai@xilinx.com>
Cc: Michal Simek <michal.simek@xilinx.com>
Cc: Yoshinori Sato <ysato@users.sourceforge.jp>
Cc: uclinux-h8-devel@lists.sourceforge.jp
Luis R. Rodriguez
Wilfried Klaebe
Yann Droneaud
Deepa Dinamani
Vaishali Thakkar
Linus Torvalds
George Spelvin
Mikulas Patocka
Guenter Roeck