My main motivation of this commit is to clean up scripts/Kbuild.include
and scripts/Makefile.build.
Currently, CONFIG_TRIM_UNUSED_KSYMS works with a tricky gimmick;
possibly exported symbols are detected by letting $(CPP) replace
EXPORT_SYMBOL* with a special string '=== __KSYM_*===', which is
post-processed by sed, and passed to fixdep. The extra preprocessing
is costly, and hacking cmd_and_fixdep is ugly.
I came up with a new way to find exported symbols; insert a dummy
symbol __ksym_marker_* to each potentially exported symbol. Those
dummy symbols are picked up by $(NM), post-processed by sed, then
appended to .*.cmd files. I collected the post-process part to a
new shell script scripts/gen_ksymdeps.sh for readability. The dummy
symbols are put into the .discard.* section so that the linker
script rips them off the final vmlinux or modules.
A nice side-effect is building with CONFIG_TRIM_UNUSED_KSYMS will
be much faster.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Nicolas Pitre <nico@linaro.org>
Commit 8370edea81 ("bin2c: move bin2c in scripts/basic") moved bin2c
to the scripts/basic/ directory, incorrectly stating "Kexec wants to
use bin2c and it wants to use it really early in the build process.
See arch/x86/purgatory/ code in later patches."
Commit bdab125c93 ("Revert "kexec/purgatory: Add clean-up for
purgatory directory"") and commit d6605b6bbe ("x86/build: Remove
unnecessary preparation for purgatory") removed the redundant
purgatory build magic entirely.
That means that the move of bin2c was unnecessary in the first place.
fixdep is the only host program that deserves to sit in the
scripts/basic/ directory.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Underscores in symbol names are translated into slashes for path names.
Filesystems treat consecutive slashes as if there was only one, so
let's do the same in the dependency list for easier grepping, etc.
Signed-off-by: Nicolas Pitre <nico@linaro.org>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
The idea of using fixdep was inspired by Kconfig, but autoksyms
belongs to a different group. So, I want to move those touched
files under include/config/ksym/ to include/ksym/.
The directory include/ksym/ can be removed by 'make clean' because
it is meaningless for the external module building.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Nicolas Pitre <nico@linaro.org>
kconfig.h was excluded from consideration by fixdep by
6a5be57f0f (fixdep: fix extraneous dependencies) to avoid some false
positive hits
(1) include/config/.h
(2) include/config/h.h
(3) include/config/foo.h
(1) occurred because kconfig.h contains the string CONFIG_ in a
comment. However, since dee81e9886 (fixdep: faster CONFIG_ search), we
have a check that the part after CONFIG_ is non-empty, so this does not
happen anymore (and CONFIG_ appears by itself elsewhere, so that check
is worthwhile).
(2) comes from the include guard, __LINUX_KCONFIG_H. But with the
previous patch, we no longer match that either.
That leaves (3), which amounts to one [1] false dependency (aka stat() call
done by make), which I think we can live with:
We've already had one case [2] where the lack of include/linux/kconfig.h in
the .o.cmd file caused a missing rebuild, and while I originally thought
we should just put kconfig.h in the dependency list without parsing it
for the CONFIG_ pattern, we actually do have some real CONFIG_ symbols
mentioned in it, and one can imagine some translation unit that just
does '#ifdef __BIG_ENDIAN' but doesn't through some other header
actually depend on CONFIG_CPU_BIG_ENDIAN - so changing the target
endianness could end up rebuilding the world, minus that small
TU. Quoting Linus,
... when missing dependencies cause a missed re-compile, the resulting
bugs can be _really_ subtle.
[1] well, two, we now also have CONFIG_BOOGER/booger.h - we could change
that to FOO if we care
[2] https://lkml.org/lkml/2018/2/22/838
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Rasmus Villemoes <linux@rasmusvillemoes.dk>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
The string CONFIG_ quite often appears after other alphanumerics,
meaning that that instance cannot be referencing a Kconfig
symbol. Omitting these means make has fewer files to stat() when
deciding what needs to be rebuilt - for a defconfig build, this seems to
remove about 2% of the (wildcard ...) lines from the .o.cmd files.
Signed-off-by: Rasmus Villemoes <linux@rasmusvillemoes.dk>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
uml-config.h hasn't existed in this decade (87e299e5c7 - x86, um: get
rid of uml-config.h). The few remaining UML_CONFIG instances are defined
directly in terms of their real CONFIG symbol in common-offsets.h, so
unlike when the symbols got defined via a sed script, anything that uses
UML_CONFIG_FOO now should also automatically pick up a dependency on
CONFIG_FOO via the normal fixdep mechanism (since common-offsets.h
should at least recursively be a dependency). Hence I believe we should
actually be able to ignore the HELLO_CONFIG_BOOM cases.
Cc: Al Viro <viro@zeniv.linux.org.uk>
Cc: Richard Weinberger <richard@nod.at>
Cc: user-mode-linux-devel@lists.sourceforge.net
Signed-off-by: Rasmus Villemoes <linux@rasmusvillemoes.dk>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
str_ends_with() tests if the given token ends with a particular string.
Currently, it is used to check file paths without $(srctree).
Actually, we have one more place where this helper is useful. Use it
to check if CONFIG option ends with _MODULE.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
parse_dep_file() has too much indentation, and puts the code far to
the right. This commit refactors the code and reduces the one level
of indentation.
strrcmp() computes 'slen' by itself, but the caller already knows the
length of the token, so 'slen' can be passed via function argument.
With this, we can swap the order of strrcmp() and "*p = \0;"
Also, strrcmp() is an ambiguous function name. Flip the logic and
rename it to str_ends_with().
I added a new helper is_ignored_file() - this returns 1 if the token
represents a file that should be ignored.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
I do not mind global variables where they are useful enough. In this
case, I do not see a good reason to use global variables since they
are just referenced in shallow places. It is easy to pass them via
function arguments.
I squashed print_cmdline() into main() since it is just one line code.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Each token in the depfile is copied to the temporary buffer 's' to
terminate the token with zero. We do not need to do this any more
because the parsed buffer is now writable. Insert '\0' directly in
the buffer without calling memcpy().
<limits.h> is no longer necessary. (It was needed for PATH_MAX).
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Now, do_config_files() and print_deps() are almost the same. Only
the difference is the parser function called (parse_config_file vs
parse_dep_file).
We can reduce the code duplication by factoring out the common code
into read_file() - this function allocates a buffer and loads a file
to it. It returns the pointer to the allocated buffer. (As before,
it bails out by exit(2) for any error.) The caller must free the
buffer when done.
Having empty source files is possible; fixdep should simply skip them.
I deleted the "st.st_size == 0" check, so read_file() allocates 1-byte
buffer for an empty file. strstr() will immediately return NULL, and
this is what we expect.
On the other hand, an empty dep_file should be treated as an error.
In this case, parse_dep_file() will error out with "no targets found"
and it is a correct error message.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Commit dee81e9886 ("fixdep: faster CONFIG_ search") changed how to
read files in which CONFIG options are searched. It used malloc()
and read() instead of mmap() because it needed to zero-terminate the
buffer in order to use strstr(). print_deps() was left untouched
since there was no reason to change it.
Now, I have two motivations to change it in the same way.
- do_config_file() and print_deps() do quite similar things; they
open a file, load it onto memory, and pass it to a parser function.
If we use malloc() and read() for print_deps() too, we can factor
out the common code. (I will do this in the next commit.)
- parse_dep_file() copies each token to a temporary buffer because
it needs to zero-terminate it to be passed to printf(). It is not
possible to modify the buffer directly because it is mmap'ed with
O_RDONLY. If we load the file content into a malloc'ed buffer, we
can insert '\0' after each token, and save memcpy(). (I will do
this in the commit after next.)
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
<arpa/inet.h> was included for ntohl(), but it was removed by
commit dee81e9886 ("fixdep: faster CONFIG_ search").
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
do_config_file() should exit with an error code on internal run-time
errors, and not return if it fails as then the error in do_config_file()
would go unnoticed in the current code and allow the build to continue.
The exit with error code will make the build fail in those very
exceptional cases. If this occurs, this actually indicates a deeper
problem in the execution of the kernel build process.
Now, in these error cases, we do not explicitly free memory and close
the file handlers in do_config_file(), as this is covered by exit().
This issue in the fixdep script was introduced with its initial
implementation back in 2002 by the original author Kai Germaschewski with
this commit 04bd72170653 ("kbuild: Make dependencies at compile time")
in the linux history git tree, i.e.,
git://git.kernel.org/pub/scm/linux/kernel/git/history/history.git.
This issue was identified during the review of a previous patch that
intended to address a memory leak detected by a static analysis tool.
Link: https://lkml.org/lkml/2017/12/14/736
Suggested-by: Nicholas Mc Guire <der.herr@hofr.at>
Suggested-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: Lukas Bulwahn <lukas.bulwahn@gmail.com>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Do you think kernel build is 100% dominated by gcc? You are wrong!
One small utility called "fixdep" consistently manages to sneak into
profile's first page (unless you have small monitor of course).
The choke point is this clever code:
for (; m < end; m++) {
if (*m == INT_CONF) { p = (char *) m ; goto conf; }
if (*m == INT_ONFI) { p = (char *) m-1; goto conf; }
if (*m == INT_NFIG) { p = (char *) m-2; goto conf; }
if (*m == INT_FIG_) { p = (char *) m-3; goto conf; }
4 branches per 4 characters is not fast.
Use strstr(3), so that SSE2 etc can be used.
With this patch, fixdep is so deep at the bottom, it is hard to find it.
Signed-off-by: Alexey Dobriyan <adobriyan@gmail.com>
Signed-off-by: Michal Marek <mmarek@suse.com>
bin2c is used to create a valid C file out of a binary file where two
symbols will be globally defined: <name> and <name>_size. <name> is
passed as the first parameter of the host binary.
Building using goto-cc reported that the purgatory binary code (the only
current user of this utility) declares kexec_purgatory_size as 'size_t'
where bin2c generate <name>_size to be 'int' so in a 64-bit host where
sizeof(size_t) > sizeof(int) this type mismatch will always yield the
wrong value for big-endian architectures while for little-endian it will
be wrong if the object laid in memory directly after
kexec_purgatory_size contains non-zero value at the time of reading.
This commit changes <name>_size to be size_t instead.
Note:
Another way to fix the problem is to change the type of
kexec_purgatory_size to be 'int' as there's this check in code:
(kexec_purgatory_size <= 0)
Signed-off-by: Michael Tautschnig <tautschn@amazon.com>
Cc: Vivek Goyal <vgoyal@redhat.com>
Acked-by: Dave Young <dyoung@redhat.com>
Signed-off-by: Michal Marek <mmarek@suse.com>
Like with kconfig options, we now have the ability to compile in and
out individual EXPORT_SYMBOL() declarations based on the content of
include/generated/autoksyms.h. However we don't want the entire
world to be rebuilt whenever that file is touched.
Let's apply the same build dependency trick used for CONFIG_* symbols
where the time stamp of empty files whose paths matching those symbols
is used to trigger fine grained rebuilds. In our case the key is the
symbol name passed to EXPORT_SYMBOL().
However, unlike config options, we cannot just use fixdep to parse
the source code for EXPORT_SYMBOL(ksym) because several variants exist
and parsing them all in a separate tool, and keeping it in synch, is
not trivially maintainable. Furthermore, there are variants such as
EXPORT_SYMBOL_GPL(pci_user_read_config_##size);
that are instanciated via a macro for which we can't easily determine
the actual exported symbol name(s) short of actually running the
preprocessor on them.
Storing the symbol name string in a special ELF section doesn't work
for targets that output assembly or preprocessed source.
So the best way is really to leverage the preprocessor by having it
output actual symbol names anchored by a special sequence that can be
easily filtered out. Then the list of symbols is simply fed to fixdep
to be merged with the other dependencies.
That implies the preprocessor is executed twice for each source file.
A previous attempt relied on a warning pragma for each EXPORT_SYMBOL()
instance that was filtered apart from stderr by the build system with
a sed script during the actual compilation pass. Unfortunately the
preprocessor/compiler diagnostic output isn't stable between versions
and this solution, although more efficient, was deemed too fragile.
Because of the lowercasing performed by fixdep, there might be name
collisions triggering spurious rebuilds for similar symbols. But this
shouldn't be a big issue in practice. (This is the case for CONFIG_*
symbols and I didn't want to be different here, whatever the original
reason for doing so.)
To avoid needless build overhead, the exported symbol name gathering is
performed only when CONFIG_TRIM_UNUSED_KSYMS is selected.
Signed-off-by: Nicolas Pitre <nico@linaro.org>
Acked-by: Rusty Russell <rusty@rustcorp.com.au>
Coverity has recently added a check that will find when we don't check
the return code from fstat(2). Copy/paste the checking logic that
print_deps() has with an appropriate re-wording of the perror() message.
Signed-off-by: Tom Rini <trini@konsulko.com>
Signed-off-by: Michal Marek <mmarek@suse.com>
strrcmp only performs read access to the memory addressed by its
arguments so make them const pointers.
Signed-off-by: Nicolas Iooss <nicolas.iooss_linux@m4x.org>
Signed-off-by: Michal Marek <mmarek@suse.com>
The clear_config() is called just once at the beginning of this
program, but the global variable hashtab[] is already zero-filled
at the start-up.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: Michal Marek <mmarek@suse.com>
If the target string matches "CONFIG_", move the pointer p
forward. This saves several 7-chars adjustments.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: Michal Marek <mmarek@suse.com>
