To print the pathname that will be used by shell in the current
environment, 'command -v' is a standardized way. [1]
'which' is also often used in scripts, but it is less portable.
When I worked on commit bd55f96fa9 ("kbuild: refactor cc-cross-prefix
implementation"), I was eager to use 'command -v' but it did not work.
(The reason is explained below.)
I kept 'which' as before but got rid of '> /dev/null 2>&1' as I
thought it was no longer needed. Sorry, I was wrong.
It works well on my Ubuntu machine, but Alexey Brodkin reports noisy
warnings on CentOS7 when 'which' fails to find the given command in
the PATH environment.
$ which foo
which: no foo in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin)
Given that behavior of 'which' depends on system (and it may not be
installed by default), I want to try 'command -v' once again.
The specification [1] clearly describes the behavior of 'command -v'
when the given command is not found:
Otherwise, no output shall be written and the exit status shall reflect
that the name was not found.
However, we need a little magic to use 'command -v' from Make.
$(shell ...) passes the argument to a subshell for execution, and
returns the standard output of the command.
Here is a trick. GNU Make may optimize this by executing the command
directly instead of forking a subshell, if no shell special characters
are found in the command and omitting the subshell will not change the
behavior.
In this case, no shell special character is used. So, Make will try
to run it directly. However, 'command' is a shell-builtin command,
then Make would fail to find it in the PATH environment:
$ make ARCH=m68k defconfig
make: command: Command not found
make: command: Command not found
make: command: Command not found
In fact, Make has a table of shell-builtin commands because it must
ask the shell to execute them.
Until recently, 'command' was missing in the table.
This issue was fixed by the following commit:
| commit 1af314465e5dfe3e8baa839a32a72e83c04f26ef
| Author: Paul Smith <psmith@gnu.org>
| Date: Sun Nov 12 18:10:28 2017 -0500
|
| * job.c: Add "command" as a known shell built-in.
|
| This is not a POSIX shell built-in but it's common in UNIX shells.
| Reported by Nick Bowler <nbowler@draconx.ca>.
Because the latest release is GNU Make 4.2.1 in 2016, this commit is
not included in any released versions. (But some distributions may
have back-ported it.)
We need to trick Make to spawn a subshell. There are various ways to
do so:
1) Use a shell special character '~' as dummy
$(shell : ~; command -v $(c)gcc)
2) Use a variable reference that always expands to the empty string
(suggested by David Laight)
$(shell command$${x:+} -v $(c)gcc)
3) Use redirect
$(shell command -v $(c)gcc 2>/dev/null)
I chose 3) to not confuse people. The stderr would not be polluted
anyway, but it will provide extra safety, and is easy to understand.
Tested on Make 3.81, 3.82, 4.0, 4.1, 4.2, 4.2.1
[1] http://pubs.opengroup.org/onlinepubs/9699919799/utilities/command.html
Fixes: bd55f96fa9 ("kbuild: refactor cc-cross-prefix implementation")
Cc: linux-stable <stable@vger.kernel.org> # 5.1
Reported-by: Alexey Brodkin <abrodkin@synopsys.com>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Tested-by: Alexey Brodkin <abrodkin@synopsys.com>
Add SPDX license identifiers to all Make/Kconfig files which:
- Have no license information of any form
These files fall under the project license, GPL v2 only. The resulting SPDX
license identifier is:
GPL-2.0
Reported-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
If you want to see if your linker supports a certain flag, then ask the
linker directly with ld-option (not the compiler with cc-ldoption).
Checking for linker flag support is an antipattern that complicates the
usage of various linkers other than bfd via -fuse-ld={bfd|gold|lld}.
Cc: clang-built-linux@googlegroups.com
Suggested-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
The 'addtree' and 'flags' in scripts/Kbuild.include are so compilecated
and ugly.
As I mentioned in [1], Kbuild should stop automatic prefixing of header
search path options.
I fixed up (almost) all Makefiles in the kernel. Now 'addtree' and
'flags' have been removed.
Kbuild still caters to add $(srctree)/$(src) and $(objtree)/$(obj)
to the header search path for O= building, but never touches extra
compiler options from ccflags-y etc.
[1]: https://patchwork.kernel.org/patch/9632347/
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
There is no more direct user of this macro; it is only used by
cc-ifversion.
Calling this macro is not efficient since it invokes the compiler to
get the compiler version. CONFIG_GCC_VERSION is already calculated in
the Kconfig stage, so Makefile can reuse it.
Here is a note about the slight difference between cc-version and
CONFIG_GCC_VERSION:
When using Clang, cc-version is evaluated to '0402' because Clang
defines __GNUC__ and __GNUC__MINOR__, and looks like GCC 4.2 in the
version point of view. On the other hand, CONFIG_GCC_VERSION=0
when $(CC) is clang.
There are currently two users of cc-ifversion:
arch/mips/loongson64/Platform
arch/powerpc/Makefile
They are not affected by this change.
The format of cc-version is <major><minor>, while CONFIG_GCC_VERSION
<major><minor><patch>. I adjusted cc-ifversion for the difference of
the number of digits.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
- $(word 1, <text>) is equivalent to $(firstword <text>)
- hardcode "gcc" instead of $(CC)
- minimize the shell script part
A little more notes in case $(filter-out -%, ...) is not clear.
arch/mips/Makefile passes prefixes depending on the configuration.
CROSS_COMPILE := $(call cc-cross-prefix, $(tool-archpref)-linux- \
$(tool-archpref)-linux-gnu- $(tool-archpref)-unknown-linux-gnu-)
In the Kconfig stage (e.g. when you run 'make defconfig'), neither
CONFIG_32BIT nor CONFIG_64BIT is defined. So, $(tool-archpref) is
empty. As a result, "-linux -linux-gnu- -unknown-linux-gnu" is passed
into cc-cross-prefix. The command 'which' assumes arguments starting
with a hyphen as command options, then emits the following messages:
Illegal option -l
Illegal option -l
Illegal option -u
I think it is strange to define CROSS_COMPILE depending on the CONFIG
options since you need to feed $(CC) to Kconfig, but it is how MIPS
Makefile currently works. Anyway, it would not hurt to filter-out
invalid strings beforehand.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
In Kbuild, if_changed and friends must have FORCE as a prerequisite.
Hence, $(filter-out FORCE,$^) or $(filter-out $(PHONY),$^) is a common
idiom to get the names of all the prerequisites except phony targets.
Add real-prereqs as a shorthand.
Note:
We cannot replace $(filter %.o,$^) in cmd_link_multi-m because $^ may
include auto-generated dependencies from the .*.cmd file when a single
object module is changed into a multi object module. Refer to commit
69ea912fda ("kbuild: remove unneeded link_multi_deps"). I added some
comment to avoid accidental breakage.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Rob Herring <robh@kernel.org>
Commit eea199b445 ("kbuild: remove unnecessary LEX_PREFIX and
YACC_PREFIX") removed the last users of this macro.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
You do not have to use define ... endef for filechk_* rules.
For simple cases, the use of assignment looks cleaner, IMHO.
I updated the usage for scripts/Kbuild.include in case somebody
misunderstands the 'define ... endif' is the requirement.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Heiko Carstens <heiko.carstens@de.ibm.com>
filechk_* rules often consist of multiple 'echo' lines. They must be
surrounded with { } or ( ) to work correctly. Otherwise, only the
string from the last 'echo' would be written into the target.
Let's take care of that in the 'filechk' in scripts/Kbuild.include
to clean up filechk_* rules.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
In commit 54a702f705 ("kbuild: mark $(targets) as .SECONDARY and
remove .PRECIOUS markers"), I missed one important feature of the
.SECONDARY target:
.SECONDARY with no prerequisites causes all targets to be
treated as secondary.
... which agrees with the policy of Kbuild.
Let's move it to scripts/Kbuild.include, with no prerequisites.
Note:
If an intermediate file is generated by $(call if_changed,...), you
still need to add it to "targets" so its .*.cmd file is included.
The arm/arm64 crypto files are generated by $(call cmd,shipped),
so they do not need to be added to "targets", but need to be added
to "clean-files" so "make clean" can properly clean them away.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
With the change of rule_cc_o_c / rule_as_o_S in the last commit, each
command is executed in a separate subshell. Rip off unneeded semicolons.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
The 'define' ... 'endef' directive is useful to confine a series of
shell commands into a single macro:
define foo
[action1]
[action2]
[action3]
endif
Each action is executed in a separate subshell.
However, rule_cc_o_c and rule_as_o_S in scripts/Makefile.build are
written as follows (with a trailing semicolon in each cmd_*):
define rule_cc_o_c
[action1] ; \
[action2] ; \
[action3] ;
endef
All shell commands are concatenated with '; \' so that it looks like
a single command from the Makefile point of view. This does not
exploit the benefits of 'define' ... 'endef' form because a single
shell command can be more simply written, like this:
rule_cc_o_c = \
[action1] ; \
[action2] ; \
[action3] ;
I guess the intention for the command concatenation was to let the
'@set -e' in if_changed_rule cover all the commands.
We can improve the readability by moving '@set -e' to the 'cmd' macro.
The combo of $(call echo-cmd,*) $(cmd_*) in rule_cc_o_c and rule_as_o_S
have been replaced with $(call cmd,*). The trailing back-slashes have
been removed.
Here is a note about the performance: the commands in rule_cc_o_c and
rule_as_o_S were previously executed all together in a single subshell,
but now each line in a separate subshell. This means Make will spawn
extra subshells [1]. I measured the build performance for
x86_64_defconfig + CONFIG_MODVERSIONS + CONFIG_TRIM_UNUSED_KSYMS
and I saw slight performance regression, but I believe code readability
and maintainability wins.
[1] Precisely, GNU Make may optimize this by executing the command
directly instead of forking a subshell, if no shell special
characters are found in the command line and omitting the subshell
will not change the behavior.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
My main motivation of this commit is to clean up scripts/Kbuild.include
and scripts/Makefile.build.
Currently, CONFIG_TRIM_UNUSED_KSYMS works with a tricky gimmick;
possibly exported symbols are detected by letting $(CPP) replace
EXPORT_SYMBOL* with a special string '=== __KSYM_*===', which is
post-processed by sed, and passed to fixdep. The extra preprocessing
is costly, and hacking cmd_and_fixdep is ugly.
I came up with a new way to find exported symbols; insert a dummy
symbol __ksym_marker_* to each potentially exported symbol. Those
dummy symbols are picked up by $(NM), post-processed by sed, then
appended to .*.cmd files. I collected the post-process part to a
new shell script scripts/gen_ksymdeps.sh for readability. The dummy
symbols are put into the .discard.* section so that the linker
script rips them off the final vmlinux or modules.
A nice side-effect is building with CONFIG_TRIM_UNUSED_KSYMS will
be much faster.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Nicolas Pitre <nico@linaro.org>
Currently, fixdep writes dependencies to .*.tmp, which is renamed to
.*.cmd after everything succeeds. This is a very safe way to avoid
corrupted .*.cmd files. The if_changed_dep has carried this safety
mechanism since it was added in 2002.
If fixdep fails for some reasons or a user terminates the build while
fixdep is running, the incomplete output from the fixdep could be
troublesome.
This is my insight about some bad scenarios:
[1] If the compiler succeeds to generate *.o file, but fixdep fails
to write necessary dependencies to .*.cmd file, Make will miss
to rebuild the object when headers or CONFIG options are changed.
In this case, fixdep should not generate .*.cmd file at all so
that 'arg-check' will surely trigger the rebuild of the object.
[2] A partially constructed .*.cmd file may not be a syntactically
correct makefile. The next time Make runs, it would include it,
then fail to parse it. Once this happens, 'make clean' is be the
only way to fix it.
In fact, [1] is no longer a problem since commit 9c2af1c737 ("kbuild:
add .DELETE_ON_ERROR special target"). Make deletes a target file on
any failure in its recipe. Because fixdep is a part of the recipe of
*.o target, if it fails, the *.o is deleted anyway. However, I am a
bit worried about the slight possibility of [2].
So, here is a solution. Let fixdep directly write to a .*.cmd file,
but allow makefiles to include it only when its corresponding target
exists.
This effectively reverts commit 2982c95357 ("kbuild: remove redundant
$(wildcard ...) for cmd_files calculation"), and commit 00d78ab2ba
("kbuild: remove dead code in cmd_files calculation in top Makefile")
because now we must check the presence of targets.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
There is one more user of $(cc-name) in the top Makefile. It is supposed
to detect Clang before invoking Kconfig, so it should still be there
in the $(shell ...) form. All the other users of $(cc-name) have been
replaced with $(CONFIG_CC_IS_CLANG). Hence, scripts/Kbuild.include does
not need to define cc-name any more.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
The last user of cc-fullversion was removed by commit f2910f0e68
("powerpc: remove old GCC version checks").
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Pull Kbuild updates from Masahiro Yamada:
- optimize kallsyms slightly
- remove check for old CFLAGS usage
- add some compiler flags unconditionally instead of evaluating
$(call cc-option,...)
- fix variable shadowing in host tools
- refactor scripts/mkmakefile
- refactor various makefiles
* tag 'kbuild-v4.20' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild:
modpost: Create macro to avoid variable shadowing
ASN.1: Remove unnecessary shadowed local variable
kbuild: use 'else ifeq' for checksrc to improve readability
kbuild: remove unneeded link_multi_deps
kbuild: add -Wno-unused-but-set-variable flag unconditionally
kbuild: add -Wdeclaration-after-statement flag unconditionally
kbuild: add -Wno-pointer-sign flag unconditionally
modpost: remove leftover symbol prefix handling for module device table
kbuild: simplify command line creation in scripts/mkmakefile
kbuild: do not pass $(objtree) to scripts/mkmakefile
kbuild: remove user ID check in scripts/mkmakefile
kbuild: remove VERSION and PATCHLEVEL from $(objtree)/Makefile
kbuild: add --include-dir flag only for out-of-tree build
kbuild: remove dead code in cmd_files calculation in top Makefile
kbuild: hide most of targets when running config or mixed targets
kbuild: remove old check for CFLAGS use
kbuild: prefix Makefile.dtbinst path with $(srctree) unconditionally
kallsyms: remove left-over Blackfin code
kallsyms: reduce size a little on 64-bit
Using macros in inline assembly allows us to work around bugs
in GCC's inlining decisions.
Compile macros.S and use it to assemble all C files.
Currently only x86 will use it.
Background:
The inlining pass of GCC doesn't include an assembler, so it's not aware
of basic properties of the generated code, such as its size in bytes,
or that there are such things as discontiuous blocks of code and data
due to the newfangled linker feature called 'sections' ...
Instead GCC uses a lazy and fragile heuristic: it does a linear count of
certain syntactic and whitespace elements in inlined assembly block source
code, such as a count of new-lines and semicolons (!), as a poor substitute
for "code size and complexity".
Unsurprisingly this heuristic falls over and breaks its neck whith certain
common types of kernel code that use inline assembly, such as the frequent
practice of putting useful information into alternative sections.
As a result of this fresh, 20+ years old GCC bug, GCC's inlining decisions
are effectively disabled for inlined functions that make use of such asm()
blocks, because GCC thinks those sections of code are "large" - when in
reality they are often result in just a very low number of machine
instructions.
This absolute lack of inlining provess when GCC comes across such asm()
blocks both increases generated kernel code size and causes performance
overhead, which is particularly noticeable on paravirt kernels, which make
frequent use of these inlining facilities in attempt to stay out of the
way when running on baremetal hardware.
Instead of fixing the compiler we use a workaround: we set an assembly macro
and call it from the inlined assembly block. As a result GCC considers the
inline assembly block as a single instruction. (Which it often isn't but I digress.)
This uglifies and bloats the source code - for example just the refcount
related changes have this impact:
Makefile | 9 +++++++--
arch/x86/Makefile | 7 +++++++
arch/x86/kernel/macros.S | 7 +++++++
scripts/Kbuild.include | 4 +++-
scripts/mod/Makefile | 2 ++
5 files changed, 26 insertions(+), 3 deletions(-)
Yay readability and maintainability, it's not like assembly code is hard to read
and maintain ...
We also hope that GCC will eventually get fixed, but we are not holding
our breath for that. Yet we are optimistic, it might still happen, any decade now.
[ mingo: Wrote new changelog describing the background. ]
Tested-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Nadav Amit <namit@vmware.com>
Acked-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: Denys Vlasenko <dvlasenk@redhat.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Michal Marek <michal.lkml@markovi.net>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Sam Ravnborg <sam@ravnborg.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-kbuild@vger.kernel.org
Link: http://lkml.kernel.org/r/20181003213100.189959-3-namit@vmware.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
$(srctree) always points to the top of the source tree whether
KBUILD_SRC is set or not.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Commit a0f97e06a4 ("kbuild: enable 'make CFLAGS=...' to add
additional options to CC") renamed CFLAGS to KBUILD_CFLAGS.
Commit 222d394d30 ("kbuild: enable 'make AFLAGS=...' to add
additional options to AS") renamed AFLAGS to KBUILD_AFLAGS.
Commit 06c5040cdb ("kbuild: enable 'make CPPFLAGS=...' to add
additional options to CPP") renamed CPPFLAGS to KBUILD_CPPFLAGS.
For some reason, LDFLAGS was not renamed.
Using a well-known variable like LDFLAGS may result in accidental
override of the variable.
Kbuild generally uses KBUILD_ prefixed variables for the internally
appended options, so here is one more conversion to sanitize the
naming convention.
I did not touch Makefiles under tools/ since the tools build system
is a different world.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Reviewed-by: Palmer Dabbelt <palmer@sifive.com>
