The clean target needs ARCH=${ARCH} to clean up the tree for the correct
architecture. 'make (bin)deb-pkg' skips cleaning, but the preclean hook
may be executed if dpkg-buildpackage is directly used.
The binary-arch target does not need KERNELRELEASE because it is not
updated during the installation. KBUILD_BUILD_VERSION is not needed
either because binary-arch does not build vmlinux.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Currently, perf-tar*-src-pkg only uses 'git archive', but it is better
to make it work without relying on git.
The file, HEAD, which saves the commit hash, will be included in the
tarball only when the source tree is managed by git. The git tree is
more precisely checked; it has been copied from scripts/setlocalversion.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Change the source format from "1.0" to "3.0 (quilt)" because it works
more cleanly.
All files except .config and debian/ go into the orig tarball.
Add a single patch, debian/patches/config, and delete the ugly
extend-diff-ignore patterns.
The debian tarball will be compressed into *.debian.tar.xz by default.
If you like to use a different compression mode, you can pass the
command line option, DPKG_FLAGS=-Zgzip, for example.
The orig tarball only supports gzip for now. The combination of
gzip and xz is somewhat clumsy, but it is not a practical problem.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
If '..' belongs to the same filesystem, create a hard link instead of
a copy. In most cases, you can save disk space.
I do not want to use 'mv' because keeping linux.tar.gz is useful to
avoid unneeded rebuilding of the tarball.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
scripts/Makefile.package does not need to know the value of
KDEB_SOURCENAME because the source name can be taken from
debian/changelog by using dpkg-parsechangelog.
Move the default of KDEB_SOURCENAME (i.e. linux-upstream) to
scripts/package/mkdebian.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
If you run 'make (src)rpm-pkg', all objects are lost due to 'make clean',
which makes the incremental builds impossible.
Instead of cleaning, pass the exclude list to tar's --exclude-from
option.
Previously, the .config was contained in the source tarball.
With this commit, the source rpm consists of separate linux.tar.gz
and .config.
Remove stale comments. Now, 'make (src)rpm-pkg' works with O= option.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
The build rules of rpm-pkg and srcrpm-pkg are almost the same.
Remove the code duplication.
Change rpm-pkg to build binary packages from the source package generated
by srcrpm-pkg.
This changes the output directory of the srpm generated by 'make rpm-pkg'
because srcrpm-pkg overrides _srcrpmdir.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
If you run 'make deb-pkg', all objects are lost due to 'make clean',
which makes the incremental builds impossible.
Instead of cleaning, pass the exclude list to tar's --exclude-from
option.
Previously, *.diff.gz contained some check-in files such as
.clang-format, .cocciconfig.
With this commit, *.diff.gz will only contain the .config and debian/.
The other source files will go into the .orig tarball.
linux.tar.gz is rebuilt only when the source files that would go into
the tarball are changed.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
In short, the motivation of this commit is to build a source package
without cleaning the source tree.
The deb-pkg and (src)rpm-pkg targets first run 'make clean' before
creating a source tarball. Otherwise build artifacts such as *.o,
*.a, etc. would be included in the tarball. Yet, the tarball ends up
containing several garbage files since 'make clean' does not clean
everything.
Cleaning the tree every time is annoying since it makes the incremental
build impossible. It is desirable to create a source tarball without
cleaning the tree.
In fact, there are some ways to achieve this.
The easiest solution is 'git archive'. 'make perf-tar*-src-pkg' uses
it, but I do not like it because it works only when the source tree is
managed by git, and all files you want in the tarball must be committed
in advance.
I want to make it work without relying on git. We can do this.
Files that are ignored by git are generated files, so should be excluded
from the source tarball. We can list them out by parsing the .gitignore
files. Of course, .gitignore does not cover all the cases, but it works
well enough.
tar(1) claims to support it:
--exclude-vcs-ignores
Exclude files that match patterns read from VCS-specific ignore files.
Supported files are: .cvsignore, .gitignore, .bzrignore, and .hgignore.
The best scenario would be to use 'tar --exclude-vcs-ignores', but this
option does not work. --exclude-vcs-ignore does not understand any of
the negation (!), preceding slash, following slash, etc.. So, this option
is just useless.
Hence, I wrote this gitignore parser. The previous version [1], written
in Python, was so slow. This version is implemented in C, so it works
much faster.
I imported the code from git (commit: 23c56f7bd5f1), so we get the same
result.
This tool traverses the source tree, parsing all .gitignore files, and
prints file paths that are ignored by git.
The output is similar to 'git ls-files --ignored --directory --others
--exclude-per-directory=.gitignore', except
[1] Not sorted
[2] No trailing slash for directories
[2] is intentional because tar's --exclude-from option cannot handle
trailing slashes.
[How to test this tool]
$ git clean -dfx
$ make -s -j$(nproc) defconfig all # or allmodconifg or whatever
$ git archive -o ../linux1.tar --prefix=./ HEAD
$ tar tf ../linux1.tar | LANG=C sort > ../file-list1 # files emitted by 'git archive'
$ make scripts_package
HOSTCC scripts/list-gitignored
$ scripts/list-gitignored --prefix=./ -o ../exclude-list
$ tar cf ../linux2.tar --exclude-from=../exclude-list .
$ tar tf ../linux2.tar | LANG=C sort > ../file-list2 # files emitted by 'tar'
$ diff ../file-list1 ../file-list2 | grep -E '^(<|>)'
< ./Documentation/devicetree/bindings/.yamllint
< ./drivers/clk/.kunitconfig
< ./drivers/gpu/drm/tests/.kunitconfig
< ./drivers/hid/.kunitconfig
< ./fs/ext4/.kunitconfig
< ./fs/fat/.kunitconfig
< ./kernel/kcsan/.kunitconfig
< ./lib/kunit/.kunitconfig
< ./mm/kfence/.kunitconfig
< ./tools/testing/selftests/arm64/tags/
< ./tools/testing/selftests/arm64/tags/.gitignore
< ./tools/testing/selftests/arm64/tags/Makefile
< ./tools/testing/selftests/arm64/tags/run_tags_test.sh
< ./tools/testing/selftests/arm64/tags/tags_test.c
< ./tools/testing/selftests/kvm/.gitignore
< ./tools/testing/selftests/kvm/Makefile
< ./tools/testing/selftests/kvm/config
< ./tools/testing/selftests/kvm/settings
The source tarball contains most of files that are tracked by git. You
see some diffs, but it is just because some .gitignore files are wrong.
$ git ls-files -i -c --exclude-per-directory=.gitignore
Documentation/devicetree/bindings/.yamllint
drivers/clk/.kunitconfig
drivers/gpu/drm/tests/.kunitconfig
drivers/hid/.kunitconfig
fs/ext4/.kunitconfig
fs/fat/.kunitconfig
kernel/kcsan/.kunitconfig
lib/kunit/.kunitconfig
mm/kfence/.kunitconfig
tools/testing/selftests/arm64/tags/.gitignore
tools/testing/selftests/arm64/tags/Makefile
tools/testing/selftests/arm64/tags/run_tags_test.sh
tools/testing/selftests/arm64/tags/tags_test.c
tools/testing/selftests/kvm/.gitignore
tools/testing/selftests/kvm/Makefile
tools/testing/selftests/kvm/config
tools/testing/selftests/kvm/settings
[1]: https://lore.kernel.org/all/20230128173843.765212-1-masahiroy@kernel.org/
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Currently, setlocalversion uses any annotated tag for git-describe.
If we are at a tagged commit, it will not append the commit hash.
$ git checkout v6.2-rc1^
$ make -s defconfig kernelrelease
6.1.0-14595-g292a089d78d3
$ git tag -a foo -m foo
$ make -s kernelrelease
6.1.0
If a local tag 'foo' exists, it pretends to be a released version
'6.1.0', while there are many commits on top of it.
The output should be consistent irrespective of such a local tag.
Pass the correct release tag to --match option of git-describe.
In the mainline kernel, the SUBLEVEL is always '0', which is omitted
from the tag.
KERNELVERSION annotated tag
6.1.0 -> v6.1 (mainline)
6.2.0-rc5 -> v6.2-rc5 (mainline, release candidate)
6.1.7 -> v6.1.7 (stable)
To preserve the behavior in linux-next, use the tag derived from
localversion* files if exists. In linux-next, the local version is
specified by the localversion-next file.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Concatenate all components in the last line instead of accumulating
them into the 'res' variable.
No functional change is intended. A preparation for the next change.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
When there is a missing input file (vmlinux.o or Module.symvers), you
are likely to get a ton of unresolved symbols.
Currently, Kbuild automatically adds the -w option to allow module builds
to continue with warnings instead of errors.
This may not be what the user expects because it is generally more useful
to catch all possible issues at build time instead of at run time.
Let's not do what the user did not ask.
If you still want to build modules anyway, you can proceed by explicitly
setting KBUILD_MODPOST_WARN=1. Since you may miss a real issue, you need
to be aware of what you are doing.
Suggested-by: William McVicker <willmcvicker@google.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Tested-by: Will McVicker <willmcvicker@google.com>
If the source package fails to build, ../linux.orig is left over.
In the next run of 'make deb-pkg', you will get the following error:
dpkg-source: error: orig directory 'linux.orig' already exists, not overwriting, giving up; use -sA, -sK or -sP to override
You can manually remove ../linux.orig, but it is annoying.
Pass -sP down to dpkg-source.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Return earlier if we are not in the correct git repository. This makes
the code more readable.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
With the --short option given, scm_version() prints "+".
Just append it.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
.scmversion is used by (src)rpm-pkg and deb-pkg to carry KERNELRELEASE.
In fact, deb-pkg does not rely on it any more because the generated
debian/rules specifies KERNELRELEASE from the command line.
Do likwise for (src)rpm-pkg, and remove this feature.
For the same reason, you do not need to save LOCALVERSION in the
spec file.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
For each binary Debian package, a directory with the package name is
created in the debian directory. Correct the generated file matches in the
package's clean target, which were renamed without adjusting the target.
Fixes: 1694e94e4f ("builddeb: match temporary directory name to the package name")
Signed-off-by: Bastian Germann <bage@linutronix.de>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
No need to call chmod three times when it can do everything at once.
Signed-off-by: Sven Joachim <svenjoac@gmx.de>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
In the follow-up of commit fb3041d61f ("kbuild: fix SIGPIPE error
message for AR=gcc-ar and AR=llvm-ar"), Kees Cook pointed out that
tools should _not_ catch their own SIGPIPEs [1] [2].
Based on his feedback, LLVM was fixed [3].
However, Python's default behavior is to show noisy bracktrace when
SIGPIPE is sent. So, scripts written in Python are basically in the
same situation as the buggy llvm tools.
Example:
$ make -s allnoconfig
$ make -s allmodconfig
$ scripts/diffconfig .config.old .config | head -n1
-ALIX n
Traceback (most recent call last):
File "/home/masahiro/linux/scripts/diffconfig", line 132, in <module>
main()
File "/home/masahiro/linux/scripts/diffconfig", line 130, in main
print_config("+", config, None, b[config])
File "/home/masahiro/linux/scripts/diffconfig", line 64, in print_config
print("+%s %s" % (config, new_value))
BrokenPipeError: [Errno 32] Broken pipe
Python documentation [4] notes how to make scripts die immediately and
silently:
"""
Piping output of your program to tools like head(1) will cause a
SIGPIPE signal to be sent to your process when the receiver of its
standard output closes early. This results in an exception like
BrokenPipeError: [Errno 32] Broken pipe. To handle this case,
wrap your entry point to catch this exception as follows:
import os
import sys
def main():
try:
# simulate large output (your code replaces this loop)
for x in range(10000):
print("y")
# flush output here to force SIGPIPE to be triggered
# while inside this try block.
sys.stdout.flush()
except BrokenPipeError:
# Python flushes standard streams on exit; redirect remaining output
# to devnull to avoid another BrokenPipeError at shutdown
devnull = os.open(os.devnull, os.O_WRONLY)
os.dup2(devnull, sys.stdout.fileno())
sys.exit(1) # Python exits with error code 1 on EPIPE
if __name__ == '__main__':
main()
Do not set SIGPIPE’s disposition to SIG_DFL in order to avoid
BrokenPipeError. Doing that would cause your program to exit
unexpectedly whenever any socket connection is interrupted while
your program is still writing to it.
"""
Currently, tools/perf/scripts/python/intel-pt-events.py seems to be the
only script that fixes the issue that way.
tools/perf/scripts/python/compaction-times.py uses another approach
signal.signal(signal.SIGPIPE, signal.SIG_DFL) but the Python
documentation clearly says "Don't do it".
I cannot fix all Python scripts since there are so many.
I fixed some in the scripts/ directory.
[1]: https://lore.kernel.org/all/202211161056.1B9611A@keescook/
[2]: https://github.com/llvm/llvm-project/issues/59037
[3]: 4787efa380
[4]: https://docs.python.org/3/library/signal.html#note-on-sigpipe
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
Currently, these warnings are hidden with -Qunused-arguments in
KBUILD_CPPFLAGS. Once that option is removed, these warnings should be
turned into hard errors to make unconditionally added but unsupported
flags for the current compilation mode or target obvious due to a failed
build; otherwise, the warnings might just be ignored if the build log is
not checked.
Link: https://github.com/ClangBuiltLinux/linux/issues/1587
Signed-off-by: Nathan Chancellor <nathan@kernel.org>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
Tested-by: Linux Kernel Functional Testing <lkft@linaro.org>
Tested-by: Anders Roxell <anders.roxell@linaro.org>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
as-instr uses KBUILD_AFLAGS, but as-option uses KBUILD_CFLAGS. This can
cause as-option to fail unexpectedly when CONFIG_WERROR is set, because
clang will emit -Werror,-Wunused-command-line-argument for various -m
and -f flags in KBUILD_CFLAGS for assembler sources.
Callers of as-option and as-instr should be adding flags to
KBUILD_AFLAGS / aflags-y, not KBUILD_CFLAGS / cflags-y. Use
KBUILD_AFLAGS in all macros to clear up the initial problem.
Unfortunately, -Wunused-command-line-argument can still be triggered
with clang by the presence of warning flags or macro definitions because
'-x assembler' is used, instead of '-x assembler-with-cpp', which will
consume these flags. Switch to '-x assembler-with-cpp' in places where
'-x assembler' is used, as the compiler is always used as the driver for
out of line assembler sources in the kernel.
Finally, add -Werror to these macros so that they behave consistently
whether or not CONFIG_WERROR is set.
[nathan: Reworded and expanded on problems in commit message
Use '-x assembler-with-cpp' in a couple more places]
Link: https://github.com/ClangBuiltLinux/linux/issues/1699
Suggested-by: Masahiro Yamada <masahiroy@kernel.org>
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
Signed-off-by: Nathan Chancellor <nathan@kernel.org>
Tested-by: Linux Kernel Functional Testing <lkft@linaro.org>
Tested-by: Anders Roxell <anders.roxell@linaro.org>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
$(tmp-target) is a better fit for local use like this.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
