[ Upstream commit c61566538968ffb040acc411246fd7ad38c7e8c9 ]
The make deb-pkg target calls debian-orig which attempts to either
hard link the source .tar to the build-output location or copy the
source .tar to the build-output location. The test to determine
whether to ln or cp is incorrectly expanded by Make and consequently
always attempts to ln the source .tar. This fix corrects the escaping
of '$' so that the test is expanded by the shell rather than by Make
and appropriately selects between ln and cp.
Fixes: b44aa8c96e ("kbuild: deb-pkg: make .orig tarball a hard link if possible")
Signed-off-by: Thayne Harbaugh <thayne@mastodonlabs.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
'make srcdeb-pkg' generates a source package, which you can build
later by using dpkg-buildpackage.
In older dpkg versions, 'dpkg-buildpackage --jobs=N' sets not only
DEB_BUILD_OPTIONS but also MAKEFLAGS. Hence, passing -j or --jobs
to dpkg-buildpackage was enough for kicking the parallel execution.
The behavior was changed by commit 1d0ea9b2ba3f ("dpkg-buildpackage:
Change -j, --jobs semantics to non-force mode") of dpkg project. [1]
Since then, 'dpkg-buildpackage --jobs=N' sets only DEB_BUILD_OPTIONS,
which is not parsed by the current debian/rules. To build the package
in parallel, you need to pass the alternative --jobs-force option or
set the MAKEFLAGS environment variable.
Debian policy [2] suggests the following code snippet for debian/rules.
ifneq (,$(filter parallel=%,$(DEB_BUILD_OPTIONS)))
NUMJOBS = $(patsubst parallel=%,%,$(filter parallel=%,$(DEB_BUILD_OPTIONS)))
MAKEFLAGS += -j$(NUMJOBS)
endif
I tweaked the code to filter out parallel=1 and passed --jobs=1 to
dpkg-buildpackage from scripts/Makefile.package. It is needed to force
'make deb-pkg' without the -j option to run in serial. Please note that
dpkg-buildpackage sets parallel=<nproc> in DEB_BUILD_OPTIONS by default
(that is, --jobs=auto is the default) and --jobs=1 is needed to restore
the serial execution. When dpkg-buildpackage is invoked from Kbuild,
the number of jobs is inherited from the top level Makefile. Passing
--jobs=1 to dpkg-buildpackage allows debian/rules to skip parsing
DEB_BUILD_OPTIONS.
[1] 1d0ea9b2ba
[2] https://www.debian.org/doc/debian-policy/ch-source.html#s-debianrules-options
Reported-by: Bastian Germann <bage@linutronix.de>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Debian Policy "4.9. Main building script: debian/rules" requires
"debian/rules must start with the line #!/usr/bin/make -f". [1]
Currently, Kbuild does not follow this policy.
When Kbuild generates debian/rules, "#!$(command -v $MAKE) -f" is
expanded by shell. The resuling string may not be "#!/usr/bin/make -f".
There was a reason to opt out the Debian policy.
If you run '/path/to/my/custom/make deb-pkg', debian/rules must also be
invoked by the same Make program. If #!/usr/bin/make were hard-coded in
debian/rules, the sub-make would be executed by a possibly different
Make version.
This is problematic due to the MAKEFLAGS incompatibility, especially the
job server flag. Old Make versions used --jobserver-fds to propagate job
server file descriptors, but Make >= 4.2 uses --jobserver-auth. The flag
disagreement between the parent/child Makes would result in a process
fork explosion.
However, having a non-standard path in the shebang causes another issue;
the generated source package is not portable as such a path does not
exist in other build environments.
This commit solves those conflicting demands.
Hard-code '#!/usr/bin/make -f' in debian/rules to create a portable and
Debian-compliant source package.
Pass '--rules-file=$(MAKE) -f debian/rules' when dpkg-buildpackage is
invoked from Makefile so that debian/rules is executed by the same Make
program as used to start Kbuild.
[1] https://www.debian.org/doc/debian-policy/ch-source.html#main-building-script-debian-rules
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Tested-by: Nathan Chancellor <nathan@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
Commit 8818039f95 ("kbuild: add ability to make source rpm buildable
using koji") added the BuildRequires: field.
Checking the build dependency is fine, but one annoyance is that
'make (bin)rpm-pkg' fails on non-rpm systems [1]. For example, Debian
provides rpmbuild via 'apt install rpm', but of course cannot meet the
requirement listed in the BuildRequires: field.
It is possible to pass RPMOPTS=--nodeps to work around it, but it is
reasonable to do it automatically.
If 'rpm -q rpm' fails, it is not an RPM-managed system. (The command
'rpm' is not installed at all, or was installed by other means.)
In that case, pass --nodeps to skip the build dependency check.
[1]: https://lore.kernel.org/linux-kbuild/Y6mkdYQYmjUz7bqV@li-4a3a4a4c-28e5-11b2-a85c-a8d192c6f089.ibm.com/
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Merge the similar build targets.
Also, make the output location consistent.
Previously, source packages were created in the build directory,
while binary packages under ~/rpmbuild/RPMS/.
Now, Kbuild creates the rpmbuild/ directory in the build directory,
and saves all packages under it.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Currently, 'make rpm-pkg' always builds the kernel from the pristine
source tree in the ~/rpmbuild/BUILD/ directory.
Build the kernel incrementally just like 'make binrpm-pkg'.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Now kernel.spec and binkernel.spec have the exactly same contents.
Use kernel.spec for binrpm-pkg as well.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
scripts/package/mkspec preprocesses the spec file by sed, but it is
unreadable. This commit removes the last portion of the sed scripting.
Remove the $S$M prefixes from the conditionally generated lines.
Instead, surround the code with %if %{with_devel} ... %endif.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
To reduce the preprocess of the spec file, invoke the kernel build
from rpmbuild.
Run init/build-version to increment the release number not only for
binrpm-pkg but also for srcrpm-pkg and rpm-pkg.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
There are some cases where we want to run a command with the same
environment variables as Kbuild uses. For example, 'make coccicheck'
invokes scripts/coccicheck from the top Makefile so that the script can
reference to ${LINUXINCLUDE}, ${KBUILD_EXTMOD}, etc. The top Makefile
defines several phony targets that run a script.
We do it also for an internally used script, which results in a somewhat
complex call graph.
One example:
debian/rules binary-arch
-> make intdeb-pkg
-> scripts/package/builddeb
It is also tedious to add a dedicated target like 'intdeb-pkg' for each
use case.
Add a generic target 'run-command' to run an arbitrary command in an
environment with all Kbuild variables set.
The usage is:
$ make run-command KBUILD_RUN_COMMAND=<command>
The concept is similar to:
$ dpkg-architecture -c <command>
This executes <command> in an environment which has all DEB_* variables
defined.
Convert the existing 'make intdeb-pkg'.
Another possible usage is to interrogate a Make variable.
$ make run-command KBUILD_RUN_COMMAND='echo $(KBUILD_CFLAGS)'
might be useful to see KBUILD_CFLAGS set by the top Makefile.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Currently, $MAKE will expand to the GNU Make program that created the
source RPM. This is problematic if you carry it to a different build
host to run 'rpmbuild' there.
Consider this command:
$ /path/to/my/custom/make srcrpm-pkg
The spec file in the SRPM will record '/path/to/my/custom/make', which
exists only on that build environment.
To create a portable SRPM, the spec file should avoid hard-coding $MAKE.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Pull Kbuild updates from Masahiro Yamada:
- Refactor scripts/kallsyms to make it faster and easier to maintain
- Clean up menuconfig
- Provide Clang with hard-coded target triple instead of CROSS_COMPILE
- Use -z pack-relative-relocs flags instead of --use-android-relr-tags
for arm64 CONFIG_RELR
- Add srcdeb-pkg target to build only a Debian source package
- Add KDEB_SOURCE_COMPRESS option to specify the compression for a
Debian source package
- Misc cleanups and fixes
* tag 'kbuild-v6.4' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild:
kbuild: deb-pkg: specify targets in debian/rules as .PHONY
sparc: unify sparc32/sparc64 archhelp
kbuild: rpm-pkg: remove kernel-drm PROVIDES
kbuild: deb-pkg: add KDEB_SOURCE_COMPRESS to specify source compression
kbuild: add srcdeb-pkg target
Makefile: use -z pack-relative-relocs
kbuild: clang: do not use CROSS_COMPILE for target triple
kconfig: menuconfig: reorder functions to remove forward declarations
kconfig: menuconfig: remove unused M_EVENT macro
kconfig: menuconfig: remove OLD_NCURSES macro
kbuild: builddeb: Eliminate debian/arch use
scripts/kallsyms: update the usage in the comment block
scripts/kallsyms: decrease expand_symbol() / cleanup_symbol_name() calls
scripts/kallsyms: change the output order
scripts/kallsyms: move compiler-generated symbol patterns to mksysmap
scripts/kallsyms: exclude symbols generated by itself dynamically
scripts/mksysmap: use sed with in-line comments
scripts/mksysmap: remove comments described in nm(1)
scripts/kallsyms: remove redundant code for omitting U and N
kallsyms: expand symbol name into comment for debugging
Add KDEB_SOURCE_COMPRESS to specify the compression for the orig and
debian tarballs. (cf. the existing KDEB_COMPRESS is used to specify
the compression for binary packages.)
Supported algorithms are gzip, bzip2, lzma, and xz, all of which are
supported by dpkg-source.
The current default is gzip. You can change it via the environment
variable, for example, 'KDEB_SOURCE_COMPRESS=xz make deb-pkg'.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
This new target builds only the debian source package.
Unify the build rules of deb-pkg, srcdeb-pkg, bindeb-pkg to avoid
code duplication.
--no-check-builddeps is added to srcdeb-pkg so that build dependencies
will not be checked.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
Since commit f8d94c4e40 ("kbuild: do not create intermediate *.tar
for source tarballs"), 'make rpm-pkg' fails because the prefix of the
source tarball is 'linux.tar/' instead of 'linux/'. $(basename $@)
strips only '.gz' from the filename linux.tar.gz.
You need to strip two suffixes from compressed tarballs and one suffix
from uncompressed tarballs (for example 'perf-6.3.0.tar' generated by
'make perf-tar-src-pkg').
One tricky fix might be --prefix=$(firstword $(subst .tar, ,$@))/
but I think it is better to hard-code the prefix.
Fixes: f8d94c4e40 ("kbuild: do not create intermediate *.tar for source tarballs")
Reported-by: Jiwei Sun <sunjw10@lenovo.com>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
Commit 05e96e96a3 ("kbuild: use git-archive for source package
creation") split the compression as a separate step to factor out
the common build rules.
With the previous commit, we got back to the situation where source
tarballs are compressed on-the-fly.
There is no reason to keep the separate compression rules.
Generate the comressed tar packages directly.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nathan Chancellor <nathan@kernel.org>
Since commit 05e96e96a3 ("kbuild: use git-archive for source package
creation"), a source tarball is created in two steps; create *.tar file
then compress it. I split the compression as a separate rule because I
just thought 'git archive' supported only gzip.
For other compression algorithms, I could pipe the two commands:
$ git archive HEAD | xz > linux.tar.xz
I read git-archive(1) carefully, and I realized GIT had provided a
more elegant way:
$ git -c tar.tar.xz.command=xz archive -o linux.tar.xz HEAD
This commit uses 'tar.tar.*.command' configuration to specify the
compression backend so we can compress a source tarball on-the-fly.
GIT commit 767cf4579f0e ("archive: implement configurable tar filters")
is more than a decade old, so it should be available on almost all build
environments.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nathan Chancellor <nathan@kernel.org>
The two commands, cmd_archive_linux and cmd_archive_perf, are similar.
Merge them to make it easier to add more changes to the git-archive
command.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nathan Chancellor <nathan@kernel.org>
When the source tree is dirty and contains untracked files, package
builds may fail, for example, when a broken symlink exists, a file
path contains whitespaces, etc.
Since commit 05e96e96a3 ("kbuild: use git-archive for source package
creation"), the source tarball only contains committed files because
it is created by 'git archive'. scripts/package/gen-diff-patch tries
to address the diff from HEAD, but including untracked files by the
hand-crafted script introduces more complexity. I wrote a patch [1] to
make it work in most cases, but still wonder if this is what we should
aim for.
To simplify the code, this patch just gives up untracked files. Going
forward, it is your responsibility to do 'git add' for what you want in
the source package. The script shows a warning just in case you forgot
to do so. It should be checked only when building source packages.
[1]: https://lore.kernel.org/all/CAK7LNAShbZ56gSh9PrbLnBDYKnjtTkHMoCXeGrhcxMvqXGq9=g@mail.gmail.com/2-0001-kbuild-make-package-builds-more-robust.patch
Fixes: 05e96e96a3 ("kbuild: use git-archive for source package creation")
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
Prepare to add more files to the source RPM.
Also, fix the build error when KCONFIG_CONFIG is set:
error: Bad file: ./.config: No such file or directory
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Currently, perf-tar*-src-pkg only uses 'git archive', but it is better
to make it work without relying on git.
The file, HEAD, which saves the commit hash, will be included in the
tarball only when the source tree is managed by git. The git tree is
more precisely checked; it has been copied from scripts/setlocalversion.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Change the source format from "1.0" to "3.0 (quilt)" because it works
more cleanly.
All files except .config and debian/ go into the orig tarball.
Add a single patch, debian/patches/config, and delete the ugly
extend-diff-ignore patterns.
The debian tarball will be compressed into *.debian.tar.xz by default.
If you like to use a different compression mode, you can pass the
command line option, DPKG_FLAGS=-Zgzip, for example.
The orig tarball only supports gzip for now. The combination of
gzip and xz is somewhat clumsy, but it is not a practical problem.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
If '..' belongs to the same filesystem, create a hard link instead of
a copy. In most cases, you can save disk space.
I do not want to use 'mv' because keeping linux.tar.gz is useful to
avoid unneeded rebuilding of the tarball.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
