Commit Graph
587 Commits
Author SHA1 Message Date
Michael Tokarev 96420a30e0 tests/: spelling fixes
with some rewording in
 tests/qemu-iotests/298
 tests/qtest/fuzz/generic_fuzz.c
 tests/unit/test-throttle.c
as suggested by Eric.

Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
Reviewed-by: Eric Blake <eblake@redhat.com>
2023-09-08 13:08:52 +03:00
Alex Bennée 6445c2cace tests/docker: cleanup non-verbose output
Even with --quiet docker will spam the sha256 to the console. Avoid
this by redirecting stdout. While we are at it fix the name we echo
which was broken during 0b1a649047 (tests/docker: use direct RUNC call
to build containers).

Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230829161528.2707696-3-alex.bennee@linaro.org>
2023-08-30 14:57:44 +01:00
Daniel P. BerrangéandAlex Bennée 2f7350cd43 gitlab: enable ccache for many build jobs
The `ccache` tool can be very effective at reducing compilation times
when re-running pipelines with only minor changes each time. For example
a fresh 'build-system-fedora' job will typically take 20 minutes on the
gitlab.com shared runners. With ccache this is reduced to as little as
6 minutes.

Normally meson would auto-detect existance of ccache in $PATH and use
it automatically, but the way we wrap meson from configure breaks this,
as we're passing in an config file with explicitly set compiler paths.
Thus we need to add $CCACHE_WRAPPERSPATH to the front of $PATH. For
unknown reasons if doing this in msys though, gcc becomes unable to
invoke 'cc1' when run from meson. For msys we thus set CC='ccache gcc'
before invoking 'configure' instead.

A second problem with msys is that cache misses are incredibly
expensive, so enabling ccache massively slows down the build when
the cache isn't well populated. This is suspected to be a result of
the cost of spawning processes under the msys architecture. To deal
with this we set CCACHE_DEPEND=1 which enables ccache's 'depend_only'
strategy. This avoids extra spawning of the pre-processor during
cache misses, with the downside that is it less likely ccache will
find a cache hit after semantically benign compiler flag changes.
This is the lesser of two evils, as otherwise we can't use ccache
at all under msys and remain inside the job time limit.

If people are finding ccache to hurt their pipelines, it can be
disabled by setting the 'CCACHE_DISABLE=1' env variable against
their gitlab fork CI settings.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20230804111054.281802-2-berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230829161528.2707696-2-alex.bennee@linaro.org>
2023-08-30 14:57:33 +01:00
Paolo Bonzini 7ace219303 tests/docker: add python3-tomli dependency to containers
Instead of having CI pick tomli from the vendored wheel at configure
time, place it in the containers.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-08-28 09:55:48 +02:00
Paolo Bonzini dcb8541b0b lcitool: bump libvirt-ci submodule and regenerate
This brings in a newer version of the pipewire mapping, so rename it.

Python 3.9 and 3.10 do not seem to work in OpenSUSE LEAP 15.5 (weird,
because 3.9 persisted from 15.3 to 15.4) so bump the Python runtime
version to 3.11.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-08-28 09:55:47 +02:00
Paolo Bonzini 87f77f58d8 dockerfiles: bump tricore cross compiler container to Debian 11
With the release of version 12 on June 10, 2023, Debian 10 is
not supported anymore.  Modify the cross compiler container to
build on a newer version.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-08-28 09:55:05 +02:00
Marc-André Lureau 62259d816c tests/lcitool: add pipewire
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20230506163735.3481387-3-marcandre.lureau@redhat.com>
2023-07-17 15:22:56 +04:00
Alex Bennée 0101dd71b0 tests/docker: convert riscv64-cross to lcitool
We still need to base this on Debian Sid until riscv64 is promoted to
a release architecture (or another distro provides a full cross
compile target). We use the new qemu-minimal project description to
avoid bringing in all the extra dependencies because every extra
package is another chance for sid to fail.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230630180423.558337-16-alex.bennee@linaro.org>
2023-07-03 12:51:53 +01:00
Alex Bennée 690be80412 tests/lcitool: add an explicit gcc-native package
We need a native compiler to build the hexagon codegen tools. In our
current images we already have a gcc as a side effect of a broken
dependency between gcovr and lcov but this will be fixed when we move
to bookworm. See
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=987818 for details.

Update the packages while we are at it.

Acked-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230630180423.558337-14-alex.bennee@linaro.org>
2023-07-03 12:51:47 +01:00
Erik SkultetyandAlex Bennée 76eb439c0f tests/lcitool: Bump fedora container versions
Fedora 37 -> 38

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Acked-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <c9b00e573a7a80fc6ce5c68595382f5c916a9195.1685528076.git.eskultet@redhat.com>
[AJB: Dropped alpine (in prev commit), reflow commit msg]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230630180423.558337-13-alex.bennee@linaro.org>
2023-07-03 12:51:40 +01:00
Alex Bennée c7374742d8 tests/lcitool: update to latest version
We need this for the riscv64 and gcc-native mappings. As the older
alpine release has been dropped from the mappings we also need to bump
the version of alpine we use.

Acked-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230630180423.558337-12-alex.bennee@linaro.org>
2023-07-03 12:51:37 +01:00
Alex Bennée 4ab20b513e tests/docker: add test-fuzz
Running the fuzzer requires some hoop jumping and some problems only
show up in containers. This basically replicates the build-oss-fuzz
job from our CI so we can run in the same containers we use in CI.

Reviewed-by: Alexander Bulekov <alxndr@bu.edu>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230630180423.558337-10-alex.bennee@linaro.org>
2023-07-03 12:51:26 +01:00
Paolo Bonzini a2696204de tests/docker: simplify HOST_ARCH definition
ARCH is always empty, so just define HOST_ARCH as the result of uname.

Acked-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-05-26 10:17:32 +02:00
John SnowandPaolo Bonzini a22a4b29ad tests/docker: add python3-venv dependency
Several debian-based tests need the python3-venv dependency as a
consequence of Debian debundling the "ensurepip" module normally
included with Python.

As mkvenv.py stands as of this commit, Debian requires EITHER:

(A) setuptools and pip, or
(B) ensurepip

mkvenv is a few seconds faster if you have setuptools and pip, so
developers should prefer the first requirement. For the purposes of CI,
the time-save is a wash; it's only a matter of who is responsible for
installing pip and when; the timing is about the same.

Arbitrarily, I chose adding ensurepip to the test configuration because
it is normally part of the Python stdlib, and always having it allows us
a more consistent cross-platform environment.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20230511035435.734312-12-jsnow@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-05-18 08:53:51 +02:00
Ani SinhaandThomas Huth da9000784c tests/lcitool: Add mtools and xorriso and remove genisoimage as dependencies
Bios bits avocado tests need mformat (provided by the mtools package) and
xorriso tools in order to run within gitlab CI containers. Add those
dependencies within the Dockerfiles so that containers can be built with
those tools present and bios bits avocado tests can be run there.

xorriso package conflicts with genisoimage package on some distributions.
Therefore, it is not possible to have both the packages at the same time
in the container image uniformly for all distribution flavors. Further,
on some distributions like RHEL, both xorriso and genisoimage
packages provide /usr/bin/genisoimage and on some other distributions like
Fedora, only genisoimage package provides the same utility.
Therefore, this change removes the dependency on geninsoimage for building
container images altogether keeping only xorriso package. At the same time,
cdrom-test.c is updated to use and check for existence of only xorrisofs.

Signed-off-by: Ani Sinha <anisinha@redhat.com>
Message-Id: <20230504154611.85854-3-anisinha@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-05-16 09:14:18 +02:00
Alex Bennée 3217b84f3c tests/docker: bump the xtensa base to debian:11-slim
Stretch is going out of support so things like security updates will
fail. As the toolchain itself is binary it hopefully won't mind the
underlying OS being updated.

Message-Id: <20230503091244.1450613-3-alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reported-by: Richard Henderson <richard.henderson@linaro.org>
2023-05-10 15:58:05 +01:00
Peter KrempaandPaolo Bonzini 5f9efbbcf6 tests: lcitool: Switch to OpenSUSE Leap 15.4
Since OpenSUSE Leap 15 counts as a single major release of an LTS distribution,
lcitool has changed the target name to remove the minor version.  Adjust the
mappings and refresh script.

This also updates the dockerfile to 15.4, since the 15.3 version is EOL now:

https://get.opensuse.org/leap/15.3

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <a408b7f241ac59e5944db6ae2360a792305c36e0.1681735482.git.pkrempa@redhat.com>
[Adjust for target name change and reword commit message. - Paolo]
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-04-20 11:17:36 +02:00
Peter KrempaandPaolo Bonzini dacc122e5e tests: libvirt-ci: Update to commit '2fa24dce8bc'
Update to commit which has fixes needed for OpenSUSE 15.4 and
re-generate output files.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Message-Id: <bd11b5954d3dd1e989699370af2b9e2e0c77194a.1681735482.git.pkrempa@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-04-20 11:17:36 +02:00
Marco LiebelandAlex Bennée 6e3be02291 Use hexagon toolchain version 16.0.0
Signed-off-by: Marco Liebel <quic_mliebel@quicinc.com>
Reviewed-by: Brian Cain <bcain@quicinc.com>
Message-Id: <20230329142108.1199509-1-quic_mliebel@quicinc.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230403134920.2132362-8-alex.bennee@linaro.org>
2023-04-04 15:56:44 +01:00
Alex Bennée 55154c5785 tests/docker: all add DOCKER_BUILDKIT to RUNC environment
It seems we also need to pass DOCKER_BUILDKIT as an argument to docker
itself to get the full benefit of caching.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Suggested-by: Fabiano Rosas <farosas@suse.de>
Tested-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230315174331.2959-3-alex.bennee@linaro.org>
2023-03-22 15:06:57 +00:00
Marc-André Lureau e387ef472f tests/docker: fix a win32 error due to portability
docker.py is run during configure, and produces an error: No module
named 'pwd'.

Use a more portable and recommended alternative to lookup the user
"login name".

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20230306122751.2355515-4-marcandre.lureau@redhat.com>
2023-03-13 15:39:31 +04:00
Alex Bennée bbe307639d tests/docker: add zstdtools to the images
We need this to be able to run the tuxrun_baseline tests in CI which
in turn helps us reduce overhead running other tests. We need to
update libvirt-ci and refresh the generated files by running 'make
lcitool-refresh' to get the new mapping.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20230228190653.1602033-24-alex.bennee@linaro.org>
2023-03-01 12:45:16 +00:00
Alex Bennée 789bc54117 tests/docker: use direct RUNC call to run test jobs
If we build them without the script we can certainly run them without
it.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230228190653.1602033-22-alex.bennee@linaro.org>
2023-03-01 12:45:11 +00:00
Alex Bennée 0b1a649047 tests/docker: use direct RUNC call to build containers
We don't really need stuff from docker.py to do the build as we have
everything we need with a direct call. We do rely on the dockerfiles
being able to tweak the UID/name mapping as the last step.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230228190653.1602033-21-alex.bennee@linaro.org>
2023-03-01 12:45:11 +00:00
Alex Bennée 93bd2954f4 tests/docker: add USER stanzas to non-lci images
These are flat but not generated by lcitool so we need to manually
update them with the `useradd` stanza.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230228190653.1602033-20-alex.bennee@linaro.org>
2023-03-01 12:45:11 +00:00