Some of our Python scripts still include the line
from __future__ import print_function
which is intended to allow a Python 2 to handle the Python 3 print()
syntax. This particular part of the future arrived many years ago,
and our minimum Python version is 3.9, so we don't need to keep
this line around.
NB: the scripts in tests/tcg/*/gdbstub/ are run with whatever Python
gdb was built against, but we can safely assume that that was a
Python 3 because our supported distros are all on Python 3. In any
case these are only run as part of "make check-tcg", not by
end-users.
Commit created with:
sed -i -e '/import print_function/d' $(git grep -l 'from __future__')
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-id: 20250819102409.2117969-1-peter.maydell@linaro.org
We can now delete the old Perl kernel-doc script. For posterity,
this is a complete diff of the local changes that we were carrying
between the kernel's Perl script as of kernel commit 72b97d0b911872ba
(the last time we synced it) and our local copy:
--- /tmp/kdoc 2025-08-14 10:42:47.620331939 +0100
+++ scripts/kernel-doc 2025-02-17 10:44:34.528421457 +0000
@@ -1,5 +1,5 @@
#!/usr/bin/env perl
-# SPDX-License-Identifier: GPL-2.0
+# SPDX-License-Identifier: GPL-2.0-only
use warnings;
use strict;
@@ -224,12 +224,12 @@
my $type_fp_param = '\@(\w+)\(\)'; # Special RST handling for func ptr params
my $type_fp_param2 = '\@(\w+->\S+)\(\)'; # Special RST handling for structs with func ptr params
my $type_env = '(\$\w+)';
-my $type_enum = '\&(enum\s*([_\w]+))';
-my $type_struct = '\&(struct\s*([_\w]+))';
-my $type_typedef = '\&(typedef\s*([_\w]+))';
-my $type_union = '\&(union\s*([_\w]+))';
-my $type_member = '\&([_\w]+)(\.|->)([_\w]+)';
-my $type_fallback = '\&([_\w]+)';
+my $type_enum = '#(enum\s*([_\w]+))';
+my $type_struct = '#(struct\s*([_\w]+))';
+my $type_typedef = '#(([A-Z][_\w]*))';
+my $type_union = '#(union\s*([_\w]+))';
+my $type_member = '#([_\w]+)(\.|->)([_\w]+)';
+my $type_fallback = '(?!)'; # this never matches
my $type_member_func = $type_member . '\(\)';
# Output conversion substitutions.
@@ -1745,6 +1745,9 @@
)+
\)\)\s+//x;
+ # Strip QEMU specific compiler annotations
+ $prototype =~ s/QEMU_[A-Z_]+ +//;
+
# Yes, this truly is vile. We are looking for:
# 1. Return type (may be nothing if we're looking at a macro)
# 2. Function name
@@ -2057,7 +2060,7 @@
}
elsif (/$doc_decl/o) {
$identifier = $1;
- if (/\s*([\w\s]+?)(\(\))?\s*-/) {
+ if (/\s*([\w\s]+?)(\s*-|:)/) {
$identifier = $1;
}
@@ -2067,7 +2070,7 @@
$contents = "";
$section = $section_default;
$new_start_line = $. + 1;
- if (/-(.*)/) {
+ if (/[-:](.*)/) {
# strip leading/trailing/multiple spaces
$descr= $1;
$descr =~ s/^\s*//;
These changes correspond to:
06e2329636 license: Update deprecated SPDX tag GPL-2.0 to GPL-2.0-only
(a bulk change which we won't bother to re-apply to this third-party script)
b30df2751e scripts/kernel-doc: strip QEMU_ from function definitions
4cf4179441 docs: tweak kernel-doc for QEMU coding standards
We have already applied the equivalent of these changes to the
Python code in libs/kdoc/ in the preceding commits.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
This commit makes the equivalent changes to the Python script that we
had for the old Perl script in commit 4cf4179441 ("docs: tweak
kernel-doc for QEMU coding standards"). To repeat the rationale from
that commit:
Surprisingly, QEMU does have a pretty consistent doc comment style and
it is not very different from the Linux kernel's. Of the documentation
"sigils", only "#" separates the QEMU doc comment style from Linux's,
and it has 200+ instances vs. 6 for the kernel's '&struct foo' (all in
accel/tcg/translate-all.c), so it's clear that the two standards are
different in this respect. In addition, our structs are typedefed and
recognized by CamelCase names.
Note that in 4cf4179441 we used '(?!)' as our type_fallback regex;
this is strictly not quite a replacement for the upstream
'\&([_\w]+)', because the latter includes a group that can later be
matched with \1, and the former does not. The old perl script did
not care about this, but the python version does, so we must include
the extra set of brackets to ensure we have a group.
This commit does not include all the same changes that 4cf4179441
did. Of the missing pieces, some had already gone in an earlier
kernel-doc update; the parts we still had but do not include here are:
@@ -2057,7 +2060,7 @@
}
elsif (/$doc_decl/o) {
$identifier = $1;
- if (/\s*([\w\s]+?)(\(\))?\s*-/) {
+ if (/\s*([\w\s]+?)(\s*-|:)/) {
$identifier = $1;
}
@@ -2067,7 +2070,7 @@
$contents = "";
$section = $section_default;
$new_start_line = $. + 1;
- if (/-(.*)/) {
+ if (/[-:](.*)/) {
# strip leading/trailing/multiple spaces
$descr= $1;
$descr =~ s/^\s*//;
The second of these is already in the upstream version: the line r =
KernRe("[-:](.*)") in process_name() matches the regex we have. The
first change has been refactored into the doc_begin_data and
doc_begin_func changes. Since the output HTML for QEMU's
documentation has no relevant changes with the new kerneldoc, we
assume that this too has been handled upstream.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Message-id: 20250814171324.1614516-6-peter.maydell@linaro.org
This commit is the Python version of our older commit
b30df2751e ("scripts/kernel-doc: strip QEMU_ from function definitions").
Some versions of Sphinx get confused if function attributes are
left on the C code from kernel-doc; strip out any QEMU_* prefixes
from function prototypes.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Message-id: 20250814171324.1614516-5-peter.maydell@linaro.org
We last synced our copy of kerneldoc with Linux back in 2020. In the
interim, upstream has entirely rewritten the script in Python, and
the new Python version is split into a main script plus some
libraries in the kernel's scripts/lib/kdoc.
Import all these files. These are the versions as of kernel commit
0cc53520e68be, with no local changes.
We use the same lib/kdoc/ directory as the kernel does here, so we
can avoid having to edit the top-level script just to adjust a
pathname, even though it is probably not the naming we would have
picked if this was a purely QEMU script.
The Sphinx conf.py still points at the Perl version of the script,
so this Python code will not be invoked to build the docs yet.
NB: checkpatch complains about many things in this commit,
including the use of "GPL-2.0" rather than "GPL-2.0-only" in
the SPDX tags, but since this is a third party import we can
ignore this.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Message-id: 20250814171324.1614516-4-peter.maydell@linaro.org
Newer versions of Perl (5.41.x and up) emit a warning for code in
kernel-doc:
Possible precedence problem between ! and pattern match (m//) at /scripts/kernel-doc line 1597.
This is because the code does:
if (!$param =~ /\w\.\.\.$/) {
In Perl, the ! operator has higher precedence than the =~
pattern-match binding, so the effect of this condition is to first
logically-negate the string $param into a true-or-false value and
then try to pattern match it against the regex, which in this case
will always fail. This is almost certainly not what the author
intended.
In the new Python version of kernel-doc in the Linux kernel,
the equivalent code is written:
if KernRe(r'\w\.\.\.$').search(param):
# For named variable parameters of the form `x...`,
# remove the dots
param = param[:-3]
else:
# Handles unnamed variable parameters
param = "..."
which is a more sensible way of writing the behaviour you would
get if you put in brackets to make the regex match first and
then negate the result.
Take this as the intended behaviour, and update the Perl to match.
For QEMU, this produces no change in output, presumably because we
never used the "unnamed variable parameters" syntax.
Cc: qemu-stable@nongnu.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Message-id: 20250819115648.2125709-1-peter.maydell@linaro.org
In commit bd0da3a3d4 we changed make-release so that instead of
cloning every git submodule of EDK2 we only cloned a fixed list.
The original motivation for this was that one of the submodules:
* was from a non-github repo
* that repo had a "SSL certificate expired" failure
* wasn't actually needed for the set of EDK2 binaries we build
and at the time we were trying to build the EDK2 binaries in one of
our CI jobs.
Unfortunately this change meant that we were exposed to bugs where
EDK2 adds a new submodule and the sources we ship in the release
tarball won't build any more. In particular, in EDK2 commit
c6bb7d54beb05 the MipiSysTLib submodule was added, causing failure of
the ROM build in our tarball starting from QEMU release 8.2.0:
/tmp/qemu-10.0.0/roms/edk2/MdePkg/MdePkg.dec(32): error 000E: File/directory not found in workspace
Library/MipiSysTLib/mipisyst/library/include is not found in packages path:
/tmp/qemu-10.0.0/roms/.
/tmp/qemu-10.0.0/roms/edk2
(Building from a QEMU git checkout works fine.)
In the intervening time EDK2 moved the submodule that had a problem
to be one they mirrored themselves (and at time of writing all their
submodules are hosted on github), and we stopped trying to build
EDK2 binaries in our own CI jobs with commit 690ceb7193.
Go back to cloning every EDK2 submodule, so we don't have an
untested explicit list of submodules which will break without
our noticing it.
This increases the size of the QEMU tarball .tar.xz file from
133M to 139M in my testing.
Cc: qemu-stable@nongnu.org
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/3041
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Michael Tokarev <mjt@tls.msk.ru>
Message-ID: <20250721153341.2910800-1-peter.maydell@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Some distros prefer to avoid vendored crate sources, and instead use
local sources from e.g. ``/usr/share/cargo/registry``. Add a
script, inspired by the Mesa spec file(*), that automatically
performs this task. The script is meant to be invoked after unpacking
the QEMU tarball.
(*) This is the hack that Mesa uses:
export MESON_PACKAGE_CACHE_DIR="%{cargo_registry}/"
%define inst_crate_nameversion() %(basename %{cargo_registry}/%{1}-*)
%define rewrite_wrap_file() sed -e "/source.*/d" -e "s/%{1}-.*/%{inst_crate_nameversion %{1}}/" -i subprojects/%{1}.wrap
%rewrite_wrap_file proc-macro2
... more %rewrite_wrap_file invocations follow ...
Reviewed-by: Neal Gompa <ngompa@fedoraproject.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Currently the tracing 'log' back emits special code to add timestamps
to trace points sent via qemu_log(). This current impl is a bad design
for a number of reasons.
* It changes the QEMU headers, such that 'error-report.h' content
is visible to all files using tracing, but only when the 'log'
backend is enabled. This has led to build failure bugs as devs
rarely test without the (default) 'log' backend enabled, and
CI can't cover every scenario for every trace backend.
* It bloats the trace points definitions which are inlined into
every probe location due to repeated inlining of timestamp
formatting code, adding MBs of overhead to QEMU.
* The tracing subsystem should not be treated any differently
from other users of qemu_log. They all would benefit from
having timestamps present.
* The timestamp emitted with the tracepoints is in a needlessly
different format to that used by error_report() in response
to '-msg timestamp=on'.
This fixes all these issues simply by moving timestamp formatting
into qemu_log, using the same approach as for error_report.
The code before:
static inline void _nocheck__trace_qcrypto_tls_creds_get_path(void * creds, const char * filename, const char * path)
{
if (trace_event_get_state(TRACE_QCRYPTO_TLS_CREDS_GET_PATH) && qemu_loglevel_mask(LOG_TRACE)) {
if (message_with_timestamp) {
struct timeval _now;
gettimeofday(&_now, NULL);
qemu_log("%d@%zu.%06zu:qcrypto_tls_creds_get_path " "TLS creds path creds=%p filename=%s path=%s" "\n",
qemu_get_thread_id(),
(size_t)_now.tv_sec, (size_t)_now.tv_usec
, creds, filename, path);
} else {
qemu_log("qcrypto_tls_creds_get_path " "TLS creds path creds=%p filename=%s path=%s" "\n", creds, filename, path);
}
}
}
and after:
static inline void _nocheck__trace_qcrypto_tls_creds_get_path(void * creds, const char * filename, const char * path)
{
if (trace_event_get_state(TRACE_QCRYPTO_TLS_CREDS_GET_PATH) && qemu_loglevel_mask(LOG_TRACE)) {
qemu_log("qcrypto_tls_creds_get_path " "TLS creds path creds=%p filename=%s path=%s" "\n", creds, filename, path);
}
}
The log and error messages before:
$ qemu-system-x86_64 -trace qcrypto* -object tls-creds-x509,id=tls0,dir=$HOME/tls -msg timestamp=on
2986097@1753122905.917608:qcrypto_tls_creds_x509_load TLS creds x509 load creds=0x55d925bd9490 dir=/var/home/berrange/tls
2986097@1753122905.917621:qcrypto_tls_creds_get_path TLS creds path creds=0x55d925bd9490 filename=ca-cert.pem path=<none>
2025-07-21T18:35:05.917626Z qemu-system-x86_64: Unable to access credentials /var/home/berrange/tls/ca-cert.pem: No such file or directory
and after:
$ qemu-system-x86_64 -trace qcrypto* -object tls-creds-x509,id=tls0,dir=$HOME/tls -msg timestamp=on
2025-07-21T18:43:28.089797Z qcrypto_tls_creds_x509_load TLS creds x509 load creds=0x55bf5bf12380 dir=/var/home/berrange/tls
2025-07-21T18:43:28.089815Z qcrypto_tls_creds_get_path TLS creds path creds=0x55bf5bf12380 filename=ca-cert.pem path=<none>
2025-07-21T18:43:28.089819Z qemu-system-x86_64: Unable to access credentials /var/home/berrange/tls/ca-cert.pem: No such file or directory
The binary size before:
$ ls -alh qemu-system-x86_64
-rwxr-xr-x. 1 berrange berrange 87M Jul 21 19:39 qemu-system-x86_64
$ strip qemu-system-x86_64
$ ls -alh qemu-system-x86_64
-rwxr-xr-x. 1 berrange berrange 30M Jul 21 19:39 qemu-system-x86_64
and after:
$ ls -alh qemu-system-x86_64
-rwxr-xr-x. 1 berrange berrange 85M Jul 21 19:41 qemu-system-x86_64
$ strip qemu-system-x86_64
$ ls -alh qemu-system-x86_64
-rwxr-xr-x. 1 berrange berrange 29M Jul 21 19:41 qemu-system-x86_64
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Message-id: 20250721185452.3016488-1-berrange@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
When TLS 1.3 is negotiated on a TLS session, GNUTLS will perform
automatic rekeying of the session after 16 million records. This
is done for all algorithms except CHACHA20_POLY1305 which does
not require rekeying.
Unfortunately the rekeying breaks GNUTLS' promise that it is safe
to use a gnutls_session_t object concurrently from multiple threads
if they are exclusively calling gnutls_record_send/recv.
This patch implements a workaround for QEMU that adds a mutex lock
around any gnutls_record_send/recv call to serialize execution
within GNUTLS code. When GNUTLS calls into the push/pull functions
we can release the lock so the OS level I/O calls can at least
have some parallelism.
The big downside of this is that the actual encryption/decryption
code is fully serialized, which will halve performance of that
cipher operations if two threads are contending.
The workaround is not enabled by default, since most use of GNUTLS
in QEMU does not tickle the problem, only non-multifd migration
with a return path open is affected. Fortunately the migration
code also won't trigger the halving of performance, since only
the outbound channel diretion needs to sustain high data rates,
the inbound direction is low volume.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Fabiano Rosas <farosas@suse.de>
Link: https://lore.kernel.org/qemu-devel/20250718150514.2635338-2-berrange@redhat.com
[add stub for qcrypto_tls_session_require_thread_safety; fix unused var]
Signed-off-by: Fabiano Rosas <farosas@suse.de>
* rust: miscellaneous fixes
* rust: qemu-api-macros: cleanup and add unit tests for TryInto
* rust: log: implement io::Write, avoid memory allocations
when logging constant strings
* target/i386: fix usage of properties whenever accelerators
change the default (e.g. vendor)
* target/i386: add support for TDVMCALL_SETUP_EVENT_NOTIFY_INTERRUPT
* target/i386: add support for booting an SEV VM from an IGVM file
* target/i386: unify cache model descriptions between CPUID 2,
CPUID 4 and AMD specific CPUID 0x80000006
* target/i386: introduce cache models for recent Intel CPU models
* target/i386: mark some 0x80000000-0x80000008 bits as reserved on Intel
* target/i386: cleanups
# -----BEGIN PGP SIGNATURE-----
#
# iQFIBAABCgAyFiEE8TM4V0tmI4mGbHaCv/vSX3jHroMFAmh0v+sUHHBib256aW5p
# QHJlZGhhdC5jb20ACgkQv/vSX3jHroOQUQf8CTsCnl2xYrnrkVfSVj6kuAE+JYD6
# oLSXsOEG4yrVknuhwIfVsqNScmleJCdz85ej7CZxy3vzzgjLfmy7nwifKEIKku7E
# XO/Q3HbB898MnzqceQRmwe1AzELoj1Lave215CPhUBo60LCRPwaIZsiHprnNZgXi
# TyHlmywDVRjyFLtKkx3El0dnLAhFqPWeGh81CD5lPLZZJ+Wt2FuAw2zqSOGB2ztM
# FkJmunFJiaTItjyCN/uNvBSbDKecAHgCXvSCVNG3+I4U2R0gK1lcwm3TRo7yKia+
# HUHGa3UEXoIqlRfXdX6zuc8tW1/u6SPv+8WX53t204PAeSWDUrtIe9jZ4A==
# =y4/a
# -----END PGP SIGNATURE-----
# gpg: Signature made Mon 14 Jul 2025 04:29:31 EDT
# gpg: using RSA key F13338574B662389866C7682BFFBD25F78C7AE83
# gpg: issuer "pbonzini@redhat.com"
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" [full]
# gpg: aka "Paolo Bonzini <pbonzini@redhat.com>" [full]
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4 E2F7 7E15 100C CD36 69B1
# Subkey fingerprint: F133 3857 4B66 2389 866C 7682 BFFB D25F 78C7 AE83
* tag 'for-upstream' of https://gitlab.com/bonzini/qemu: (77 commits)
i386/cpu: Honor maximum value for CPUID.8000001DH.EAX[25:14]
i386/cpu: Fix overflow of cache topology fields in CPUID.04H
i386/cpu: Fix cpu number overflow in CPUID.01H.EBX[23:16]
i386/cpu: Fix number of addressable IDs field for CPUID.01H.EBX[23:16]
i386/cpu: Reorder CPUID leaves in cpu_x86_cpuid()
tests/vm: bump FreeBSD image to 14.3
tests/functional: test_x86_cpu_model_versions: remove dead tests
i386/cpu: Mark CPUID 0x80000008 ECX bits[0:7] & [12:15] as reserved for Intel/Zhaoxin
i386/cpu: Mark CPUID 0x80000007[EBX] as reserved for Intel
i386/cpu: Mark EBX/ECX/EDX in CPUID 0x80000000 leaf as reserved for Intel
i386/cpu: Enable 0x1f leaf for YongFeng by default
i386/cpu: Enable 0x1f leaf for SapphireRapids by default
i386/cpu: Enable 0x1f leaf for GraniteRapids by default
i386/cpu: Enable 0x1f leaf for SierraForest by default
i386/cpu: Enable 0x1f leaf for SierraForest by default
i386/cpu: Add a "x-force-cpuid-0x1f" property
i386/cpu: Introduce cache model for YongFeng
i386/cpu: Introduce cache model for SapphireRapids
i386/cpu: Introduce cache model for GraniteRapids
i386/cpu: Introduce cache model for SierraForest
...
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Generated command documentation lacks information on return value in
several cases, e.g. query-tpm.
The obvious fix would be to require a Returns: section when a command
returns something.
However, note that many existing Returns: sections are pretty useless:
the description is basically the return type, which then gets rendered
like "Return: <Type> – <basically the return type>". This suggests
that a description is often not really necessary, and requiring one
isn't useful.
Instead, generate the obvious minimal thing when Returns: is absent:
"Return: <Type>".
This auto-generated Return documentation is placed is as follows:
1. If we have arguments, return goes right after them.
2. Else if we have errors, return goes right before them.
3. Else if we have features, return goes right before them.
4. Else return goes right after the intro
To facilitate this algorithm, a "TODO:" hack line is used to separate
the intro from the remainder of the documentation block in cases where
there are no other sections to separate the intro from e.g. examples and
additional detail meant to appear below the key sections of interest.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250711051045.51110-3-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[_insert_near_kind() code replaced by something simpler, commit
message amended to explain why we're doing this]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Peter Maydell
Philippe Mathieu-Daudé
Manos PitsidianakisandPaolo Bonzini
Stefan Hajnoczi
Daniel P. BerrangéandFabiano Rosas
Richard HendersonandPeter Maydell
Tanish DesaiandStefan Hajnoczi
Thomas Huth
Stefano GarzarellaandPaolo Bonzini
John SnowandMarkus Armbruster