The current logic scans qemu.git/subprojects/ from *.wrap files to
determine whether or not to include the associated directories in the
release tarballs. However, the script assumes that it is being run from
the top-level of the source directory, which may not always be the case.
In particular, when generating releases via, e.g.:
make qemu-9.2.1.tar.xz
the $CWD will either be an arbitrary external build directory, or
qemu.git/build, and the exclusions will not be processed as expected.
Fix this by using the $src parameter passed to the script as the root
directory for the various subproject/ paths referenced by this logic.
Also, the error case at the beginning of the subproject_dir() will not
result in the error message being printed, and will instead produce an
error message about "error" not being a valid command. Fix this by using
basic shell commands.
Fixes: be27b5149c ("make-release: only leave tarball of wrap-file subprojects")
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Michael Tokarev <mjt@tls.msk.ru>
Cc: qemu-stable@nongnu.org
Signed-off-by: Michael Roth <michael.roth@amd.com>
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
If the 'stap' binary is missing in $PATH, a huge trace is thrown
$ qemu-trace-stap list /usr/bin/qemu-system-x86_64
Traceback (most recent call last):
File "/usr/bin/qemu-trace-stap", line 169, in <module>
main()
File "/usr/bin/qemu-trace-stap", line 165, in main
args.func(args)
File "/usr/bin/qemu-trace-stap", line 83, in cmd_run
subprocess.call(stapargs)
File "/usr/lib64/python3.12/subprocess.py", line 389, in call
with Popen(*popenargs, **kwargs) as p:
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.12/subprocess.py", line 1026, in {}init{}
self._execute_child(args, executable, preexec_fn, close_fds,
File "/usr/lib64/python3.12/subprocess.py", line 1955, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'stap'
With this change the user now gets
$ qemu-trace-stap list /usr/bin/qemu-system-x86_64
Unable to find 'stap' in $PATH
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-id: 20241206114524.1666664-1-berrange@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
This replaces use of the constants from the QapiSpecialFeatures
enum, with constants from the auto-generate QapiFeatures enum
in qapi-features.h
The 'deprecated' and 'unstable' features still have a little bit of
special handling, being force defined to be the 1st + 2nd features
in the enum, regardless of whether they're used in the schema. This
retains compatibility with common code that references the features
via the QapiSpecialFeatures constants.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20250205123550.2754387-5-berrange@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Imports tidied up with isort]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
This updates the QAPI code generation to refer to 'features' instead
of 'special_features', in preparation for generalizing their exposure.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20250205123550.2754387-4-berrange@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Imports tidied up with isort]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
The "special_features" field / parameter holds the subset of schema
features that are for internal code use. Specifically 'DEPRECATED'
and 'UNSTABLE'.
This special casing of internal features is going to be removed, so
prepare for that by renaming to 'features'. Using a fixed size type
is also best practice for bit fields.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20250205123550.2754387-3-berrange@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
The general expectation is that header files should follow the same
file/path naming scheme as the corresponding source file. There are
various historical exceptions to this practice in QEMU, with one of
the most notable being the include/qapi/qmp/ directory.
include/qapi/qmp/dispatch.h corresponds mostly to qapi/qmp-registry.c.
Move and rename it to include/qapi/qmp-registry.h.
Now just qerror.h is left in include/qapi/qmp/. Since it's deprecated
& (slowly) getting eliminated anyway, it isn't worth moving.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Zhao Liu <zhao1.liu@intel.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20241118151235.2665921-3-armbru@redhat.com>
The general expectation is that header files should follow the same
file/path naming scheme as the corresponding source file. There are
various historical exceptions to this practice in QEMU, with one of
the most notable being the include/qapi/qmp/ directory. Most of the
headers there correspond to source files in qobject/.
This patch corrects most of that inconsistency by creating
include/qobject/ and moving the headers for qobject/ there.
This also fixes MAINTAINERS for include/qapi/qmp/dispatch.h:
scripts/get_maintainer.pl now reports "QAPI" instead of "No
maintainers found".
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Zhao Liu <zhao1.liu@intel.com>
Acked-by: Halil Pasic <pasic@linux.ibm.com> #s390x
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20241118151235.2665921-2-armbru@redhat.com>
[Rebased]
Dumping coroutines don't yet work with coredumps. Let's make it work.
We still kept most of the old code because they can be either more
flexible, or prettier. Only add the fallbacks when they stop working.
Currently the raw unwind is pretty ugly, but it works, like this:
(gdb) qemu bt
#0 process_incoming_migration_co (opaque=0x0) at ../migration/migration.c:788
#1 0x000055ae6c0dc4d9 in coroutine_trampoline (i0=-1711718576, i1=21934) at ../util/coroutine-ucontext.c:175
#2 0x00007f9f59d72f40 in ??? () at /lib64/libc.so.6
#3 0x00007ffd549214a0 in ??? ()
#4 0x0000000000000000 in ??? ()
Coroutine at 0x7f9f4c57c748:
#0 0x55ae6c0dc9a8 in qemu_coroutine_switch<+120> () at ../util/coroutine-ucontext.c:321
#1 0x55ae6c0da2f8 in qemu_aio_coroutine_enter<+356> () at ../util/qemu-coroutine.c:293
#2 0x55ae6c0da3f1 in qemu_coroutine_enter<+34> () at ../util/qemu-coroutine.c:316
#3 0x55ae6baf775e in migration_incoming_process<+43> () at ../migration/migration.c:876
#4 0x55ae6baf7ab4 in migration_ioc_process_incoming<+490> () at ../migration/migration.c:1008
#5 0x55ae6bae9ae7 in migration_channel_process_incoming<+145> () at ../migration/channel.c:45
#6 0x55ae6bb18e35 in socket_accept_incoming_migration<+118> () at ../migration/socket.c:132
#7 0x55ae6be939ef in qio_net_listener_channel_func<+131> () at ../io/net-listener.c:54
#8 0x55ae6be8ce1a in qio_channel_fd_source_dispatch<+78> () at ../io/channel-watch.c:84
#9 0x7f9f5b26728c in g_main_context_dispatch_unlocked.lto_priv<+315> ()
#10 0x7f9f5b267555 in g_main_context_dispatch<+36> ()
#11 0x55ae6c0d91a7 in glib_pollfds_poll<+90> () at ../util/main-loop.c:287
#12 0x55ae6c0d9235 in os_host_main_loop_wait<+128> () at ../util/main-loop.c:310
#13 0x55ae6c0d9364 in main_loop_wait<+203> () at ../util/main-loop.c:589
#14 0x55ae6bac212a in qemu_main_loop<+41> () at ../system/runstate.c:835
#15 0x55ae6bfdf522 in qemu_default_main<+19> () at ../system/main.c:37
#16 0x55ae6bfdf55f in main<+40> () at ../system/main.c:48
#17 0x7f9f59d42248 in __libc_start_call_main<+119> ()
#18 0x7f9f59d4230b in __libc_start_main_impl<+138> ()
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-ID: <20241212204801.1420528-4-peterx@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
There're a bunch of code trying to fetch fs_base in different ways. IIUC
the simplest way instead is "$fs_base". It also has the benefit that it'll
work for both live gdb session or coredumps.
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-ID: <20241212204801.1420528-3-peterx@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
semihosting, plugin and doc updates:
- log a guest_error for failed semihosting open()
- clean up semihosting includes to reduce build duplication
- re-factor misc device initialisation to fail with &error_exit
- propagate Error * to gdbserver_start sub-functions
- fix 32-bit build of plugins and re-enable by default
- ensure IRQs don't preempt io recompiled instructions
- remove usage of gcc_struct to enable clang builds
- enable clang/lld to build plugins on windows
- various small kdoc typo fixes
- add perl scripts to editorconfig
- remove unused field from MemoryRegion
- make kdoc script a dependency so doc rebuilds get triggered
- expand developer documentation:
- notes on git-publish
- describe usage of b4
- setting up build dependencies
- codebase layout
- add a glossary of common terms
- optimise the windows ndis script
# -----BEGIN PGP SIGNATURE-----
#
# iQEzBAABCgAdFiEEZoWumedRZ7yvyN81+9DbCVqeKkQFAmeKO8sACgkQ+9DbCVqe
# KkTbBQf9HRlspCl2r5a8K9O1ymylKiZ653OBWMStGTQ8xPXeLDFhT+ION34VBgBh
# LXHEcjIHn24cN2s1BO5+xJs0nzqYe7UEAK6JQmdX3/HEuf8VmaVslvhm+nCWKoIL
# JQbsHno92wh6vvTWQu53zijEuG5HdBseWiwQKHbE1oSRc2CikG70o902AL9zXAsp
# mpUYWxUmWwg5uQATztp4XfylJBcSX3SiVgv22jXLqBj9drXPftl/E33fcWXxTj5f
# AM3kz9fxaCfo5+znmYw3R1tT/Hv52Q6hW+oNAm34XeWp1/+ho27QMRrpIi/dpdwX
# mCbvJwI75sCnD91p9NW7vZIYZJKsLg==
# =SLCY
# -----END PGP SIGNATURE-----
# gpg: Signature made Fri 17 Jan 2025 06:15:23 EST
# gpg: using RSA key 6685AE99E75167BCAFC8DF35FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" [unknown]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg: There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8 DF35 FBD0 DB09 5A9E 2A44
* tag 'pull-10.0-gdb-plugins-doc-updates-170125-1' of https://gitlab.com/stsquad/qemu: (37 commits)
scripts/nsis.py: Run dependency check for each DLL file only once
docs: add a glossary
docs/devel: add a codebase section
docs/devel: add information on how to setup build environments
docs/devel: add b4 for patch retrieval
docs/devel: add git-publish for patch submitting
docs/sphinx: include kernel-doc script as a dependency
include/exec: remove warning_printed from MemoryRegion
include/exec: fix some copy and paste errors in kdoc
tests/qtest: fix some copy and paste errors in kdoc
editorconfig: update for perl scripts
plugins: fix kdoc annotation
plugins: enable linking with clang/lld
docs/devel/style: add a section about bitfield, and disallow them for packed structures
win32: remove usage of attribute gcc_struct
accel/tcg: also suppress asynchronous IRQs for cpu_io_recompile
configure: reenable plugins by default for 32-bit hosts
contrib/plugins/hotpages: fix 32-bit build
contrib/plugins/hwprofile: fix 32-bit build
contrib/plugins/cflow: fix 32-bit build
...
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
* rust: miscellaneous changes
* target/i386: small code generation improvements
* target/i386: various cleanups and fixes
* cpu: remove env->nr_cores
# -----BEGIN PGP SIGNATURE-----
#
# iQFIBAABCAAyFiEE8TM4V0tmI4mGbHaCv/vSX3jHroMFAmeBoIgUHHBib256aW5p
# QHJlZGhhdC5jb20ACgkQv/vSX3jHroOD2gf+NK7U1EhNIrsbBsbtu2i7+tnbRKIB
# MTu+Mxb2wz4C7//pxq+vva4bgT3iOuL9RF19PRe/63CMD65xMiwyyNrEWX2HbRIJ
# 5dytLLLdef3yMhHh2x1uZfm54g12Ppvn9kulMCbPawrlqWgg1sZbkUBrRtFzS45c
# NeYjGWWSpBDe7LtsrgSRYLPnz6wWEiy3tDpu2VoDtjrE86UVDXwyzpbtBk9Y8jPi
# CKdvLyQeO9xDE5OoXMjJMlJeQq3D9iwYEprXUqy+RUZtpW7YmqMCf2JQ4dAjVCad
# 07v/kITF4brGCVnzDcDA6W7LqHpBu1w+Hn23yLw3HEDDBt11o9JjQCl9qA==
# =xIQ4
# -----END PGP SIGNATURE-----
# gpg: Signature made Fri 10 Jan 2025 17:34:48 EST
# gpg: using RSA key F13338574B662389866C7682BFFBD25F78C7AE83
# gpg: issuer "pbonzini@redhat.com"
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" [full]
# gpg: aka "Paolo Bonzini <pbonzini@redhat.com>" [full]
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4 E2F7 7E15 100C CD36 69B1
# Subkey fingerprint: F133 3857 4B66 2389 866C 7682 BFFB D25F 78C7 AE83
* tag 'for-upstream' of https://gitlab.com/bonzini/qemu: (38 commits)
i386/cpu: Set and track CPUID_EXT3_CMP_LEG in env->features[FEAT_8000_0001_ECX]
i386/cpu: Set up CPUID_HT in x86_cpu_expand_features() instead of cpu_x86_cpuid()
cpu: Remove nr_cores from struct CPUState
i386/cpu: Hoist check of CPUID_EXT3_TOPOEXT against threads_per_core
i386/cpu: Track a X86CPUTopoInfo directly in CPUX86State
i386/topology: Introduce helpers for various topology info of different level
i386/topology: Update the comment of x86_apicid_from_topo_ids()
i386/cpu: Drop cores_per_pkg in cpu_x86_cpuid()
i386/cpu: Drop the variable smp_cores and smp_threads in x86_cpu_pre_plug()
i386/cpu: Extract a common fucntion to setup value of MSR_CORE_THREAD_COUNT
target/i386/kvm: Replace ARRAY_SIZE(msr_handlers) with KVM_MSR_FILTER_MAX_RANGES
target/i386/kvm: Clean up error handling in kvm_arch_init()
target/i386/kvm: Return -1 when kvm_msr_energy_thread_init() fails
target/i386/kvm: Clean up return values of MSR filter related functions
target/i386/confidential-guest: Fix comment of x86_confidential_guest_kvm_type()
target/i386/kvm: Drop workaround for KVM_X86_DISABLE_EXITS_HTL typo
target/i386/kvm: Only save/load kvmclock MSRs when kvmclock enabled
target/i386/kvm: Remove local MSR_KVM_WALL_CLOCK and MSR_KVM_SYSTEM_TIME definitions
target/i386/kvm: Add feature bit definitions for KVM CPUID
i386/cpu: Mark avx10_version filtered when prefix is NULL
...
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
The vmclock device addresses the problem of live migration with
precision clocks. The tolerances of a hardware counter (e.g. TSC) are
typically around ±50PPM. A guest will use NTP/PTP/PPS to discipline that
counter against an external source of 'real' time, and track the precise
frequency of the counter as it changes with environmental conditions.
When a guest is live migrated, anything it knows about the frequency of
the underlying counter becomes invalid. It may move from a host where
the counter running at -50PPM of its nominal frequency, to a host where
it runs at +50PPM. There will also be a step change in the value of the
counter, as the correctness of its absolute value at migration is
limited by the accuracy of the source and destination host's time
synchronization.
The device exposes a shared memory region to guests, which can be mapped
all the way to userspace. In the first phase, this merely advertises a
'disruption_marker', which indicates that the guest should throw away any
NTP synchronization it thinks it has, and start again.
Because the region can be exposed all the way to userspace, applications
can still use time from a fast vDSO 'system call', and check the
disruption marker to be sure that their timestamp is indeed truthful.
The structure also allows for the precise time, as known by the host, to
be exposed directly to guests so that they don't have to wait for NTP to
resync from scratch.
The values and fields are based on the nascent virtio-rtc specification,
and the intent is that a version (hopefully precisely this version) of
this structure will be included as an optional part of that spec. In the
meantime, a simple ACPI device along the lines of VMGENID is perfectly
sufficient and is compatible with what's being shipped in certain
commercial hypervisors.
Linux guest support was merged into the 6.13-rc1 kernel:
https://git.kernel.org/torvalds/c/205032724226
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Reviewed-by: Paul Durrant <paul@xen.org>
Message-Id: <07fd5e2f529098ad4d7cab1423fe9f4a03a9cc14.camel@infradead.org>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
The QEMU source archive is including the sources downloaded from crates.io
in both tarball form (in subprojects/packagecache) and expanded/patched
form (in the subprojects directory). The former is the more authoritative
form, as it has a hash that can be verified in the wrap file and checked
against the download URL, so keep that one only. This works also with
--disable-download; when building QEMU for the first time from the
tarball, Meson will print something like
Using proc-macro2-1-rs source from cache.
for each subproject, and then go on to extract the tarball and apply the
overlay or the patches in subprojects/packagefiles.
Reported-by: Michael Tokarev <mjt@tls.msk.ru>
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/2719
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
rustc will check that every reachable #[cfg] matches a list of
the expected config names and values. Recent versions of rustc are
also complaining about #[cfg(test)], even if it is basically a standard
part of the language. So, always allow it.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Currently, if an array of pointers contains a NULL pointer, that
pointer will be encoded as '0' in the stream. Since the JSON writer
doesn't define a "pointer" type, that '0' will now be an uint8, which
is different from the original type being pointed to, e.g. struct.
(we're further calling uint8 "nullptr", but that's irrelevant to the
issue)
That mixed-type array shouldn't be compressed, otherwise data is lost
as the code currently makes the whole array have the type of the first
element:
css = {NULL, NULL, ..., 0x5555568a7940, NULL};
{"name": "s390_css", "instance_id": 0, "vmsd_name": "s390_css",
"version": 1, "fields": [
...,
{"name": "css", "array_len": 256, "type": "nullptr", "size": 1},
...,
]}
In the above, the valid pointer at position 254 got lost among the
compressed array of nullptr.
While we could disable the array compression when a NULL pointer is
found, the JSON part of the stream still makes part of downtime, so we
should avoid writing unecessary bytes to it.
Keep the array compression in place, but if NULL and non-NULL pointers
are mixed break the array into several type-contiguous pieces :
css = {NULL, NULL, ..., 0x5555568a7940, NULL};
{"name": "s390_css", "instance_id": 0, "vmsd_name": "s390_css",
"version": 1, "fields": [
...,
{"name": "css", "array_len": 254, "type": "nullptr", "size": 1},
{"name": "css", "type": "struct", "struct": {"vmsd_name": "s390_css_img", ... }, "size": 768},
{"name": "css", "type": "nullptr", "size": 1},
...,
]}
Now each type-discontiguous region will become a new JSON entry. The
reader should interpret this as a concatenation of values, all part of
the same field.
Parsing the JSON with analyze-script.py now shows the proper data
being pointed to at the places where the pointer is valid and
"nullptr" where there's NULL:
"s390_css (14)": {
...
"css": [
"nullptr",
"nullptr",
...
"nullptr",
{
"chpids": [
{
"in_use": "0x00",
"type": "0x00",
"is_virtual": "0x00"
},
...
]
},
"nullptr",
}
Reviewed-by: Peter Xu <peterx@redhat.com>
Message-Id: <20250109185249.23952-7-farosas@suse.de>
Signed-off-by: Fabiano Rosas <farosas@suse.de>
Rename vmstate_info_nullptr from "uint64_t" to "nullptr". This vmstate
actually reads and writes just a byte, so the proper name would be
uint8. However, since this is a marker for a NULL pointer, it's
convenient to have a more explicit name that can be identified by the
consumers of the JSON part of the stream.
Change the name to "nullptr" and add support for it in the
analyze-migration.py script. Arbitrarily use the name of the type as
the value of the field to avoid the script showing 0x30 or '0', which
could be confusing for readers.
Reviewed-by: Peter Xu <peterx@redhat.com>
Message-Id: <20250109185249.23952-5-farosas@suse.de>
Signed-off-by: Fabiano Rosas <farosas@suse.de>
The parsing for the S390StorageAttributes section is currently leaving
an unconsumed token that is later interpreted by the generic code as
QEMU_VM_EOF, cutting the parsing short.
The migration will issue a STATTR_FLAG_DONE between iterations, which
the script consumes correctly, but there's a final STATTR_FLAG_EOS at
.save_complete that the script is ignoring. Since the EOS flag is a
u64 0x1ULL and the stream is big endian, on little endian hosts a byte
read from it will be 0x0, the same as QEMU_VM_EOF.
Fixes: 81c2c9dd5d ("tests/qtest/migration-test: Fix analyze-migration.py for s390x")
Reviewed-by: Peter Xu <peterx@redhat.com>
Message-Id: <20250109185249.23952-4-farosas@suse.de>
Signed-off-by: Fabiano Rosas <farosas@suse.de>
The analyze-migration script was seen failing in s390x in misterious
ways. It seems we're reaching the VMSDFieldStruct constructor without
any fields, which would indicate an empty .subsection entry, a
VMSTATE_STRUCT with no fields or a vmsd with no fields. We don't have
any of those, at least not without the unmigratable flag set, so this
should never happen.
Add some debug statements so that we can see what's going on the next
time the issue happens.
Reviewed-by: Peter Xu <peterx@redhat.com>
Message-Id: <20250109185249.23952-2-farosas@suse.de>
Signed-off-by: Fabiano Rosas <farosas@suse.de>
