Currently, if an array of pointers contains a NULL pointer, that
pointer will be encoded as '0' in the stream. Since the JSON writer
doesn't define a "pointer" type, that '0' will now be an uint8, which
is different from the original type being pointed to, e.g. struct.
(we're further calling uint8 "nullptr", but that's irrelevant to the
issue)
That mixed-type array shouldn't be compressed, otherwise data is lost
as the code currently makes the whole array have the type of the first
element:
css = {NULL, NULL, ..., 0x5555568a7940, NULL};
{"name": "s390_css", "instance_id": 0, "vmsd_name": "s390_css",
"version": 1, "fields": [
...,
{"name": "css", "array_len": 256, "type": "nullptr", "size": 1},
...,
]}
In the above, the valid pointer at position 254 got lost among the
compressed array of nullptr.
While we could disable the array compression when a NULL pointer is
found, the JSON part of the stream still makes part of downtime, so we
should avoid writing unecessary bytes to it.
Keep the array compression in place, but if NULL and non-NULL pointers
are mixed break the array into several type-contiguous pieces :
css = {NULL, NULL, ..., 0x5555568a7940, NULL};
{"name": "s390_css", "instance_id": 0, "vmsd_name": "s390_css",
"version": 1, "fields": [
...,
{"name": "css", "array_len": 254, "type": "nullptr", "size": 1},
{"name": "css", "type": "struct", "struct": {"vmsd_name": "s390_css_img", ... }, "size": 768},
{"name": "css", "type": "nullptr", "size": 1},
...,
]}
Now each type-discontiguous region will become a new JSON entry. The
reader should interpret this as a concatenation of values, all part of
the same field.
Parsing the JSON with analyze-script.py now shows the proper data
being pointed to at the places where the pointer is valid and
"nullptr" where there's NULL:
"s390_css (14)": {
...
"css": [
"nullptr",
"nullptr",
...
"nullptr",
{
"chpids": [
{
"in_use": "0x00",
"type": "0x00",
"is_virtual": "0x00"
},
...
]
},
"nullptr",
}
Reviewed-by: Peter Xu <peterx@redhat.com>
Message-Id: <20250109185249.23952-7-farosas@suse.de>
Signed-off-by: Fabiano Rosas <farosas@suse.de>
Rename vmstate_info_nullptr from "uint64_t" to "nullptr". This vmstate
actually reads and writes just a byte, so the proper name would be
uint8. However, since this is a marker for a NULL pointer, it's
convenient to have a more explicit name that can be identified by the
consumers of the JSON part of the stream.
Change the name to "nullptr" and add support for it in the
analyze-migration.py script. Arbitrarily use the name of the type as
the value of the field to avoid the script showing 0x30 or '0', which
could be confusing for readers.
Reviewed-by: Peter Xu <peterx@redhat.com>
Message-Id: <20250109185249.23952-5-farosas@suse.de>
Signed-off-by: Fabiano Rosas <farosas@suse.de>
The parsing for the S390StorageAttributes section is currently leaving
an unconsumed token that is later interpreted by the generic code as
QEMU_VM_EOF, cutting the parsing short.
The migration will issue a STATTR_FLAG_DONE between iterations, which
the script consumes correctly, but there's a final STATTR_FLAG_EOS at
.save_complete that the script is ignoring. Since the EOS flag is a
u64 0x1ULL and the stream is big endian, on little endian hosts a byte
read from it will be 0x0, the same as QEMU_VM_EOF.
Fixes: 81c2c9dd5d ("tests/qtest/migration-test: Fix analyze-migration.py for s390x")
Reviewed-by: Peter Xu <peterx@redhat.com>
Message-Id: <20250109185249.23952-4-farosas@suse.de>
Signed-off-by: Fabiano Rosas <farosas@suse.de>
The analyze-migration script was seen failing in s390x in misterious
ways. It seems we're reaching the VMSDFieldStruct constructor without
any fields, which would indicate an empty .subsection entry, a
VMSTATE_STRUCT with no fields or a vmsd with no fields. We don't have
any of those, at least not without the unmigratable flag set, so this
should never happen.
Add some debug statements so that we can see what's going on the next
time the issue happens.
Reviewed-by: Peter Xu <peterx@redhat.com>
Message-Id: <20250109185249.23952-2-farosas@suse.de>
Signed-off-by: Fabiano Rosas <farosas@suse.de>
Headers in include/sysemu/ are not only related to system
*emulation*, they are also used by virtualization. Rename
as system/ which is clearer.
Files renamed manually then mechanical change using sed tool.
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Tested-by: Lei Yang <leiyang@redhat.com>
Message-Id: <20241203172445.28576-1-philmd@linaro.org>
Qtest pull request
- TIMEOUT_MULTIPLIER setting to allow tests to take longer when asan is enabled
- New qtest_system_reset() wrapper to properly wait for a system reset
- Split of migration-test.c into multiple files under qtest/migration/
# -----BEGIN PGP SIGNATURE-----
#
# iQJEBAABCAAuFiEEqhtIsKIjJqWkw2TPx5jcdBvsMZ0FAmda+84QHGZhcm9zYXNA
# c3VzZS5kZQAKCRDHmNx0G+wxnXWZEACcxW/P9EiLxIeCf4ke7/qq+8HSOOLdQrEa
# hP5ZCzLlY0K8KQ9fhTGvc6UgRBsScXcLuj4eoR8Gke9e4tID39XVdkion/x6OCiv
# grC2Ll9/BNrWtR8DiXs3krsU1CXhav6qcT56bMiXV3Qo+KsTmB7RkpXpbkZLZxkn
# P2aqZlYFS0zZ3PZQceCasQEzB6Dls7o1wNXvgSJXr2E4IuIEcbyWtQpr5DfctnfN
# STbqGTgMXADyjlqh3CWwSLsZQvIVoN9+3twOrZn4j0D+fyEsTjKsZPlUpRtr72ap
# FVT+zVIUfxieBlDTsRSwcieD1ePFUUjl7ikQFwypGs33TI9VDTsB88IG7h0/080o
# 7dYbSMSG0pGEq9QIB8jkV26OZi3y00L/zmLMNDsN6nxKN4mSySlfVD/B/Jb70Smy
# lwbLklnr921kqsF89Bsq8YRyg8bbGXtx2ifGROgqF+JtrxRW0ElF9K/h4HQtPjuD
# nZJ4rtrDIv1iMTLJwbNu39fhxvuQowdBYTrbPMgw+xIv++LodqYJSDxbEX+FaDqI
# LMf772jiAgesl7qe1d9TJizhvonyi9PIdcUILg0SH0H1KvoSjpS3mlGgvo8zgvh+
# zvhhzkFGDIHmlCvOaIYyCpadJ5SMMS627q/Q13X3DHjUTkpPUhxGgc9oO7T39+iA
# 4dOLDRjUcg==
# =H6xv
# -----END PGP SIGNATURE-----
# gpg: Signature made Thu 12 Dec 2024 10:05:50 EST
# gpg: using RSA key AA1B48B0A22326A5A4C364CFC798DC741BEC319D
# gpg: issuer "farosas@suse.de"
# gpg: Good signature from "Fabiano Rosas <farosas@suse.de>" [unknown]
# gpg: aka "Fabiano Almeida Rosas <fabiano.rosas@suse.com>" [unknown]
# gpg: WARNING: The key's User ID is not certified with a trusted signature!
# gpg: There is no indication that the signature belongs to the owner.
# Primary key fingerprint: AA1B 48B0 A223 26A5 A4C3 64CF C798 DC74 1BEC 319D
* tag 'qtest-20241212-pull-request' of https://gitlab.com/farosas/qemu: (22 commits)
tests/qtest/migration: Split validation tests + misc
tests/qtest/migration-test: Fix and enable test_ignore_shared
tests/qtest/migration: Split CPR tests
tests/qtest/migration: Split precopy tests
tests/qtest/migration: Split file tests
tests/qtest/migration: Split postcopy tests
tests/qtest/migration: Split compression tests from migration-test.c
tests/qtest/migration: Split TLS tests from migration-test.c
tests/qtest/migration: Move common test code
tests/qtest/migration: Isolate test initialization
tests/qtest/migration: Move kvm_dirty_ring_supported to utils
tests/qtest/migration: Move ufd_version_check to utils
tests/qtest/migration: Rename migration-helpers.c
tests/qtest/migration: Move qmp helpers to a separate file
tests/qtest/migration: Move bootfile code to its own file
tests/migration: Disambiguate guestperf vs. a-b
tests/qtest/migration: Stop calling everything "test"
tests/qtest/migration: Standardize hook names
tests/qtest: Use qtest_system_reset_nowait() where appropriate
tests/qtest: Use qtest_system_reset() instead of open-coded versions
...
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Some tests need more time when qemu is built with
"--enable-asan --enable-ubsan"
As was discussed here:
https://patchew.org/QEMU/20241112120100.176492-2-frolov@swemel.ru/
TIMEOUT_MULTIPLIER enviroment variable will be
a useful option, allowing non-invasive timeouts
increasing for a specific build.
Signed-off-by: Dmitry Frolov <frolov@swemel.ru>
Tested-by: Fabiano Rosas <farosas@suse.de>
[changed from ifndef to ?=]
Signed-off-by: Fabiano Rosas <farosas@suse.de>
Make Cargo use unknown_lints = "allow" as well. This is more future
proof as we might add new lints to rust/Cargo.toml that are not supported
by older versions of rustc or clippy.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
An extra benefit of workspaces is that they allow to place lint level
settings in a single Cargo.toml; the settings are then inherited by
packages in the workspace.
Correspondingly, teach rustc_args.py to get the unexpected_cfgs
configuration from the workspace Cargo.toml.
Note that it is still possible to allow or deny warnings per crate or
module, via the #![] attribute syntax. The rust/qemu-api/src/bindings.rs
file is an example.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Cargo.toml makes it possible to describe the desired lint level settings
in a nice format. We can extend this to Meson-built crates, by teaching
rustc_args.py to fetch lint and --check-cfg arguments from Cargo.toml.
--check-cfg arguments come from the unexpected_cfgs lint as well as crate
features
Start with qemu-api, since it already has a [lints.rust] table and
an invocation of rustc_args.py.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Parse the Cargo.toml file, looking for the unexpected_cfgs
configuration. When generating --cfg options from the
config-host.h file, only use those that are included in the
configuration.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
checkpatch.pl lints for spaces around operators including / (slash).
Code lines starting with #include are ignored, as slashes in those
represent path separators.
In Objective-C code, #import is often used in preference to #include,
as preprocessor-based multiple-#include defenses are considered
non-idiomatic in that language.
This change extends checkpatch.pl to treat #import lines in the same
way as #include, avoiding false positives for "missing" spaces
around path separators on those lines.
Signed-off-by: Phil Dennis-Jordan <phil@philjordan.eu>
Message-ID: <20241024123555.25861-1-phil@philjordan.eu>
Signed-off-by: Thomas Huth <thuth@redhat.com>
* rust: cleanups
* rust: integration tests
* rust/pl011: add support for migration
* rust/pl011: add TYPE_PL011_LUMINARY device
* rust: add support for older compilers and bindgen
# -----BEGIN PGP SIGNATURE-----
#
# iQFIBAABCAAyFiEE8TM4V0tmI4mGbHaCv/vSX3jHroMFAmcrrtIUHHBib256aW5p
# QHJlZGhhdC5jb20ACgkQv/vSX3jHroPIBwf/W0Jo87UauGYufhEmoPvWG1EAQEqP
# EzNTzem9Iw92VdiSTkAtED0/TSd8RBJOwDfjjusVXZtuMPwpRNgXaFhYTT5gFTMj
# Nk3NZGaX/mbNrtdrukdx9mvUWeovytdZDZccTNkpc3oyiqY9NEz06wZ0tCNJEot6
# qO3dEtKXTOQTdx2R3o0oS+2OFDGEEPxZ0PuXN3sClN4iZhGfcIDsjGAWxEh6mCDy
# VxqKPdax1Ig1w7M+JMclnpOsVHwcefjHiToNPwhCEGelJ9BZilkViuvBzsVRJJz3
# ptYyywBE0FT8MiKQ/wyf7U64qoizJuIgHoQnUGj98hdgvbUUiW5jcBNY3A==
# =s591
# -----END PGP SIGNATURE-----
# gpg: Signature made Wed 06 Nov 2024 18:00:50 GMT
# gpg: using RSA key F13338574B662389866C7682BFFBD25F78C7AE83
# gpg: issuer "pbonzini@redhat.com"
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" [full]
# gpg: aka "Paolo Bonzini <pbonzini@redhat.com>" [full]
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4 E2F7 7E15 100C CD36 69B1
# Subkey fingerprint: F133 3857 4B66 2389 866C 7682 BFFB D25F 78C7 AE83
* tag 'for-upstream-rust' of https://gitlab.com/bonzini/qemu: (39 commits)
dockerfiles: install bindgen from cargo on Ubuntu 22.04
rust: make rustfmt optional
rust: allow older version of bindgen
rust: do not use --generate-cstr
rust: allow version 1.63.0 of rustc
rust: clean up detection of the language
rust: do not use MaybeUninit::zeroed()
rust: introduce alternative implementation of offset_of!
rust: create a cargo workspace
rust: synchronize dependencies between subprojects and Cargo.lock
rust: silence unknown warnings for the sake of old compilers
rust: introduce a c_str macro
rust: use std::os::raw instead of core::ffi
rust: fix cfgs of proc-macro2 for 1.63.0
rust: patch bilge-impl to allow compilation with 1.63.0
rust/pl011: Use correct masks for IBRD and FBRD
rust/pl011: remove commented out C code
rust/pl011: add TYPE_PL011_LUMINARY device
rust/pl011: move CLK_NAME static to function scope
rust/pl011: add support for migration
...
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Because Ubuntu 22.04 has a very old version of bindgen, that
does not have the important option --allowlist-file, it will
not be able to use --enable-rust out of the box. Instead,
install the latest version of bindgen-cli via "cargo install"
in the container, following QEMU's own documentation.
Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Occasionally, we may need to silence warnings and clippy lints that
were only introduced in newer Rust compiler versions. However, this
would fail when compiling with an older rustc:
error: unknown lint: `non_local_definitions`
--> rust/qemu-api/rust-qemu-api-tests.p/structured/offset_of.rs:79:17
So by default we need to block the unknown_lints warning. To avoid
misspelled lints or other similar issues, re-enable it in the CI job
that uses nightly rust.
Reviewed-by: Zhao Liu <zhao1.liu@intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Some applications want to use low priority realtime signals (e.g.,
SIGRTMAX). Currently QEMU cannot map all target realtime signals to
host realtime signals, and chooses to sacrifice the end of the target
realtime signal range.
Allow users to choose how to map target realtime signals to host
realtime signals using the new -t option, the new QEMU_RTSIG_MAP
environment variable, and the new -Drtsig_map=\"...\" meson flag.
To simplify things, the meson flag is not per-target, because the
intended use case is app-specific qemu-user builds.
The mapping is specified using the "tsig hsig count[,...]" syntax.
Target realtime signals [tsig,tsig+count) are mapped to host realtime
signals [hsig,hsig+count). Care is taken to avoid double and
out-of-range mappings.
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <20241029232211.206766-2-iii@linux.ibm.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
pull-loongarch-20241102
# -----BEGIN PGP SIGNATURE-----
#
# iLMEAAEKAB0WIQS4/x2g0v3LLaCcbCxAov/yOSY+3wUCZyXbXgAKCRBAov/yOSY+
# 37a9BADZ7vI2idWNXdH+mLNDZNSOxfdKp6ggNgKS3S48Hi2zR72MEhwvR9dGlHDL
# 98agrbV7/jI9Z+0dLAxvlyl1MvXfnn2sXYgUuZp6IAaQzFBa11HBAK7UFh3sTA4A
# gD4oPwl8AdJiFvDN6vNjS+dO0ls+j/YMaoLkAKLv15dlWtg4Rw==
# =EZnr
# -----END PGP SIGNATURE-----
# gpg: Signature made Sat 02 Nov 2024 07:57:18 GMT
# gpg: using RSA key B8FF1DA0D2FDCB2DA09C6C2C40A2FFF239263EDF
# gpg: Good signature from "Song Gao <m17746591750@163.com>" [unknown]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg: There is no indication that the signature belongs to the owner.
# Primary key fingerprint: B8FF 1DA0 D2FD CB2D A09C 6C2C 40A2 FFF2 3926 3EDF
* tag 'pull-loongarch-20241102' of https://gitlab.com/gaosong/qemu:
target/loongarch: Add steal time support on migration
hw/loongarch/boot: Use warn_report when no kernel filename
linux-headers: Update to Linux v6.12-rc5
linux-headers: loongarch: Add kvm_para.h
linux-headers: Add unistd_64.h
target/loongarch/kvm: Implement LoongArch PMU extension
target/loongarch: Implement lbt registers save/restore function
target/loongarch: Add loongson binary translation feature
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
since 6.11, unistd.h includes header file unistd_64.h directly on
some platforms, here add unistd_64.h on these platforms. Affected
platforms are ARM64, LoongArch64 and Riscv. Otherwise there will
be compiling error such as:
linux-headers/asm/unistd.h:3:10: fatal error: asm/unistd_64.h: No such file or directory
#include <asm/unistd_64.h>
Signed-off-by: Bibo Mao <maobibo@loongson.cn>
Acked-by: Song Gao <gaosong@loongson.cn>
Message-Id: <20241028023809.1554405-2-maobibo@loongson.cn>
Signed-off-by: Song Gao <gaosong@loongson.cn>
Fabiano Rosas
Dehan MengandKonstantin Kostiuk
Philippe Mathieu-Daudé
Stefan Hajnoczi
Dmitry FrolovandFabiano Rosas
Zhao LiuandPaolo Bonzini
Pierrick BouvierandAlex Bennée
Phil Dennis-JordanandThomas Huth
Peter Maydell
Ilya LeoshkevichandRichard Henderson
Bibo MaoandSong Gao