Migration pull request
- Avihai's fix to allow vmstate iterators to not starve for VFIO
- Maksim's fix on additional check on precopy load error
- Fabiano's fix on fdatasync() hang in mapped-ram
- Jonathan's fix on vring cached access over MMIO regions
- Cedric's cleanup patches 1-4 out of his error report series
- Yu's fix for RDMA migration (which used to be broken even for 8.2)
- Anthony's small cleanup/fix on err message
- Steve's patches on privatize migration.h
- Xiang's patchset to enable zero page detections in multifd threads
# -----BEGIN PGP SIGNATURE-----
#
# iIgEABYKADAWIQS5GE3CDMRX2s990ak7X8zN86vXBgUCZe9+uBIccGV0ZXJ4QHJl
# ZGhhdC5jb20ACgkQO1/MzfOr1wamaQD/SvmpMEcuRndT9LPSxzXowAGDZTBpYUfv
# 5XAbx80dS9IBAO8PJJgQJIBHBeacyLBjHP9CsdVtgw5/VW+wCsbfV4AB
# =xavb
# -----END PGP SIGNATURE-----
# gpg: Signature made Mon 11 Mar 2024 21:59:20 GMT
# gpg: using EDDSA key B9184DC20CC457DACF7DD1A93B5FCCCDF3ABD706
# gpg: issuer "peterx@redhat.com"
# gpg: Good signature from "Peter Xu <xzpeter@gmail.com>" [marginal]
# gpg: aka "Peter Xu <peterx@redhat.com>" [marginal]
# gpg: WARNING: This key is not certified with sufficiently trusted signatures!
# gpg: It is not certain that the signature belongs to the owner.
# Primary key fingerprint: B918 4DC2 0CC4 57DA CF7D D1A9 3B5F CCCD F3AB D706
* tag 'migration-20240311-pull-request' of https://gitlab.com/peterx/qemu: (34 commits)
migration/multifd: Add new migration test cases for legacy zero page checking.
migration/multifd: Enable multifd zero page checking by default.
migration/multifd: Implement ram_save_target_page_multifd to handle multifd version of MigrationOps::ram_save_target_page.
migration/multifd: Implement zero page transmission on the multifd thread.
migration/multifd: Add new migration option zero-page-detection.
migration/multifd: Allow clearing of the file_bmap from multifd
migration/multifd: Allow zero pages in file migration
migration: purge MigrationState from public interface
migration: delete unused accessors
migration: privatize colo interfaces
migration: migration_file_set_error
migration: migration_is_device
migration: migration_thread_is_self
migration: export vcpu_dirty_limit_period
migration: export migration_is_running
migration: export migration_is_active
migration: export migration_is_setup_or_active
migration: remove migration.h references
migration: export fewer options
migration: Fix format in error message
...
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
In the commit 54c4ea8f3a ("hw/core/machine-smp: Deprecate unsupported
'parameter=1' SMP configurations"), the SMP related item is put under
the section "User-mode emulator command line arguments" instead of
"System emulator command line arguments".
-smp is a system emulator command, so move SMP configurations item to
system emulator section.
Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20240312071512.3283513-1-zhao1.liu@linux.intel.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Commit bc38feddeb ("io: fsync before closing a file channel") added a
fsync/fdatasync at the closing point of the QIOChannelFile to ensure
integrity of the migration stream in case of QEMU crash.
The decision to do the sync at qio_channel_close() was not the best
since that function runs in the main thread and the fsync can cause
QEMU to hang for several minutes, depending on the migration size and
disk speed.
To fix the hang, remove the fsync from qio_channel_file_close().
At this moment, the migration code is the only user of the fsync and
we're taking the tradeoff of not having a sync at all, leaving the
responsibility to the upper layers.
Fixes: bc38feddeb ("io: fsync before closing a file channel")
Reviewed-by: "Daniel P. Berrangé" <berrange@redhat.com>
Signed-off-by: Fabiano Rosas <farosas@suse.de>
Link: https://lore.kernel.org/r/20240305195629.9922-1-farosas@suse.de
Link: https://lore.kernel.org/r/20240305174332.2553-1-farosas@suse.de
[peterx: add more comment to the qio_channel_close()]
Signed-off-by: Peter Xu <peterx@redhat.com>
QEMU includes some models of old Arm machine types which are
a bit problematic for us because:
* they're written in a very old way that uses numerous APIs that we
would like to get away from (eg they don't use qdev, they use
qemu_system_reset_request(), they use vmstate_register(), etc)
* they've been that way for a decade plus and nobody particularly has
stepped up to try to modernise the code (beyond some occasional
work here and there)
* we often don't have test cases for them, which means that if we
do try to do the necessary refactoring work on them we have no
idea if they even still work at all afterwards
All these machine types are also of hardware that has largely passed
away into history and where I would not be surprised to find that
e.g. the Linux kernel support was never tested on real hardware
any more.
After some consultation with the Linux kernel developers, we
are going to deprecate:
All PXA2xx machines:
akita Sharp SL-C1000 (Akita) PDA (PXA270)
borzoi Sharp SL-C3100 (Borzoi) PDA (PXA270)
connex Gumstix Connex (PXA255)
mainstone Mainstone II (PXA27x)
spitz Sharp SL-C3000 (Spitz) PDA (PXA270)
terrier Sharp SL-C3200 (Terrier) PDA (PXA270)
tosa Sharp SL-6000 (Tosa) PDA (PXA255)
verdex Gumstix Verdex Pro XL6P COMs (PXA270)
z2 Zipit Z2 (PXA27x)
All OMAP2 machines:
n800 Nokia N800 tablet aka. RX-34 (OMAP2420)
n810 Nokia N810 tablet aka. RX-44 (OMAP2420)
One of the OMAP1 machines:
cheetah Palm Tungsten|E aka. Cheetah PDA (OMAP310)
Rationale:
* for QEMU dropping individual machines is much less beneficial
than if we can drop support for an entire SoC
* the OMAP2 QEMU code in particular is large, old and unmaintained,
and none of the OMAP2 kernel maintainers said they were using
QEMU in any of their testing/development
* although there is a setup that is booting test kernels on some
of the PXA2xx machines, nobody seemed to be using them as part
of their active kernel development and my impression from the
email thread is that PXA is the closest of all these SoC families
to being dropped from the kernel soon
* nobody said they were using cheetah, so it's entirely
untested and quite probably broken
* on the other hand the OMAP1 sx1 model does seem to be being
used as part of kernel development, and there was interest
in keeping collie around
In particular, the mainstone, tosa and z2 machine types have
already been dropped from Linux.
Mark all these machine types as deprecated.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-id: 20240308171621.3749894-1-peter.maydell@linaro.org
Misc HW patch queue
- hmp: Shorter 'info qtree' output (Zoltan)
- qdev: Add a granule_mode property (Eric)
- Some ERRP_GUARD() fixes (Zhao)
- Doc & style fixes in docs/interop/firmware.json (Thomas)
- hw/xen: Housekeeping (Phil)
- hw/ppc/mac99: Change timebase frequency 25 -> 100 MHz (Mark)
- hw/intc/apic: Memory leak fix (Paolo)
- hw/intc/grlib_irqmp: Ensure ncpus value is in range (Clément)
- hw/m68k/mcf5208: Add support for reset (Angelo)
- hw/i386/pc: Housekeeping (Phil)
- hw/core/smp: Remove/deprecate parameter=0,1 adapting test-smp-parse (Zhao)
# -----BEGIN PGP SIGNATURE-----
#
# iQIzBAABCAAdFiEE+qvnXhKRciHc/Wuy4+MsLN6twN4FAmXstpMACgkQ4+MsLN6t
# wN6XBw//dNItFhf1YX+Au4cNoQVDgHE9RtzEIGOnwcL1CgrA9rAQQfLRE5KWM6sN
# 1qiPh+T6SPxtiQ2rw4AIpsI7TXjO72b/RDWpUUSwnfH39eC77pijkxIK+i9mYI9r
# p0sPjuP6OfolUFYeSbYX+DmNZh1ONPf27JATJQEf0st8dyswn7lTQvJEaQ97kwxv
# UKA0JD5l9LZV8Zr92cgCzlrfLcbVblJGux9GYIL09yN78yqBuvTm77GBC/rvC+5Q
# fQC5PQswJZ0+v32AXIfSysMp2R6veo4By7VH9Lp51E/u9jpc4ZbcDzxzaJWE6zOR
# fZ01nFzou1qtUfZi+MxNiDR96LP6YoT9xFdGYfNS6AowZn8kymCs3eo7M9uvb+rN
# A2Sgis9rXcjsR4e+w1YPBXwpalJnLwB0QYhEOStR8wo1ceg7GBG6zHUJV89OGzsA
# KS8X0aV1Ulkdm/2H6goEhzrcC6FWLg8pBJpfKK8JFWxXNrj661xM0AAFVL9we356
# +ymthS2x/RTABSI+1Lfsoo6/SyXoimFXJJWA82q9Yzoaoq2gGMWnfwqxfix6JrrA
# PuMnNP5WNvh04iWcNz380P0psLVteHWcVfTRN3JvcJ9iJ2bpjcU1mQMJtvSF9wBn
# Y8kiJTUmZCu3br2e5EfxmypM/h8y29VD/1mxPk8Dtcq3gjx9AU4=
# =juZH
# -----END PGP SIGNATURE-----
# gpg: Signature made Sat 09 Mar 2024 19:20:51 GMT
# gpg: using RSA key FAABE75E12917221DCFD6BB2E3E32C2CDEADC0DE
# gpg: Good signature from "Philippe Mathieu-Daudé (F4BUG) <f4bug@amsat.org>" [full]
# Primary key fingerprint: FAAB E75E 1291 7221 DCFD 6BB2 E3E3 2C2C DEAD C0DE
* tag 'hw-misc-20240309' of https://github.com/philmd/qemu: (43 commits)
hw/m68k/mcf5208: add support for reset
tests/unit/test-smp-parse: Test "parameter=0" SMP configurations
tests/unit/test-smp-parse: Test smp_props.has_clusters
tests/unit/test-smp-parse: Test the full 7-levels topology hierarchy
tests/unit/test-smp-parse: Test "drawers" and "books" combination case
tests/unit/test-smp-parse: Test "drawers" parameter in -smp
tests/unit/test-smp-parse: Test "books" parameter in -smp
tests/unit/test-smp-parse: Make test cases aware of the book/drawer
tests/unit/test-smp-parse: Bump max_cpus to 4096
tests/unit/test-smp-parse: Use CPU number macros in invalid topology case
tests/unit/test-smp-parse: Drop the unsupported "dies=1" case
hw/core/machine-smp: Calculate total CPUs once in machine_parse_smp_config()
hw/core/machine-smp: Deprecate unsupported "parameter=1" SMP configurations
hw/core/machine-smp: Remove deprecated "parameter=0" SMP configurations
docs/interop/firmware.json: Fix doc for FirmwareFlashMode
docs/interop/firmware.json: Align examples
hw/intc/grlib_irqmp: abort realize when ncpus value is out of range
mac_newworld: change timebase frequency from 100MHz to 25MHz for mac99 machine
hmp: Add option to info qtree to omit details
qdev: Add a granule_mode property
...
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Currently, it was allowed for users to specify the unsupported
topology parameter as "1". For example, x86 PC machine doesn't
support drawer/book/cluster topology levels, but user could specify
"-smp drawers=1,books=1,clusters=1".
This is meaningless and confusing, so that the support for this kind of
configurations is marked deprecated since 9.0. And report warning
message for such case like:
qemu-system-x86_64: warning: Deprecated CPU topology (considered invalid):
Unsupported clusters parameter mustn't be specified as 1
qemu-system-x86_64: warning: Deprecated CPU topology (considered invalid):
Unsupported books parameter mustn't be specified as 1
qemu-system-x86_64: warning: Deprecated CPU topology (considered invalid):
Unsupported drawers parameter mustn't be specified as 1
Users have to ensure that all the topology members described with -smp
are supported by the target machine.
Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20240308160148.3130837-3-zhao1.liu@linux.intel.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
The "parameter=0" SMP configurations have been marked as deprecated
since v6.2.
For these cases, -smp currently returns the warning and adjusts the
zeroed parameters to 1 by default.
Remove the above compatibility logic in v9.0, and return error directly
if any -smp parameter is set as 0.
Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Prasad Pandit <pjp@fedoraproject.org>
Message-ID: <20240308160148.3130837-2-zhao1.liu@linux.intel.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Update bios-bits docs to add more details on why a pre-OS environment for
testing bioses is useful. Add author's FOSDEM talk link. Also improve the
formating of the document while at it.
Signed-off-by: Ani Sinha <anisinha@redhat.com>
Reviewed-by: Michael Tokarev <mjt@tls.msk.ru>
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
Features supported :
- the 8 STM32L4x5 GPIOs are initialized with their reset values
(except IDR, see below)
- input mode : setting a pin in input mode "externally" (using input
irqs) results in an out irq (transmitted to SYSCFG)
- output mode : setting a bit in ODR sets the corresponding out irq
(if this line is configured in output mode)
- pull-up, pull-down
- push-pull, open-drain
Difference with the real GPIOs :
- Alternate Function and Analog mode aren't implemented :
pins in AF/Analog behave like pins in input mode
- floating pins stay at their last value
- register IDR reset values differ from the real one :
values are coherent with the other registers reset values
and the fact that AF/Analog modes aren't implemented
- setting I/O output speed isn't supported
- locking port bits isn't supported
- ADC function isn't supported
- GPIOH has 16 pins instead of 2 pins
- writing to registers LCKR, AFRL, AFRH and ASCR is ineffective
Signed-off-by: Arnaud Minier <arnaud.minier@telecom-paris.fr>
Signed-off-by: Inès Varhol <ines.varhol@telecom-paris.fr>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Acked-by: Alistair Francis <alistair.francis@wdc.com>
Message-id: 20240305210444.310665-2-ines.varhol@telecom-paris.fr
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
A few deficiencies in the current device model need to be noted.
1. FIFOs are not used. All sends and receives are done directly.
2. Repeated starts are not emulated. Repeated starts can be triggered in real
hardware by sending a new read transfer request in the window time between
transfer active set of write transfer request and done bit set of the same.
Signed-off-by: Rayhan Faizel <rayhan.faizel@gmail.com>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Message-id: 20240224191038.2409945-2-rayhan.faizel@gmail.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Migartion pull request for 20240304
- Bryan's fix on multifd compression level API
- Fabiano's mapped-ram series (base + multifd only)
- Steve's amend on cpr document in qapi/
# -----BEGIN PGP SIGNATURE-----
#
# iIgEABYKADAWIQS5GE3CDMRX2s990ak7X8zN86vXBgUCZeUjKhIccGV0ZXJ4QHJl
# ZGhhdC5jb20ACgkQO1/MzfOr1wbv5QD/ZexBUsmZA5qyxgGvZ2yvlUBEGNOvtmKY
# kRdiYPU7khMA/0N43rn4LcqKCoq4+T+EAnYizGjIyhH/7BRUyn4DUxgO
# =AeEn
# -----END PGP SIGNATURE-----
# gpg: Signature made Mon 04 Mar 2024 01:26:02 GMT
# gpg: using EDDSA key B9184DC20CC457DACF7DD1A93B5FCCCDF3ABD706
# gpg: issuer "peterx@redhat.com"
# gpg: Good signature from "Peter Xu <xzpeter@gmail.com>" [marginal]
# gpg: aka "Peter Xu <peterx@redhat.com>" [marginal]
# gpg: WARNING: This key is not certified with sufficiently trusted signatures!
# gpg: It is not certain that the signature belongs to the owner.
# Primary key fingerprint: B918 4DC2 0CC4 57DA CF7D D1A9 3B5F CCCD F3AB D706
* tag 'migration-next-pull-request' of https://gitlab.com/peterx/qemu: (27 commits)
migration/multifd: Document two places for mapped-ram
tests/qtest/migration: Add a multifd + mapped-ram migration test
migration/multifd: Add mapped-ram support to fd: URI
migration/multifd: Support incoming mapped-ram stream format
migration/multifd: Support outgoing mapped-ram stream format
migration/multifd: Prepare multifd sync for mapped-ram migration
migration/multifd: Add incoming QIOChannelFile support
migration/multifd: Add outgoing QIOChannelFile support
migration/multifd: Add a wrapper for channels_created
migration/multifd: Allow receiving pages without packets
migration/multifd: Allow multifd without packets
migration/multifd: Decouple recv method from pages
migration/multifd: Rename MultiFDSend|RecvParams::data to compress_data
tests/qtest/migration: Add tests for mapped-ram file-based migration
migration/ram: Add incoming 'mapped-ram' migration
migration/ram: Add outgoing 'mapped-ram' migration
migration: Add mapped-ram URI compatibility check
migration/ram: Introduce 'mapped-ram' migration capability
migration/qemu-file: add utility methods for working with seekable channels
io: fsync before closing a file channel
...
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
# Conflicts:
# migration/ram.c
The macOS jobs in our CI recently started failing, complaining that
the distutils module is not available anymore. And indeed, according to
https://peps.python.org/pep-0632/ it's been deprecated since a while
and now likely got removed in recent Python versions.
Fortunately, we only use it for a version check via LooseVersion here
which we don't really need anymore - according to Repology.org, these
are the versions of sphinx-rtd-theme that are currently used by the
various distros:
centos_stream_8: 0.3.1
centos_stream_9: 0.5.1
fedora_38: 1.1.1
fedora_39: 1.2.2
freebsd: 1.0.0
haikuports_master: 1.2.1
openbsd: 1.2.2
opensuse_leap_15_5: 0.5.1
pkgsrc_current: 2.0.0
debian_11: 0.5.1
debian_12: 1.2.0
ubuntu_20_04: 0.4.3
ubuntu_22_04: 1.0.0
ubuntu_24_04: 2.0.0
So except for CentOS 8, all distros are using a newer version of
sphinx-rtd-theme, and for CentOS 8 we don't support compiling with
the Sphinx of the distro anymore anyway, since it's based on the
Python 3.6 interpreter there. For compiling on CentOS 8, you have
to use the alternative Python 3.8 interpreter which comes without
Sphinx, so that needs the Sphinx installed via pip in the venv
instead, and that is using a newer version, too, according to our
pythondeps.toml file.
Thus we can simply drop the version check now to get rid of the
distutils dependency here.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Michael Tokarev <mjt@tls.msk.ru>
Message-id: 20240304130403.129543-1-thuth@redhat.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Avoid "JSON" when talking about the QAPI schema syntax. Capitalize
QEMU. Don't claim all HMP commands live in monitor/hmp-cmds.c (this
was never true). Fix punctuation and drop inappropriate "the" here
and there.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240227115617.237875-3-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
The tutorial doesn't match reality since at least 2013. Repairing it
involves fixing the following issues:
* Update for commit 6d32717155 (aio / timers: Remove alarm timers):
replace the broken examples. Instead of having one for returning a
struct and another for returning a list of structs, do just one for
the latter. This resolves the FIXME added in commit
e218052f92 (aio / timers: De-document -clock) back in 2014.
* Update for commit 895a2a80e0 (qapi: Use 'struct' instead of 'type'
in schema).
* Update for commit 3313b6124b (qapi: add qapi2texi script): add
required documentation to the schema snippets, and drop section
"Command Documentation".
* Update for commit a3c45b3e62 (qapi: New special feature flag
"unstable"): supply the required feature, deemphasize the x- prefix.
* Update for commit dd98234c05 (qapi: introduce x-query-roms QMP
command): rephrase from "add new command" to "examine existing
command".
* Update for commit 9492718b7c (qapi misc: Elide redundant has_FOO in
generated C): hello-world's message argument no longer comes with a
has_message, add a second argument that does.
* Update for moved and renamed files.
While there, update QMP version output to current output.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240227115617.237875-2-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[Whitespace tidied up, typo fixed]
We use section "Returns" for documenting both success and error
response of commands.
I intend to generate better command success response documentation.
Easier when "Returns" documents just he success response.
Create new section tag "Errors". The next two commits will move error
response documentation from "Returns" sections to "Errors" sections.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-ID: <20240227113921.236097-4-armbru@redhat.com>
Add a new migration capability 'mapped-ram'.
The core of the feature is to ensure that RAM pages are mapped
directly to offsets in the resulting migration file instead of being
streamed at arbitrary points.
The reasons why we'd want such behavior are:
- The resulting file will have a bounded size, since pages which are
dirtied multiple times will always go to a fixed location in the
file, rather than constantly being added to a sequential
stream. This eliminates cases where a VM with, say, 1G of RAM can
result in a migration file that's 10s of GBs, provided that the
workload constantly redirties memory.
- It paves the way to implement O_DIRECT-enabled save/restore of the
migration stream as the pages are ensured to be written at aligned
offsets.
- It allows the usage of multifd so we can write RAM pages to the
migration file in parallel.
For now, enabling the capability has no effect. The next couple of
patches implement the core functionality.
Acked-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Fabiano Rosas <farosas@suse.de>
Link: https://lore.kernel.org/r/20240229153017.2221-8-farosas@suse.de
Signed-off-by: Peter Xu <peterx@redhat.com>
This option controls the host page size. From the mis-usage in
our own testsuite, this is easily confused with guest page size.
The only thing that occurs when changing the host page size is
that stuff breaks, because one cannot actually change the host
page size. Therefore reject all but the no-op setting as part
of the deprecation process.
Reviewed-by: Warner Losh <imp@bsdimp.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Acked-by: Helge Deller <deller@gmx.de>
Message-Id: <20240102015808.132373-27-richard.henderson@linaro.org>
Peter Maydell
Zhao LiuandPhilippe Mathieu-Daudé
Fabiano RosasandPeter Xu
Ani SinhaandPeter Maydell
Thomas WeißschuhandPhilippe Mathieu-Daudé
Inès VarholandPeter Maydell
Arnaud MinierandPeter Maydell
Rayhan FaizelandPeter Maydell
Thomas HuthandPeter Maydell
Markus Armbruster
Richard Henderson