Migration pull 2021-05-13
Fix of the 2021-05-11 version, with a fix to build on the armhf
cross.
The largest change in this set is David's changes for ram block size
changing; then there's a pile of other cleanups and fixes.
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
# gpg: Signature made Thu 13 May 2021 18:36:06 BST
# gpg: using RSA key 45F5C71B4A0CB7FB977A9FA90516331EBC5BFDE7
# gpg: Good signature from "Dr. David Alan Gilbert (RH2) <dgilbert@redhat.com>" [full]
# Primary key fingerprint: 45F5 C71B 4A0C B7FB 977A 9FA9 0516 331E BC5B FDE7
* remotes/dgilbert/tags/pull-migration-20210513a:
tests/migration: introduce multifd into guestperf
tests/qtest/migration-test: Use g_autofree to avoid leaks on error paths
tests/migration-test: Fix "true" vs true
migration/ram: Use offset_in_ramblock() in range checks
migration/multifd: Print used_length of memory block
migration/ram: Handle RAM block resizes during postcopy
migration/ram: Simplify host page handling in ram_load_postcopy()
migration/ram: Discard RAM when growing RAM blocks after ram_postcopy_incoming_init()
exec: Relax range check in ram_block_discard_range()
migration/ram: Handle RAM block resizes during precopy
numa: Make all callbacks of ram block notifiers optional
numa: Teach ram block notifiers about resizeable ram blocks
util: vfio-helpers: Factor out and fix processing of existing ram blocks
migration: Drop redundant query-migrate result @blocked
migration/ram: Optimize ram_save_host_page()
migration/ram: Reduce unnecessary rate limiting
migrate/ram: remove "ram_bulk_stage" and "fpo_enabled"
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Result @blocked is redundant. Unfortunately, we realized this too
close to the release to risk dropping it, so we deprecated it
instead, in commit e11ce6c06.
Since it was deprecated from the start, we can delete it without
the customary grace period. Do so.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20210429140424.2802929-1-armbru@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
The QMP dispatcher coroutine holds the qmp_queue_lock over a yield
point, where it expects to be rescheduled from the main context. If a
CHR_EVENT_CLOSED event is received just then, it can race and block the
main thread on the mutex in monitor_qmp_cleanup_queue_and_resume.
monitor_resume does not need to be called from main context, so we can
call it immediately after popping a request from the queue, which allows
us to drop the qmp_queue_lock mutex before yielding.
Suggested-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Message-Id: <20210322154024.15011-1-s.reiter@proxmox.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Cc: qemu-stable@nongnu.org
Signed-off-by: Markus Armbruster <armbru@redhat.com>
This patch provides a new qmp to reload display configuration
without restart VM, but only reloading the vnc tls certificates
is implemented.
Example:
{"execute": "display-reload", "arguments":{"type": "vnc", "tls-certs": true}}
Signed-off-by: Zihao Chang <changzihao1@huawei.com>
Message-Id: <20210316075845.1476-4-changzihao1@huawei.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
This policy suppresses deprecated bits in output, and thus permits
"testing the future". Implement it for QMP command query-qmp-schema:
suppress information on deprecated commands, events and object type
members, i.e. anything that has the special feature flag "deprecated".
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210318155519.1224118-8-armbru@redhat.com>
QMP commands return their response as a generated QAPI type, which the
monitor core converts to JSON via QObject.
query-qmp-schema's response is the generated introspection data. This
is a QLitObject since commit 7d0f982bfb "qapi: generate a literal
qobject for introspection", v2.12). Before, it was a string. Instead
of converting QLitObject / string -> QObject -> QAPI type
SchemaInfoList -> QObject -> JSON, we take a shortcut: the command is
'gen': false, so it can return the QObject instead of the QAPI type.
Slightly simpler and more efficient.
The next commit will filter the response for output policy, and this
is easier in the SchemaInfoList representation. Drop the shortcut.
This replaces the manual command registration by a generated one. The
manual registration makes the command available before the machine is
built by passing flag QCO_ALLOW_PRECONFIG. To keep it available
there, we need need to add 'allow-preconfig': true to its definition
in the schema.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210318155519.1224118-7-armbru@redhat.com>
This switches the HMP command object_add from a QemuOpts-based parser to
user_creatable_add_from_str() which uses a keyval parser and enforces
the QAPI schema.
Apart from being a cleanup, this makes non-scalar properties and help
accessible. In order for help to be printed to the monitor instead of
stdout, the printf() calls in the help functions are changed to
qemu_printf().
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Acked-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
This converts object-add from 'gen': false to the ObjectOptions QAPI
type. As an immediate benefit, clients can now use QAPI schema
introspection for user creatable QOM objects.
It is also the first step towards making the QAPI schema the only
external interface for the creation of user creatable objects. Once all
other places (HMP and command lines of the system emulator and all
tools) go through QAPI, too, some object implementations can be
simplified because some checks (e.g. that mandatory options are set) are
already performed by QAPI, and in another step, QOM boilerplate code
could be generated from the schema.
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Acked-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
The generic 'migrate_set_parameters' command handle all types of param.
Only the QMP commands were documented in the deprecations page, but the
rationale for deprecating applies equally to HMP, and the replacements
exist. Furthermore the HMP commands are just shims to the QMP commands,
so removing the latter breaks the former unless they get re-implemented.
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The code comment suggests removing QAPIEvent_(str|lookup) symbols too,
however, these are both auto-generated as standard for any enum in
QAPI. As such it they'll exist whether we use them or not.
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
This is only semantically useful for QMP.
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
The VNC ACL concept has been replaced by the pluggable "authz" framework
which does not use monitor commands.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Removed various qemu_mutex_lock and their respective qemu_mutex_unlock
calls and used lock guard macros (QEMU_LOCK_GUARD and
WITH_QEMU_LOCK_GUARD). This simplifies the code by
eliminating qemu_mutex_unlock calls.
Signed-off-by: Mahmoud Mandour <ma.mandourr@gmail.com>
Message-Id: <20210311031538.5325-6-ma.mandourr@gmail.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Before this patch, monitor_qmp_dispatcher_co() used to check whether
shutdown is requested only when it would have to wait for new requests.
If there were still some queued requests, it would try to execute all of
them before shutting down.
This can be surprising when the queued QMP commands take long or hang
because Ctrl-C may not actually exit QEMU as soon as possible.
Change monitor_qmp_dispatcher_co() so that it additionally checks
whether shutdown is request before it gets a new request from the queue.
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20210212172028.288825-3-kwolf@redhat.com>
Tested-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Commit 357bda95 already tried to fix the order in monitor_cleanup() by
moving shutdown of the dispatcher coroutine further to the start.
However, it didn't go far enough:
iothread_stop() makes sure that all pending work (bottom halves) in the
AioContext of the monitor iothread is completed. iothread_destroy()
depends on this and fails an assertion if there is still a pending BH.
While the dispatcher coroutine is running, it will try to resume the
monitor after taking a request out of the queue, which involves a BH.
The dispatcher is run until it terminates in the AIO_WAIT_WHILE() loop.
However, adding new BHs between iothread_stop() and iothread_destroy()
is forbidden.
Fix this by stopping the dispatcher first before shutting down the other
parts of the monitor. This means we can now receive requests that aren't
handled any more when QEMU is shutting down, but this is unlikely to be
a problem for QMP clients.
Fixes: 357bda9590
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20210212172028.288825-2-kwolf@redhat.com>
Tested-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Make snapshot deletion consistent with the snapshot save
and load commands by using a wrapper around the blockdev
layer. The main difference is that we get upfront validation
of the passed in device list (if any).
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20210204124834.774401-10-berrange@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
The traditional HMP "savevm" command will overwrite an existing snapshot
if it already exists with the requested name. This new flag allows this
to be controlled allowing for safer behaviour with a future QMP command.
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20210204124834.774401-8-berrange@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
When running snapshot operations, there are various rules for which
blockdevs are included/excluded. While this provides reasonable default
behaviour, there are scenarios that are not well handled by the default
logic. Some of the conditions do not have a single correct answer.
Thus there needs to be a way for the mgmt app to provide an explicit
list of blockdevs to perform snapshots across. This can be achieved
by passing a list of node names that should be used.
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20210204124834.774401-5-berrange@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
None of the callers care about the errno value since there is a full
Error object populated. This gives consistency with save_snapshot()
which already just returns a boolean value.
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
[PMD: Return false/true instead of -1/0, document function]
Acked-by: Pavel Dovgalyuk <pavel.dovgalyuk@ispras.ru>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210204124834.774401-4-berrange@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
The bdrv_all_*_snapshot functions return a BlockDriverState pointer
for the invalid backend, which the callers then use to report an
error message. In some cases multiple callers are reporting the
same error message, but with slightly different text. In the future
there will be more error scenarios for some of these methods, which
will benefit from fine grained error message reporting. So it is
helpful to push error reporting down a level.
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
[PMD: Initialize variables]
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210204124834.774401-2-berrange@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Update 'info migrate' to display migration blocking information.
If the outbound migration is not blocked, there is no change, however
if it is blocked a message is displayed with a list of reasons why,
e.g.
qemu-system-x86_64 -nographic -smp 4 -m 4G -M pc,usb=on \
-chardev null,id=n -device usb-serial,chardev=n \
-virtfs local,path=/home,mount_tag=fs,security_model=none \
-drive if=virtio,file=myimage.qcow2
(qemu) info migrate
globals:
store-global-state: on
only-migratable: off
send-configuration: on
send-section-footer: on
decompress-error-check: on
clear-bitmap-shift: 18
Outgoing migration blocked:
Migration is disabled when VirtFS export path '/home' is mounted in the guest using mount_tag 'fs'
non-migratable device: 0000:00:01.2/1/usb-serial
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20210202135522.127380-3-dgilbert@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Commit 741d4086c8 "migration: Use proper types in json" (v2.12.0)
switched MigrationParameters to narrower integer types, and removed
the simplified qmp_migrate_set_parameters()'s argument checking
accordingly.
Good idea, except qmp_migrate_set_parameters() takes
MigrateSetParameters, not MigrationParameters. Its job is updating
migrate_get_current()->parameters (which *is* of type
MigrationParameters) according to its argument. The integers now get
truncated silently. Reproducer:
---> {'execute': 'query-migrate-parameters'}
<--- {"return": {[...] "compress-threads": 8, [...]}}
---> {"execute": "migrate-set-parameters", "arguments": {"compress-threads": 257}}
<--- {"return": {}}
---> {'execute': 'query-migrate-parameters'}
<--- {"return": {[...] "compress-threads": 1, [...]}}
Fix by resynchronizing MigrateSetParameters with MigrationParameters.
Fixes: 741d4086c8
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20210202141734.2488076-2-armbru@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
