izzy/xemu
0
0
Fork 0
mirror of https://github.com/izzy2lost/xemu.git synced 2026-07-06 00:20:22 -07:00
Code Issues Packages Projects Releases Wiki Activity

Merge tag 'migration-20250711-pull-request' of https://gitlab.com/farosas/qemu into staging

Browse Source 
Migration pull request

- General cleanups around: postcopy, bg-snapshot, migration hooks,
  migration completion and formatting of 'info migrate'.

- Overhaul of postcopy blocktime tracking.

# -----BEGIN PGP SIGNATURE-----
#
# iQJEBAABCAAuFiEEqhtIsKIjJqWkw2TPx5jcdBvsMZ0FAmhxGdgQHGZhcm9zYXNA
# c3VzZS5kZQAKCRDHmNx0G+wxnahoD/9uNXirlmRk3tDnhiJsiYx+HnXYPFEORSZq
# zlpUyqvhQ1POp3Fa5pRf+bJ5mmPw8h8PdOR2StMpnW2Xa1OatAZj5m1uityAVWOl
# EkVfZLl0j6j9HCCmE3c4dztOGIBsd9YY0GWizL05XHYZPrdX4zOpolMN4m53RwQY
# HUVD6T2y9eFDnCO6MsoA9EfmkFYCRvqlS0VzTcYzQFN4H+QHlcpDfweqJpTLPa+1
# trahAN9PBuMjoewjDqwkNkf0CLaCXHszAfj6yv62Vi8Cbp9DDPywIYJKFnxspElW
# Fjg1b4MdsbYZNmeKgIawzgTOL1RrojvKkoi7KWp3D7M+/ZZl9kBwQuUcBXKI7N0R
# Y0GNfkkTycn18nM0JU/6QWSuVeiPbLArxQUGP1cLgvcHSSNgD9JxWbNBu5+1fFOG
# Gg3qnyYatJ6xJDiCrdKqV8fwozNlm/G6b9BiCDeVq+4nA2OKQ0shiNA1GZHvVSQL
# X4uAPexETdHfA/LeA2w5sgVBEw7BewBdjLntZDIFsyBnLrvqrDcU5Aav0wiHoI8U
# QBC2aIpJfMLHiIQ93mVX96NltXC7KvJTIZVl3iwfiYEYCvQtTYgdJ09ELXFJYxFX
# XpTTazqpmPSfuZpPRgx9YbDP/kS8Fg/PTOlPeD0T/frFgd1S6Thh6OW455PavMp8
# ht2lE4sxjA==
# =vtRD
# -----END PGP SIGNATURE-----
# gpg: Signature made Fri 11 Jul 2025 10:04:08 EDT
# gpg:                using RSA key AA1B48B0A22326A5A4C364CFC798DC741BEC319D
# gpg:                issuer "farosas@suse.de"
# gpg: Good signature from "Fabiano Rosas <farosas@suse.de>" [unknown]
# gpg:                 aka "Fabiano Almeida Rosas <fabiano.rosas@suse.com>" [unknown]
# gpg: WARNING: The key's User ID is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: AA1B 48B0 A223 26A5 A4C3  64CF C798 DC74 1BEC 319D

* tag 'migration-20250711-pull-request' of https://gitlab.com/farosas/qemu: (26 commits)
  migration: Rename save_live_complete_precopy_thread to save_complete_precopy_thread
  migration/postcopy: Add latency distribution report for blocktime
  migration/postcopy: blocktime allows track / report non-vCPU faults
  migration/postcopy: Optimize blocktime fault tracking with hashtable
  migration/postcopy: Cleanup the total blocktime accounting
  migration/postcopy: Cache the tid->vcpu mapping for blocktime
  migration/postcopy: Initialize blocktime context only until listen
  migration/postcopy: Report fault latencies in blocktime
  migration/postcopy: Add blocktime fault counts per-vcpu
  migration/postcopy: Bring blocktime layer to ns level
  migration/postcopy: Drop PostcopyBlocktimeContext.start_time
  migration/postcopy: Make all blocktime vars 64bits
  migration/postcopy: Drop all atomic ops in blocktime feature
  migration/postcopy: Push blocktime start/end into page req mutex
  migration: Add option to set postcopy-blocktime
  migration/postcopy: Avoid clearing dirty bitmap for postcopy too
  migration: Rewrite the migration complete detect logic
  migration/ram: Add tracepoints for ram_save_complete()
  migration/ram: One less indent for ram_find_and_save_block()
  migration: qemu_savevm_complete*() helpers
  ...

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
This commit is contained in:
Stefan Hajnoczi
2025-07-13 01:45:30 -04:00
parent 0edc2afe0c beeac2df5f
commit 52af79811f
24 changed files with 803 additions and 326 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/devel/migration/main.rst
+2 -2
View File
@@ -508,8 +508,8 @@ An iterative device must provide:
the point that stream bandwidth limits tell it to stop. Each call
generates one section.
- A ``save_live_complete_precopy`` function that must transmit the
last section for the device containing any remaining data.
- A ``save_complete`` function that must transmit the last section for
the device containing any remaining data.
- A ``load_state`` function used to load sections generated by
any of the save functions that generate sections.
docs/devel/migration/postcopy.rst
+17 -19
View File
@@ -33,25 +33,6 @@ will now cause the transition from precopy to postcopy.
It can be issued immediately after migration is started or any
time later on. Issuing it after the end of a migration is harmless.
Blocktime is a postcopy live migration metric, intended to show how
long the vCPU was in state of interruptible sleep due to pagefault.
That metric is calculated both for all vCPUs as overlapped value, and
separately for each vCPU. These values are calculated on destination
side. To enable postcopy blocktime calculation, enter following
command on destination monitor:
``migrate_set_capability postcopy-blocktime on``
Postcopy blocktime can be retrieved by query-migrate qmp command.
postcopy-blocktime value of qmp command will show overlapped blocking
time for all vCPU, postcopy-vcpu-blocktime will show list of blocking
time per vCPU.
.. note::
During the postcopy phase, the bandwidth limits set using
``migrate_set_parameter`` is ignored (to avoid delaying requested pages that
the destination is waiting for).
Postcopy internals
==================
@@ -312,3 +293,20 @@ explicitly) to be sent in a separate preempt channel, rather than queued in
the background migration channel. Anyone who cares about latencies of page
faults during a postcopy migration should enable this feature. By default,
it's not enabled.
Postcopy blocktime statistics
-----------------------------
Blocktime is a postcopy live migration metric, intended to show how
long the vCPU was in state of interruptible sleep due to pagefault.
That metric is calculated both for all vCPUs as overlapped value, and
separately for each vCPU. These values are calculated on destination
side. To enable postcopy blocktime calculation, enter following
command on destination monitor:
``migrate_set_capability postcopy-blocktime on``
Postcopy blocktime can be retrieved by query-migrate qmp command.
postcopy-blocktime value of qmp command will show overlapped blocking
time for all vCPU, postcopy-vcpu-blocktime will show list of blocking
time per vCPU.
docs/devel/migration/vfio.rst
+8 -8
View File
@@ -75,12 +75,12 @@ VFIO implements the device hooks for the iterative approach as follows:
in the non-multifd mode.
In the multifd mode it just emits either a dummy EOS marker.
* A ``save_live_complete_precopy`` function that sets the VFIO device in
_STOP_COPY state and iteratively copies the data for the VFIO device until
the vendor driver indicates that no data remains.
In the multifd mode it just emits a dummy EOS marker.
* A ``save_complete`` function that sets the VFIO device in _STOP_COPY
state and iteratively copies the data for the VFIO device until the
vendor driver indicates that no data remains. In the multifd mode it
just emits a dummy EOS marker.
* A ``save_live_complete_precopy_thread`` function that in the multifd mode
* A ``save_complete_precopy_thread`` function that in the multifd mode
provides thread handler performing multifd device state transfer.
It sets the VFIO device to _STOP_COPY state, iteratively reads the data
from the VFIO device and queues it for multifd transmission until the vendor
@@ -195,12 +195,12 @@ Live migration save path
|
Then the VFIO device is put in _STOP_COPY state
(FINISH_MIGRATE, _ACTIVE, _STOP_COPY)
.save_live_complete_precopy() is called for each active device
.save_complete() is called for each active device
For the VFIO device: in the non-multifd mode iterate in
.save_live_complete_precopy() until
.save_complete() until
pending data is 0
In the multifd mode this iteration is done in
.save_live_complete_precopy_thread() instead.
.save_complete_precopy_thread() instead.
|
(POSTMIGRATE, _COMPLETED, _STOP_COPY)
Migraton thread schedules cleanup bottom half and exits
hw/ppc/spapr.c
+1 -1
View File
@@ -2518,7 +2518,7 @@ static void htab_save_cleanup(void *opaque)
static SaveVMHandlers savevm_htab_handlers = {
.save_setup = htab_save_setup,
.save_live_iterate = htab_save_iterate,
.save_live_complete_precopy = htab_save_complete,
.save_complete = htab_save_complete,
.save_cleanup = htab_save_cleanup,
.load_state = htab_load,
};
hw/s390x/s390-stattrib.c
+1 -1
View File
@@ -338,7 +338,7 @@ static const TypeInfo qemu_s390_stattrib_info = {
static SaveVMHandlers savevm_s390_stattrib_handlers = {
.save_setup = cmma_save_setup,
.save_live_iterate = cmma_save_iterate,
.save_live_complete_precopy = cmma_save_complete,
.save_complete = cmma_save_complete,
.state_pending_exact = cmma_state_pending,
.state_pending_estimate = cmma_state_pending,
.save_cleanup = cmma_save_cleanup,
hw/vfio/migration-multifd.c
+2 -2
View File
@@ -583,7 +583,7 @@ vfio_save_complete_precopy_thread_config_state(VFIODevice *vbasedev,
/*
* This thread is spawned by the migration core directly via
* .save_live_complete_precopy_thread SaveVMHandler.
* .save_complete_precopy_thread SaveVMHandler.
*
* It exits after either:
* * completing saving the remaining device state and device config, OR:
@@ -592,7 +592,7 @@ vfio_save_complete_precopy_thread_config_state(VFIODevice *vbasedev,
* multifd_device_state_save_thread_should_exit() returning true.
*/
bool
vfio_multifd_save_complete_precopy_thread(SaveLiveCompletePrecopyThreadData *d,
vfio_multifd_save_complete_precopy_thread(SaveCompletePrecopyThreadData *d,
Error **errp)
{
VFIODevice *vbasedev = d->handler_opaque;
hw/vfio/migration-multifd.h
+1 -1
View File
@@ -26,7 +26,7 @@ bool vfio_multifd_load_state_buffer(void *opaque, char *data, size_t data_size,
void vfio_multifd_emit_dummy_eos(VFIODevice *vbasedev, QEMUFile *f);
bool
vfio_multifd_save_complete_precopy_thread(SaveLiveCompletePrecopyThreadData *d,
vfio_multifd_save_complete_precopy_thread(SaveCompletePrecopyThreadData *d,
Error **errp);
int vfio_multifd_switchover_start(VFIODevice *vbasedev);
hw/vfio/migration.c
+2 -2
View File
@@ -824,7 +824,7 @@ static const SaveVMHandlers savevm_vfio_handlers = {
.state_pending_exact = vfio_state_pending_exact,
.is_active_iterate = vfio_is_active_iterate,
.save_live_iterate = vfio_save_iterate,
.save_live_complete_precopy = vfio_save_complete_precopy,
.save_complete = vfio_save_complete_precopy,
.save_state = vfio_save_state,
.load_setup = vfio_load_setup,
.load_cleanup = vfio_load_cleanup,
@@ -835,7 +835,7 @@ static const SaveVMHandlers savevm_vfio_handlers = {
*/
.load_state_buffer = vfio_multifd_load_state_buffer,
.switchover_start = vfio_switchover_start,
.save_live_complete_precopy_thread = vfio_multifd_save_complete_precopy_thread,
.save_complete_precopy_thread = vfio_multifd_save_complete_precopy_thread,
};
/* ---------------------------------------------------------------------- */
include/migration/misc.h
+4 -4
View File
@@ -119,19 +119,19 @@ bool migrate_uri_parse(const char *uri, MigrationChannel **channel,
Error **errp);
/* migration/multifd-device-state.c */
typedef struct SaveLiveCompletePrecopyThreadData {
SaveLiveCompletePrecopyThreadHandler hdlr;
typedef struct SaveCompletePrecopyThreadData {
SaveCompletePrecopyThreadHandler hdlr;
char *idstr;
uint32_t instance_id;
void *handler_opaque;
} SaveLiveCompletePrecopyThreadData;
} SaveCompletePrecopyThreadData;
bool multifd_queue_device_state(char *idstr, uint32_t instance_id,
char *data, size_t len);
bool multifd_device_state_supported(void);
void
multifd_spawn_device_state_save_thread(SaveLiveCompletePrecopyThreadHandler hdlr,
multifd_spawn_device_state_save_thread(SaveCompletePrecopyThreadHandler hdlr,
char *idstr, uint32_t instance_id,
void *opaque);
include/migration/register.h
+14 -22
View File
@@ -78,51 +78,43 @@ typedef struct SaveVMHandlers {
void (*save_cleanup)(void *opaque);
/**
* @save_live_complete_postcopy
*
* Called at the end of postcopy for all postcopyable devices.
*
* @f: QEMUFile where to send the data
* @opaque: data pointer passed to register_savevm_live()
*
* Returns zero to indicate success and negative for error
*/
int (*save_live_complete_postcopy)(QEMUFile *f, void *opaque);
/**
* @save_live_complete_precopy
* @save_complete
*
* Transmits the last section for the device containing any
* remaining data at the end of a precopy phase. When postcopy is
* enabled, devices that support postcopy will skip this step,
* where the final data will be flushed at the end of postcopy via
* @save_live_complete_postcopy instead.
* remaining data at the end phase of migration.
*
* For precopy, this will be invoked _during_ the switchover phase
* after source VM is stopped.
*
* For postcopy, this will be invoked _after_ the switchover phase
* (except some very unusual cases, like PMEM ramblocks), while
* destination VM can be running.
*
* @f: QEMUFile where to send the data
* @opaque: data pointer passed to register_savevm_live()
*
* Returns zero to indicate success and negative for error
*/
int (*save_live_complete_precopy)(QEMUFile *f, void *opaque);
int (*save_complete)(QEMUFile *f, void *opaque);
/**
* @save_live_complete_precopy_thread (invoked in a separate thread)
* @save_complete_precopy_thread (invoked in a separate thread)
*
* Called at the end of a precopy phase from a separate worker thread
* in configurations where multifd device state transfer is supported
* in order to perform asynchronous transmission of the remaining data in
* parallel with @save_live_complete_precopy handlers.
* parallel with @save_complete handlers.
* When postcopy is enabled, devices that support postcopy will skip this
* step.
*
* @d: a #SaveLiveCompletePrecopyThreadData containing parameters that the
* @d: a #SaveCompletePrecopyThreadData containing parameters that the
* handler may need, including this device section idstr and instance_id,
* and opaque data pointer passed to register_savevm_live().
* @errp: pointer to Error*, to store an error if it happens.
*
* Returns true to indicate success and false for errors.
*/
SaveLiveCompletePrecopyThreadHandler save_live_complete_precopy_thread;
SaveCompletePrecopyThreadHandler save_complete_precopy_thread;
/* This runs both outside and inside the BQL. */
include/qemu/typedefs.h
+3 -3
View File
@@ -109,7 +109,7 @@ typedef struct QString QString;
typedef struct RAMBlock RAMBlock;
typedef struct Range Range;
typedef struct ReservedRegion ReservedRegion;
typedef struct SaveLiveCompletePrecopyThreadData SaveLiveCompletePrecopyThreadData;
typedef struct SaveCompletePrecopyThreadData SaveCompletePrecopyThreadData;
typedef struct SHPCDevice SHPCDevice;
typedef struct SSIBus SSIBus;
typedef struct TCGCPUOps TCGCPUOps;
@@ -135,7 +135,7 @@ typedef struct IRQState *qemu_irq;
typedef void (*qemu_irq_handler)(void *opaque, int n, int level);
typedef bool (*MigrationLoadThread)(void *opaque, bool *should_quit,
Error **errp);
typedef bool (*SaveLiveCompletePrecopyThreadHandler)(SaveLiveCompletePrecopyThreadData *d,
Error **errp);
typedef bool (*SaveCompletePrecopyThreadHandler)(SaveCompletePrecopyThreadData *d,
Error **errp);
#endif /* QEMU_TYPEDEFS_H */
migration/block-dirty-bitmap.c
+1 -2
View File
@@ -1248,8 +1248,7 @@ static bool dirty_bitmap_has_postcopy(void *opaque)
static SaveVMHandlers savevm_dirty_bitmap_handlers = {
.save_setup = dirty_bitmap_save_setup,
.save_live_complete_postcopy = dirty_bitmap_save_complete,
.save_live_complete_precopy = dirty_bitmap_save_complete,
.save_complete = dirty_bitmap_save_complete,
.has_postcopy = dirty_bitmap_has_postcopy,
.state_pending_exact = dirty_bitmap_state_pending,
.state_pending_estimate = dirty_bitmap_state_pending,
migration/migration-hmp-cmds.c
+114 -45
View File
@@ -52,6 +52,88 @@ static void migration_global_dump(Monitor *mon)
ms->clear_bitmap_shift);
}
static const gchar *format_time_str(uint64_t us)
{
const char *units[] = {"us", "ms", "sec"};
int index = 0;
while (us > 1000) {
us /= 1000;
if (++index >= (sizeof(units) - 1)) {
break;
}
}
return g_strdup_printf("%"PRIu64" %s", us, units[index]);
}
static void migration_dump_blocktime(Monitor *mon, MigrationInfo *info)
{
if (info->has_postcopy_blocktime) {
monitor_printf(mon, "Postcopy Blocktime (ms): %" PRIu32 "\n",
info->postcopy_blocktime);
}
if (info->has_postcopy_vcpu_blocktime) {
uint32List *item = info->postcopy_vcpu_blocktime;
const char *sep = "";
int count = 0;
monitor_printf(mon, "Postcopy vCPU Blocktime (ms):\n [");
while (item) {
monitor_printf(mon, "%s%"PRIu32, sep, item->value);
item = item->next;
/* Each line 10 vcpu results, newline if there's more */
sep = ((++count % 10 == 0) && item) ? ",\n " : ", ";
}
monitor_printf(mon, "]\n");
}
if (info->has_postcopy_latency) {
monitor_printf(mon, "Postcopy Latency (ns): %" PRIu64 "\n",
info->postcopy_latency);
}
if (info->has_postcopy_non_vcpu_latency) {
monitor_printf(mon, "Postcopy non-vCPU Latencies (ns): %" PRIu64 "\n",
info->postcopy_non_vcpu_latency);
}
if (info->has_postcopy_vcpu_latency) {
uint64List *item = info->postcopy_vcpu_latency;
const char *sep = "";
int count = 0;
monitor_printf(mon, "Postcopy vCPU Latencies (ns):\n [");
while (item) {
monitor_printf(mon, "%s%"PRIu64, sep, item->value);
item = item->next;
/* Each line 10 vcpu results, newline if there's more */
sep = ((++count % 10 == 0) && item) ? ",\n " : ", ";
}
monitor_printf(mon, "]\n");
}
if (info->has_postcopy_latency_dist) {
uint64List *item = info->postcopy_latency_dist;
int count = 0;
monitor_printf(mon, "Postcopy Latency Distribution:\n");
while (item) {
g_autofree const gchar *from = format_time_str(1UL << count);
g_autofree const gchar *to = format_time_str(1UL << (count + 1));
monitor_printf(mon, " [ %8s - %8s ]: %10"PRIu64"\n",
from, to, item->value);
item = item->next;
count++;
}
}
}
void hmp_info_migrate(Monitor *mon, const QDict *qdict)
{
bool show_all = qdict_get_try_bool(qdict, "all", false);
@@ -69,7 +151,7 @@ void hmp_info_migrate(Monitor *mon, const QDict *qdict)
}
if (info->has_status) {
monitor_printf(mon, "Status: %s",
monitor_printf(mon, "Status: \t\t%s",
MigrationStatus_str(info->status));
if (info->status == MIGRATION_STATUS_FAILED && info->error_desc) {
monitor_printf(mon, " (%s)\n", info->error_desc);
@@ -78,7 +160,7 @@ void hmp_info_migrate(Monitor *mon, const QDict *qdict)
}
if (info->total_time) {
monitor_printf(mon, "Time (ms): total=%" PRIu64,
monitor_printf(mon, "Time (ms): \t\ttotal=%" PRIu64,
info->total_time);
if (info->has_setup_time) {
monitor_printf(mon, ", setup=%" PRIu64,
@@ -110,48 +192,51 @@ void hmp_info_migrate(Monitor *mon, const QDict *qdict)
}
if (info->ram) {
g_autofree char *str_psize = size_to_str(info->ram->page_size);
g_autofree char *str_total = size_to_str(info->ram->total);
g_autofree char *str_transferred = size_to_str(info->ram->transferred);
g_autofree char *str_remaining = size_to_str(info->ram->remaining);
g_autofree char *str_precopy = size_to_str(info->ram->precopy_bytes);
g_autofree char *str_multifd = size_to_str(info->ram->multifd_bytes);
g_autofree char *str_postcopy = size_to_str(info->ram->postcopy_bytes);
monitor_printf(mon, "RAM info:\n");
monitor_printf(mon, " Throughput (Mbps): %0.2f\n",
monitor_printf(mon, " Throughput (Mbps): \t%0.2f\n",
info->ram->mbps);
monitor_printf(mon, " Sizes (KiB): pagesize=%" PRIu64
", total=%" PRIu64 ",\n",
info->ram->page_size >> 10,
info->ram->total >> 10);
monitor_printf(mon, " transferred=%" PRIu64
", remain=%" PRIu64 ",\n",
info->ram->transferred >> 10,
info->ram->remaining >> 10);
monitor_printf(mon, " precopy=%" PRIu64
", multifd=%" PRIu64
", postcopy=%" PRIu64,
info->ram->precopy_bytes >> 10,
info->ram->multifd_bytes >> 10,
info->ram->postcopy_bytes >> 10);
monitor_printf(mon, " Sizes: \t\tpagesize=%s, total=%s\n",
str_psize, str_total);
monitor_printf(mon, " Transfers: \t\ttransferred=%s, remain=%s\n",
str_transferred, str_remaining);
monitor_printf(mon, " Channels: \t\tprecopy=%s, "
"multifd=%s, postcopy=%s",
str_precopy, str_multifd, str_postcopy);
if (info->vfio) {
monitor_printf(mon, ", vfio=%" PRIu64,
info->vfio->transferred >> 10);
g_autofree char *str_vfio = size_to_str(info->vfio->transferred);
monitor_printf(mon, ", vfio=%s", str_vfio);
}
monitor_printf(mon, "\n");
monitor_printf(mon, " Pages: normal=%" PRIu64 ", zero=%" PRIu64
", rate_per_sec=%" PRIu64 "\n",
info->ram->normal,
info->ram->duplicate,
monitor_printf(mon, " Page Types: \tnormal=%" PRIu64
", zero=%" PRIu64 "\n",
info->ram->normal, info->ram->duplicate);
monitor_printf(mon, " Page Rates (pps): \ttransfer=%" PRIu64,
info->ram->pages_per_second);
monitor_printf(mon, " Others: dirty_syncs=%" PRIu64,
info->ram->dirty_sync_count);
if (info->ram->dirty_pages_rate) {
monitor_printf(mon, ", dirty_pages_rate=%" PRIu64,
monitor_printf(mon, ", dirty=%" PRIu64,
info->ram->dirty_pages_rate);
}
monitor_printf(mon, "\n");
monitor_printf(mon, " Others: \t\tdirty_syncs=%" PRIu64,
info->ram->dirty_sync_count);
if (info->ram->postcopy_requests) {
monitor_printf(mon, ", postcopy_req=%" PRIu64,
info->ram->postcopy_requests);
}
if (info->ram->downtime_bytes) {
monitor_printf(mon, ", downtime_ram=%" PRIu64,
monitor_printf(mon, ", downtime_bytes=%" PRIu64,
info->ram->downtime_bytes);
}
if (info->ram->dirty_sync_missed_zero_copy) {
@@ -199,23 +284,7 @@ void hmp_info_migrate(Monitor *mon, const QDict *qdict)
info->dirty_limit_ring_full_time);
}
if (info->has_postcopy_blocktime) {
monitor_printf(mon, "Postcopy Blocktime (ms): %" PRIu32 "\n",
info->postcopy_blocktime);
}
if (info->has_postcopy_vcpu_blocktime) {
Visitor *v;
char *str;
v = string_output_visitor_new(false, &str);
visit_type_uint32List(v, NULL, &info->postcopy_vcpu_blocktime,
&error_abort);
visit_complete(v, &str);
monitor_printf(mon, "Postcopy vCPU Blocktime: %s\n", str);
g_free(str);
visit_free(v);
}
migration_dump_blocktime(mon, info);
out:
qapi_free_MigrationInfo(info);
}
migration/migration.c
+59 -28
View File
@@ -576,22 +576,27 @@ int migrate_send_rp_message_req_pages(MigrationIncomingState *mis,
}
int migrate_send_rp_req_pages(MigrationIncomingState *mis,
RAMBlock *rb, ram_addr_t start, uint64_t haddr)
RAMBlock *rb, ram_addr_t start, uint64_t haddr,
uint32_t tid)
{
void *aligned = (void *)(uintptr_t)ROUND_DOWN(haddr, qemu_ram_pagesize(rb));
bool received = false;
WITH_QEMU_LOCK_GUARD(&mis->page_request_mutex) {
received = ramblock_recv_bitmap_test_byte_offset(rb, start);
if (!received && !g_tree_lookup(mis->page_requested, aligned)) {
/*
* The page has not been received, and it's not yet in the page
* request list. Queue it. Set the value of element to 1, so that
* things like g_tree_lookup() will return TRUE (1) when found.
*/
g_tree_insert(mis->page_requested, aligned, (gpointer)1);
qatomic_inc(&mis->page_requested_count);
trace_postcopy_page_req_add(aligned, mis->page_requested_count);
if (!received) {
if (!g_tree_lookup(mis->page_requested, aligned)) {
/*
* The page has not been received, and it's not yet in the
* page request list. Queue it. Set the value of element
* to 1, so that things like g_tree_lookup() will return
* TRUE (1) when found.
*/
g_tree_insert(mis->page_requested, aligned, (gpointer)1);
qatomic_inc(&mis->page_requested_count);
trace_postcopy_page_req_add(aligned, mis->page_requested_count);
}
mark_postcopy_blocktime_begin(haddr, tid, rb);
}
}
@@ -3436,33 +3441,60 @@ static MigIterateState migration_iteration_run(MigrationState *s)
Error *local_err = NULL;
bool in_postcopy = s->state == MIGRATION_STATUS_POSTCOPY_ACTIVE;
bool can_switchover = migration_can_switchover(s);
bool complete_ready;
/* Fast path - get the estimated amount of pending data */
qemu_savevm_state_pending_estimate(&must_precopy, &can_postcopy);
pending_size = must_precopy + can_postcopy;
trace_migrate_pending_estimate(pending_size, must_precopy, can_postcopy);
if (pending_size < s->threshold_size) {
qemu_savevm_state_pending_exact(&must_precopy, &can_postcopy);
pending_size = must_precopy + can_postcopy;
trace_migrate_pending_exact(pending_size, must_precopy, can_postcopy);
if (in_postcopy) {
/*
* Iterate in postcopy until all pending data flushed. Note that
* postcopy completion doesn't rely on can_switchover, because when
* POSTCOPY_ACTIVE it means switchover already happened.
*/
complete_ready = !pending_size;
} else {
/*
* Exact pending reporting is only needed for precopy. Taking RAM
* as example, there'll be no extra dirty information after
* postcopy started, so ESTIMATE should always match with EXACT
* during postcopy phase.
*/
if (pending_size < s->threshold_size) {
qemu_savevm_state_pending_exact(&must_precopy, &can_postcopy);
pending_size = must_precopy + can_postcopy;
trace_migrate_pending_exact(pending_size, must_precopy,
can_postcopy);
}
/* Should we switch to postcopy now? */
if (must_precopy <= s->threshold_size &&
can_switchover && qatomic_read(&s->start_postcopy)) {
if (postcopy_start(s, &local_err)) {
migrate_set_error(s, local_err);
error_report_err(local_err);
}
return MIG_ITERATE_SKIP;
}
/*
* For precopy, migration can complete only if:
*
* (1) Switchover is acknowledged by destination
* (2) Pending size is no more than the threshold specified
* (which was calculated from expected downtime)
*/
complete_ready = can_switchover && (pending_size <= s->threshold_size);
}
if ((!pending_size || pending_size < s->threshold_size) && can_switchover) {
if (complete_ready) {
trace_migration_thread_low_pending(pending_size);
migration_completion(s);
return MIG_ITERATE_BREAK;
}
/* Still a significant amount to transfer */
if (!in_postcopy && must_precopy <= s->threshold_size && can_switchover &&
qatomic_read(&s->start_postcopy)) {
if (postcopy_start(s, &local_err)) {
migrate_set_error(s, local_err);
error_report_err(local_err);
}
return MIG_ITERATE_SKIP;
}
/* Just another iteration step */
qemu_savevm_state_iterate(s->to_dst_file, in_postcopy);
return MIG_ITERATE_RESUME;
@@ -3887,9 +3919,8 @@ static void *bg_migration_thread(void *opaque)
while (migration_is_active()) {
MigIterateState iter_state = bg_migration_iteration_run(s);
if (iter_state == MIG_ITERATE_SKIP) {
continue;
} else if (iter_state == MIG_ITERATE_BREAK) {
if (iter_state == MIG_ITERATE_BREAK) {
break;
}
migration/migration.h
+1 -1
View File
@@ -546,7 +546,7 @@ void migrate_send_rp_shut(MigrationIncomingState *mis,
void migrate_send_rp_pong(MigrationIncomingState *mis,
uint32_t value);
int migrate_send_rp_req_pages(MigrationIncomingState *mis, RAMBlock *rb,
ram_addr_t start, uint64_t haddr);
ram_addr_t start, uint64_t haddr, uint32_t tid);
int migrate_send_rp_message_req_pages(MigrationIncomingState *mis,
RAMBlock *rb, ram_addr_t start);
void migrate_send_rp_recv_bitmap(MigrationIncomingState *mis,
migration/multifd-device-state.c
+5 -5
View File
@@ -131,7 +131,7 @@ bool multifd_device_state_supported(void)
static void multifd_device_state_save_thread_data_free(void *opaque)
{
SaveLiveCompletePrecopyThreadData *data = opaque;
SaveCompletePrecopyThreadData *data = opaque;
g_clear_pointer(&data->idstr, g_free);
g_free(data);
@@ -139,7 +139,7 @@ static void multifd_device_state_save_thread_data_free(void *opaque)
static int multifd_device_state_save_thread(void *opaque)
{
SaveLiveCompletePrecopyThreadData *data = opaque;
SaveCompletePrecopyThreadData *data = opaque;
g_autoptr(Error) local_err = NULL;
if (!data->hdlr(data, &local_err)) {
@@ -170,18 +170,18 @@ bool multifd_device_state_save_thread_should_exit(void)
}
void
multifd_spawn_device_state_save_thread(SaveLiveCompletePrecopyThreadHandler hdlr,
multifd_spawn_device_state_save_thread(SaveCompletePrecopyThreadHandler hdlr,
char *idstr, uint32_t instance_id,
void *opaque)
{
SaveLiveCompletePrecopyThreadData *data;
SaveCompletePrecopyThreadData *data;
assert(multifd_device_state_supported());
assert(multifd_send_device_state);
assert(!qatomic_read(&multifd_send_device_state->threads_abort));
data = g_new(SaveLiveCompletePrecopyThreadData, 1);
data = g_new(SaveCompletePrecopyThreadData, 1);
data->hdlr = hdlr;
data->idstr = g_strdup(idstr);
data->instance_id = instance_id;
migration/options.c
+2
View File
@@ -187,6 +187,8 @@ const Property migration_properties[] = {
DEFINE_PROP_MIG_CAP("x-postcopy-ram", MIGRATION_CAPABILITY_POSTCOPY_RAM),
DEFINE_PROP_MIG_CAP("x-postcopy-preempt",
MIGRATION_CAPABILITY_POSTCOPY_PREEMPT),
DEFINE_PROP_MIG_CAP("postcopy-blocktime",
MIGRATION_CAPABILITY_POSTCOPY_BLOCKTIME),
DEFINE_PROP_MIG_CAP("x-colo", MIGRATION_CAPABILITY_X_COLO),
DEFINE_PROP_MIG_CAP("x-release-ram", MIGRATION_CAPABILITY_RELEASE_RAM),
DEFINE_PROP_MIG_CAP("x-return-path", MIGRATION_CAPABILITY_RETURN_PATH),
migration/postcopy-ram.c
+447 -124
View File
File diff suppressed because it is too large Load Diff
migration/postcopy-ram.h
+2
View File
@@ -196,5 +196,7 @@ void postcopy_preempt_new_channel(MigrationIncomingState *mis, QEMUFile *file);
void postcopy_preempt_setup(MigrationState *s);
int postcopy_preempt_establish_channel(MigrationState *s);
bool postcopy_is_paused(MigrationStatus status);
void mark_postcopy_blocktime_begin(uintptr_t addr, uint32_t ptid,
RAMBlock *rb);
#endif
migration/ram.c
+20 -12
View File
@@ -835,8 +835,10 @@ static inline bool migration_bitmap_clear_dirty(RAMState *rs,
* protections isn't needed as we know there will be either (1) no
* further writes if migration will complete, or (2) migration fails
* at last then tracking isn't needed either.
*
* Do the same for postcopy due to the same reason.
*/
if (!rs->last_stage) {
if (!rs->last_stage && !migration_in_postcopy()) {
/*
* Clear dirty bitmap if needed. This _must_ be called before we
* send any of the page in the chunk because we need to make sure
@@ -2286,16 +2288,18 @@ static int ram_find_and_save_block(RAMState *rs)
if (!get_queued_page(rs, pss)) {
/* priority queue empty, so just search for something dirty */
int res = find_dirty_block(rs, pss);
if (res != PAGE_DIRTY_FOUND) {
if (res == PAGE_ALL_CLEAN) {
break;
} else if (res == PAGE_TRY_AGAIN) {
continue;
} else if (res < 0) {
pages = res;
break;
}
if (res == PAGE_ALL_CLEAN) {
break;
} else if (res == PAGE_TRY_AGAIN) {
continue;
} else if (res < 0) {
pages = res;
break;
}
/* Otherwise we must have a dirty page to move */
assert(res == PAGE_DIRTY_FOUND);
}
pages = ram_save_host_page(rs, pss);
if (pages) {
@@ -3288,6 +3292,8 @@ static int ram_save_complete(QEMUFile *f, void *opaque)
RAMState *rs = *temp;
int ret = 0;
trace_ram_save_complete(rs->migration_dirty_pages, 0);
rs->last_stage = !migration_in_colo_state();
WITH_RCU_READ_LOCK_GUARD() {
@@ -3351,6 +3357,9 @@ static int ram_save_complete(QEMUFile *f, void *opaque)
}
qemu_put_be64(f, RAM_SAVE_FLAG_EOS);
trace_ram_save_complete(rs->migration_dirty_pages, 1);
return qemu_fflush(f);
}
@@ -4548,8 +4557,7 @@ void postcopy_preempt_shutdown_file(MigrationState *s)
static SaveVMHandlers savevm_ram_handlers = {
.save_setup = ram_save_setup,
.save_live_iterate = ram_save_iterate,
.save_live_complete_postcopy = ram_save_complete,
.save_live_complete_precopy = ram_save_complete,
.save_complete = ram_save_complete,
.has_postcopy = ram_has_postcopy,
.state_pending_exact = ram_state_pending_exact,
.state_pending_estimate = ram_state_pending_estimate,

Some files were not shown because too many files have changed in this diff Show More