Re: [PATCH v2 16/24] migration: Add capabilities into MigrationParameters

2025-07-04 Thread Peter Xu
on > and the new representation as members of MigrationParameters. > > Signed-off-by: Fabiano Rosas Acked-by: Peter Xu -- Peter Xu

Re: [PATCH v2 11/24] migration: Use QAPI_CLONE_MEMBERS in query_migrate_parameters

2025-07-04 Thread Peter Xu
gt; is added. > > For this to work, the has_* fields of s->parameters need to be already > set beforehand, so move migrate_mark_all_params_present() to the init > routine. > > Signed-off-by: Fabiano Rosas Reviewed-by: Peter Xu -- Peter Xu

Re: [PATCH v2 09/24] migration: Do away with usage of QERR_INVALID_PARAMETER_VALUE

2025-07-04 Thread Peter Xu
On Mon, Jun 30, 2025 at 04:58:58PM -0300, Fabiano Rosas wrote: > The QERR_INVALID_PARAMETER_VALUE macro is documented as not to be used > in new code. Remove the usage from migration/options.c. > > Signed-off-by: Fabiano Rosas Reviewed-by: Peter Xu -- Peter Xu

Re: [PATCH v2 07/24] migration: Add a flag to track block-bitmap-mapping input

2025-07-04 Thread Peter Xu
bject (s->parameters). > > Add a separate boolean to track the status of the block_bitmap_mapping > option. > > No functional change intended. > > (this was verified to not regress iotest 300, which is the test that > 3cba22c9ad refers to) > > CC: Kevin Wolf > Signed-off-by: Fabiano Rosas Acked-by: Peter Xu -- Peter Xu

Re: [PATCH v2 05/24] qapi/migration: Don't document MigrationParameter

2025-07-04 Thread Peter Xu
e > completion. > > Documenting this enum only serves to duplicate documentation between > MigrationParameter and MigrationParameters. > > Add an exception to QAPIs pragma.json and stop documenting it. > > Signed-off-by: Fabiano Rosas Acked-by: Peter Xu -- Peter Xu

Re: [PATCH v2 04/24] migration: Remove MigrateSetParameters

2025-07-04 Thread Peter Xu
ationParameters were replaced > with the ones from MigrateSetParameters which was more complete. > > Signed-off-by: Fabiano Rosas Acked-by: Peter Xu -- Peter Xu

Re: [PATCH v2 03/24] migration: Normalize tls arguments

2025-07-04 Thread Peter Xu
+*tls_opt = opt; Does tls_opt ever change? I wonder if this line is not needed, instead tls_opt_to_str() can take an "StrOrNull *opt" directly. > >>> +} > >> > >> I'd prefer something like > >> > >>if (!opt || opt->type == QTYPE_QSTRING) { > >>return; > >>} > >>qobject_unref(opt->u.n); > >>opt->type = QTYPE_QSTRING; > >>opt->u.s = g_strdup(""); > >>*tls_opt = opt; > >> > >> But this is clearly a matter of taste. > > This is better indeed. I was moving back-and-forth between > implementations and the code ended up a bit weird. Thanks! > -- Peter Xu

Re: [PATCH v2 02/24] migration: Add a qdev property for StrOrNull

2025-07-03 Thread Peter Xu
o->get) { <-- [1] return; } name = g_strdup_printf("legacy-%s", prop->name); object_class_property_add(OBJECT_CLASS(dc), name, "str", prop->info->print ? qdev_get_legacy_property : prop->info->get, NULL, NULL, (Property *)prop); } When with no get(), it seems it'll be wrongly treated as legacy property.. which further means whoever tries to get() on the property will invoke qdev_get_legacy_property(), and likely crash on accessing info->print().. The other issue is legacy property doesn't look like to provide a setter function.. as it's passing NULL to object_class_property_add(set=XXX). Likely we'll need to provide get() if without changing qdev code. > > > + > > bool migrate_auto_converge(void) > > { > > MigrationState *s = migrate_get_current(); > -- Peter Xu

Re: [PATCH V5 20/38] migration: close kvm after cpr

2025-07-03 Thread Peter Xu
On Thu, Jul 03, 2025 at 11:21:38PM +0200, Cédric Le Goater wrote: > On 7/3/25 21:45, Peter Xu wrote: > > On Wed, Jul 02, 2025 at 03:41:08PM -0400, Steven Sistare wrote: > > > The irq producer is not closed, but it is detached from the kvm consumer. > > > It's event

Re: [PATCH v2 01/24] migration: Fix leak of block_bitmap_mapping

2025-07-03 Thread Peter Xu
ration.c > > +++ b/migration/migration.c > > @@ -4050,6 +4050,7 @@ static void migration_instance_finalize(Object *obj) > > { > > MigrationState *ms = MIGRATION_OBJ(obj); > > > > + > > qapi_free_BitmapMigrationNodeAliasList(ms->parameters.block_bitmap_mapping); > > qemu_mutex_destroy

Re: [PATCH V5 20/38] migration: close kvm after cpr

2025-07-03 Thread Peter Xu
commit message, either when repost, or when merge, would be nice, on explaning irq won't get lost. Thanks, -- Peter Xu

Re: [PATCH V5 20/38] migration: close kvm after cpr

2025-07-02 Thread Peter Xu
misc.h" > > #include "migration/options.h" > > #include "migration/qemu-file.h" > > #include "migration/savevm.h" > > #include "migration/vmstate.h" > > +#include "system/kvm.h" > > #include "system/runstate.h" > > #include "trace.h" > > @@ -264,3 +266,9 @@ bool cpr_incoming_needed(void *opaque) > > MigMode mode = migrate_mode(); > > return mode == MIG_MODE_CPR_TRANSFER; > > } > > + > > +void cpr_kvm_close(void) > > +{ > > +kvm_close(); > > +vfio_kvm_device_close(); > > +} > > diff --git a/migration/migration.c b/migration/migration.c > > index 4098870..8f23cff 100644 > > --- a/migration/migration.c > > +++ b/migration/migration.c > > @@ -337,6 +337,7 @@ void migration_object_init(void) > > ram_mig_init(); > > dirty_bitmap_mig_init(); > > +cpr_transfer_init(); > > /* Initialize cpu throttle timers */ > > cpu_throttle_init(); > > diff --git a/hw/vfio/meson.build b/hw/vfio/meson.build > > index 73d29f9..98134a7 100644 > > --- a/hw/vfio/meson.build > > +++ b/hw/vfio/meson.build > > @@ -17,6 +17,8 @@ vfio_ss.add(when: 'CONFIG_VFIO_IGD', if_true: > > files('igd.c')) > > specific_ss.add_all(when: 'CONFIG_VFIO', if_true: vfio_ss) > > +system_ss.add(when: 'CONFIG_VFIO', if_false: files('vfio-stubs.c')) > > + > > system_ss.add(when: 'CONFIG_VFIO_XGMAC', if_true: > > files('calxeda-xgmac.c')) > > system_ss.add(when: 'CONFIG_VFIO_AMD_XGBE', if_true: files('amd-xgbe.c')) > > system_ss.add(when: 'CONFIG_VFIO', if_true: files( -- Peter Xu

Re: [PATCH] migration: Rename save_live_complete_precopy_thread to save_complete_precopy_thread

2025-06-26 Thread Peter Xu
ot;precopy" keyword is retained. > > [1]: https://lore.kernel.org/all/20250613140801.474264-7-pet...@redhat.com/ > > Signed-off-by: Juraj Marcin > --- > This patch is based on the following series: > https://lore.kernel.org/all/20250613140801.474264-1-pet...@redhat.com/ queued, thanks. -- Peter Xu

Re: [PATCH 1/3] migration: Pass error object to report it to the caller

2025-06-25 Thread Peter Xu
> >const char *suffix) > > { > > Error *err; > > int saved_errno = errno; > > > > if (errp == NULL) { > > return; > > } > > assert(*errp == NULL); > > ... > > } > > > > So here we can keep local_err but use error_propagate(). > > Please correct me if I am wrong, as far as I have checked, > qemu_loadvm_state() is called at 3 places > - load_snapshot > - qmp_xen_load_devices_state > - process_incoming_migration_co > and in all these functions, Error **errp is passed. > I did not find a function that passes NULL. > Is it still required to declare and pass a local_err object? In that case (this should include all the callers of the three call sites never passing in NULL) we're almost asserting errp!=NULL, then it should be OK. Thanks, -- Peter Xu

Re: [PATCH v3 00/11] migration: Some enhancements and cleanups for 10.1

2025-06-25 Thread Peter Xu
queued.

Re: [PATCH v3 00/14] migration/postcopy: Blocktime tracking overhaul

2025-06-25 Thread Peter Xu
On Fri, Jun 13, 2025 at 10:12:03AM -0400, Peter Xu wrote: > This series is based on the other series I posted here: queued. -- Peter Xu

Re: [PATCH v3 06/11] migration: Rename save_live_complete_precopy to save_complete

2025-06-25 Thread Peter Xu
On Wed, Jun 25, 2025 at 01:13:48PM +0200, Juraj Marcin wrote: > Sure, I will send it today. Thanks. In case it helps, can fetch head here: https://gitlab.com/peterx/qemu/-/tree/migration-staging -- Peter Xu

Re: [PATCH 1/3] migration: Pass error object to report it to the caller

2025-06-24 Thread Peter Xu
ading, &vmstate_versioned, &obj, 2, NULL); > g_assert_cmpint(obj.a, ==, 10); > g_assert_cmpint(obj.b, ==, 20); > g_assert_cmpint(obj.c, ==, 30); > @@ -480,7 +480,7 @@ static void test_load_noskip(void) > > QEMUFile *loading = open_test_file(false); > TestStruct obj = { .skip_c_e = false }; > -vmstate_load_state(loading, &vmstate_skipping, &obj, 2); > +vmstate_load_state(loading, &vmstate_skipping, &obj, 2, NULL); > g_assert(!qemu_file_get_error(loading)); > g_assert_cmpint(obj.a, ==, 10); > g_assert_cmpint(obj.b, ==, 20); > @@ -504,7 +504,7 @@ static void test_load_skip(void) > > QEMUFile *loading = open_test_file(false); > TestStruct obj = { .skip_c_e = true, .c = 300, .e = 500 }; > -vmstate_load_state(loading, &vmstate_skipping, &obj, 2); > +vmstate_load_state(loading, &vmstate_skipping, &obj, 2, NULL); > g_assert(!qemu_file_get_error(loading)); > g_assert_cmpint(obj.a, ==, 10); > g_assert_cmpint(obj.b, ==, 20); > @@ -773,7 +773,7 @@ static void test_load_q(void) > TestQtailq tgt; > > QTAILQ_INIT(&tgt.q); > -vmstate_load_state(fload, &vmstate_q, &tgt, 1); > +vmstate_load_state(fload, &vmstate_q, &tgt, 1, NULL); > char eof = qemu_get_byte(fload); > g_assert(!qemu_file_get_error(fload)); > g_assert_cmpint(tgt.i16, ==, obj_q.i16); > @@ -1127,7 +1127,7 @@ static void test_gtree_load_domain(void) > > fload = open_test_file(false); > > -vmstate_load_state(fload, &vmstate_domain, dest_domain, 1); > +vmstate_load_state(fload, &vmstate_domain, dest_domain, 1, NULL); > eof = qemu_get_byte(fload); > g_assert(!qemu_file_get_error(fload)); > g_assert_cmpint(orig_domain->id, ==, dest_domain->id); > @@ -1241,7 +1241,7 @@ static void test_gtree_load_iommu(void) > qemu_fclose(fsave); > > fload = open_test_file(false); > -vmstate_load_state(fload, &vmstate_iommu, dest_iommu, 1); > +vmstate_load_state(fload, &vmstate_iommu, dest_iommu, 1, NULL); > eof = qemu_get_byte(fload); > g_assert(!qemu_file_get_error(fload)); > g_assert_cmpint(orig_iommu->id, ==, dest_iommu->id); > @@ -1376,7 +1376,7 @@ static void test_load_qlist(void) > qemu_fclose(fsave); > > fload = open_test_file(false); > -vmstate_load_state(fload, &vmstate_container, dest_container, 1); > +vmstate_load_state(fload, &vmstate_container, dest_container, 1, NULL); > eof = qemu_get_byte(fload); > g_assert(!qemu_file_get_error(fload)); > g_assert_cmpint(eof, ==, QEMU_VM_EOF); > > -- > 2.49.0 > -- Peter Xu

Re: [PATCH 2/3] migration: Use error_setg instead of error_report

2025-06-24 Thread Peter Xu
; -error_report("%s: incoming version_id %d is too old " > - "for local minimum version_id %d", > - vmsd->name, version_id, vmsd->minimum_version_id); > +error_setg(errp, "%s: incoming version_id %d is too old " > + "for local minimum version_id %d", > + vmsd->name, version_id, vmsd->minimum_version_id); > trace_vmstate_load_state_end(vmsd->name, "too old", -EINVAL); > return -EINVAL; > } > @@ -213,15 +213,15 @@ int vmstate_load_state(QEMUFile *f, const > VMStateDescription *vmsd, > } > if (ret < 0) { > qemu_file_set_error(f, ret); > -error_report("Failed to load %s:%s", vmsd->name, > - field->name); > +error_setg(errp, "Failed to load %s:%s", vmsd->name, > + field->name); > trace_vmstate_load_field_error(field->name, ret); > return ret; > } > } > } else if (field->flags & VMS_MUST_EXIST) { > -error_report("Input validation failed: %s/%s", > - vmsd->name, field->name); > +error_setg(errp, "Input validation failed: %s/%s", > + vmsd->name, field->name); > return -1; > } > field++; > > -- > 2.49.0 > -- Peter Xu

Re: [PATCH v3 06/11] migration: Rename save_live_complete_precopy to save_complete

2025-06-24 Thread Peter Xu
) until > docs/devel/migration/main.rst:511 > - A ``save_live_complete_precopy`` function that must transmit the Good catch. If I prepare this fixup to be squashed, can I get your R-b? >From 58147b11276fa193c25f35e63f41a8e34d444dd9 Mon Sep 17 00:00:00 2001 From: Peter Xu D

[PULL 03/11] ui/vnc: Update display update interval when VM state changes to RUNNING

2025-06-23 Thread Peter Xu
timer, and the display is refreshed immediately if the timer is expired. Signed-off-by: Juraj Marcin Reviewed-by: Marc-André Lureau Reviewed-by: Peter Xu Reviewed-by: Daniel P. Berrangé Link: https://lore.kernel.org/r/20250521151616.3951178-1-jmar...@redhat.com Signed-off-by: Peter Xu --- ui

[PULL 10/11] ram-block-attributes: Introduce RamBlockAttributes to manage RAMBlock with guest_memfd

2025-06-23 Thread Peter Xu
d. Tested-by: Alexey Kardashevskiy Reviewed-by: Alexey Kardashevskiy Reviewed-by: Pankaj Gupta Signed-off-by: Chenyi Qiang Link: https://lore.kernel.org/r/20250612082747.51539-5-chenyi.qi...@intel.com [peterx: squash fixup from Chenyi to fix builds] Signed-off-by: Peter Xu --- M

[PULL 05/11] tests/migration: Setup pre-listened cpr.sock to remove race-condition.

2025-06-23 Thread Peter Xu
om [peterx: null-initialize opts_target, per Steve] Signed-off-by: Peter Xu --- tests/qtest/migration/cpr-tests.c | 14 -- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/tests/qtest/migration/cpr-tests.c b/tests/qtest/migration/cpr-tests.c index 5536e14610..5e764a6787 10

[PULL 09/11] memory: Unify the definiton of ReplayRamPopulate() and ReplayRamDiscard()

2025-06-23 Thread Peter Xu
makes it cleaner. Reviewed-by: David Hildenbrand Reviewed-by: Pankaj Gupta Reviewed-by: Xiaoyao Li Signed-off-by: Chenyi Qiang Link: https://lore.kernel.org/r/20250612082747.51539-4-chenyi.qi...@intel.com Signed-off-by: Peter Xu --- include/system/memory.h | 74

[PULL 00/11] Migration staging patches

2025-06-23 Thread Peter Xu
The following changes since commit 43ba160cb4bbb193560eb0d2d7decc4b5fc599fe: Merge tag 'for-upstream' of https://gitlab.com/bonzini/qemu into staging (2025-06-21 10:00:46 -0400) are available in the Git repository at: https://gitlab.com/peterx/qemu.git tags/migration-staging-pull-request f

[PULL 08/11] memory: Change memory_region_set_ram_discard_manager() to return the result

2025-06-23 Thread Peter Xu
: Peter Xu --- include/system/memory.h | 6 +++--- hw/virtio/virtio-mem.c | 30 +- system/memory.c | 10 +++--- 3 files changed, 27 insertions(+), 19 deletions(-) diff --git a/include/system/memory.h b/include/system/memory.h index da97753e28..60983d4977

[PULL 11/11] physmem: Support coordinated discarding of RAM with guest_memfd

2025-06-23 Thread Peter Xu
org/r/20250612082747.51539-6-chenyi.qi...@intel.com Signed-off-by: Peter Xu --- include/system/ramblock.h | 1 + accel/kvm/kvm-all.c | 9 + system/physmem.c | 23 +-- 3 files changed, 31 insertions(+), 2 deletions(-) diff --git a/include/system/rambloc

[PULL 06/11] migration: Don't sync volatile memory after migration completes

2025-06-23 Thread Peter Xu
ram block writeback) Link: https://lore.kernel.org/r/1cc43f59-336f-4a12-84ad-db89e0a17...@akamai.com Signed-off-by: Peter Xu --- migration/ram.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/migration/ram.c b/migration/ram.c index fd8d83b63c..024d8b3d03 100644 --- a/migra

[PULL 07/11] memory: Export a helper to get intersection of a MemoryRegionSection with a given range

2025-06-23 Thread Peter Xu
: Alexey Kardashevskiy Reviewed-by: Pankaj Gupta Reviewed-by: David Hildenbrand Reviewed-by: Zhao Liu Reviewed-by: Xiaoyao Li Signed-off-by: Chenyi Qiang Link: https://lore.kernel.org/r/20250612082747.51539-2-chenyi.qi...@intel.com Signed-off-by: Peter Xu --- include/system/memory.h | 30

[PULL 04/11] migration: Support fd-based socket address in cpr_transfer_input

2025-06-23 Thread Peter Xu
created externally and passed via fd. Reviewed-by: Jason J. Herne Reviewed-by: Steve Sistare Signed-off-by: Jaehoon Kim Link: https://lore.kernel.org/r/20250611205610.147008-3-jh...@linux.ibm.com Signed-off-by: Peter Xu --- migration/cpr-transfer.c | 7 +-- 1 file changed, 5 insertions(+), 2

[PULL 02/11] tests/qtest: Remove migration-helpers.c

2025-06-23 Thread Peter Xu
qmp/ to include/qobject/") Signed-off-by: Fabiano Rosas Message-id: 20200310152141.13959-1-peter.mayd...@linaro.org Reviewed-by: Markus Armbruster Link: https://lore.kernel.org/r/20250523123023.19284-1-faro...@suse.de Signed-off-by: Peter Xu --- tests/qtest/migra

[PULL 01/11] migration/ram: avoid to do log clear in the last round

2025-06-23 Thread Peter Xu
y: Fabiano Rosas Link: https://lore.kernel.org/r/20250514115827.3216082-1-yanfei...@bytedance.com [peterx: add comments] Signed-off-by: Peter Xu --- migration/ram.c | 22 +++--- 1 file changed, 15 insertions(+), 7 deletions(-) diff --git a/migration/ram.c b/migration/ram.c index

Re: [PATCH 1/3] memory: reintroduce BQL-free fine-grained PIO/MMIO

2025-06-23 Thread Peter Xu
On Mon, Jun 23, 2025 at 02:51:46PM +0200, Igor Mammedov wrote: > On Fri, 20 Jun 2025 12:53:06 -0400 > Peter Xu wrote: > > > On Fri, Jun 20, 2025 at 05:14:16PM +0200, Igor Mammedov wrote: > > > This patch brings back Jan's idea [1] of BQL-free IO access, > >

Re: [PATCH 0/2] Memory and PCI definitions for emulated ATS

2025-06-23 Thread Peter Xu
On Mon, Jun 23, 2025 at 05:43:06AM +, CLEMENT MATHIEU--DRIF wrote: > Hi Peter > > On 20/06/2025 4:35 pm, Peter Xu wrote: > > Caution: External email. Do not open attachments or click links, unless > > this email comes from a known sender and you know the content is safe.

Re: [PATCH 1/3] memory: reintroduce BQL-free fine-grained PIO/MMIO

2025-06-20 Thread Peter Xu
> +!(read && mr->lockless_ro_io == true)) { > bql_lock(); > release_lock = true; > } > @@ -2935,7 +2936,7 @@ static MemTxResult > flatview_write_continue_step(MemTxAttrs attrs, > if (!memory_access_is_direct(mr, true, attrs)) { > uint64_t val; > MemTxResult result; > -bool release_lock = prepare_mmio_access(mr); > +bool release_lock = prepare_mmio_access(mr, false); > > *l = memory_access_size(mr, *l, mr_addr); > /* > @@ -3032,7 +3033,7 @@ static MemTxResult > flatview_read_continue_step(MemTxAttrs attrs, uint8_t *buf, > /* I/O case */ > uint64_t val; > MemTxResult result; > -bool release_lock = prepare_mmio_access(mr); > +bool release_lock = prepare_mmio_access(mr, true); > > *l = memory_access_size(mr, *l, mr_addr); > result = memory_region_dispatch_read(mr, mr_addr, &val, > size_memop(*l), > -- > 2.43.5 > -- Peter Xu

Re: [PATCH 0/2] Memory and PCI definitions for emulated ATS

2025-06-20 Thread Peter Xu
issions in IOMMUAccessFlags > > include/exec/memattrs.h | 3 +++ > include/hw/pci/pci.h| 9 + > include/system/memory.h | 23 +++++++++-- > 3 files changed, 33 insertions(+), 2 deletions(-) > > -- > 2.49.0 > -- Peter Xu

Re: [PATCH v7 3/5] memory: Unify the definiton of ReplayRamPopulate() and ReplayRamDiscard()

2025-06-19 Thread Peter Xu
o 6 failures (and you can ignore the rust warnings): https://gitlab.com/peterx/qemu/-/pipelines/1878653850 WASM complains a bit more than others, but worth double check from your side too. Thanks, -- Peter Xu

Re: [PATCH v7 0/5] Enable shared device assignment

2025-06-18 Thread Peter Xu
est with kernel-doc format. If you want, you can provide "git --fixup" appended to the reply (one fixup for each patch that needs fixing) to avoid a full repost. Thanks, -- Peter Xu

Re: [PATCH v7 0/5] Enable shared device assignment

2025-06-18 Thread Peter Xu
it's just that it'll be a smaller portion to be shared. Other than that, even if I don't know how to test this.. I read the patches today and they look all good. The duplication is a pure question I have above, but even if so it can also be done on top. I do plan to pick this up. Paolo/David, any comments before I do? Thanks, -- Peter Xu

Re: [Patch v2] Don't sync volatile memory

2025-06-16 Thread Peter Xu
k_writeback(rb); > +if (memory_region_is_nonvolatile(rb->mr)) { > +qemu_ram_block_writeback(rb); > + } > } > > xbzrle_load_cleanup(); > -- > 2.40.1 > -- Peter Xu

Re: [Patch] Don't sync volatile memory

2025-06-16 Thread Peter Xu
lock_writeback(rb); > +} > } > > xbzrle_load_cleanup(); > -- > 2.40.1 > > > > -- Peter Xu

[PATCH v3 02/14] migration/postcopy: Push blocktime start/end into page req mutex

2025-06-13 Thread Peter Xu
ayer. Note that we need a stub for mark_postcopy_blocktime_begin() for Windows builds. Reviewed-by: Fabiano Rosas Signed-off-by: Peter Xu --- migration/migration.h| 2 +- migration/postcopy-ram.h | 2 ++ migration/migration.c| 25 +++--- migration/postcopy-ram.c | 56 ++

Re: [PATCH] migration/postcopy: Add latency distribution report for blocktime

2025-06-13 Thread Peter Xu
On Mon, Jun 09, 2025 at 06:36:07PM -0400, Peter Xu wrote: > Add the latency distribution too for blocktime, using order-of-two buckets. > It accounts for all the faults, from either vCPU or non-vCPU threads. With > prior rework, it's very easy to achieve by adding an array to accoun

[PATCH v3 07/14] migration/postcopy: Add blocktime fault counts per-vcpu

2025-06-13 Thread Peter Xu
Add a field to count how many remote faults one vCPU has taken. So far it's still not used, but will be soon. Reviewed-by: Fabiano Rosas Signed-off-by: Peter Xu --- migration/postcopy-ram.c | 5 + 1 file changed, 5 insertions(+) diff --git a/migration/postcopy-ram.c b/migration/pos

[PATCH v3 05/14] migration/postcopy: Drop PostcopyBlocktimeContext.start_time

2025-06-13 Thread Peter Xu
Now with 64bits, the offseting using start_time is not needed anymore, because the array can always remember the whole timestamp. Then drop the unused parameter in get_low_time_offset() altogether. Reviewed-by: Fabiano Rosas Signed-off-by: Peter Xu --- migration/postcopy-ram.c | 10

[PATCH v3 00/14] migration/postcopy: Blocktime tracking overhaul

2025-06-13 Thread Peter Xu
vCPUs especially after this rewrite. If add preempt mode into picture, I think one should enable all three features for postcopy by default at some point. Comments welcomed, thanks. Peter Xu (14): migration: Add option to set postcopy-blocktime migration/postcopy: Push blocktime start/end int

[PATCH v3 11/11] migration/postcopy: Avoid clearing dirty bitmap for postcopy too

2025-06-13 Thread Peter Xu
with this optimization: Before: 268.00us (+-1.87%) After: 232.67us (+-2.01%) The test was done with a 16GB VM with 80 vCPUs, running a workload that busy random writes to 13GB memory. Cc: Yanfei Xu Reviewed-by: Fabiano Rosas Signed-off-by: Peter Xu --- migration/ram.c | 4 +++- 1 file

[PATCH v3 01/14] migration: Add option to set postcopy-blocktime

2025-06-13 Thread Peter Xu
Add a global property to allow enabling postcopy-blocktime feature. Reviewed-by: Fabiano Rosas Signed-off-by: Peter Xu --- migration/options.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/migration/options.c b/migration/options.c index 162c72cda4..4e923a2e07 100644 --- a/migration

[PATCH v3 12/14] migration/postcopy: Optimize blocktime fault tracking with hashtable

2025-06-13 Thread Peter Xu
. One tracepoint is added when walking all the fault entries. Reviewed-by: Fabiano Rosas Signed-off-by: Peter Xu --- migration/postcopy-ram.c | 259 --- migration/trace-events | 5 +- 2 files changed, 216 insertions(+), 48 deletions(-) diff --git a/migration/p

[PATCH v3 09/14] migration/postcopy: Initialize blocktime context only until listen

2025-06-13 Thread Peter Xu
;t created instead this time; the old "if" trick isn't needed when we're sure it will only happen once now. Reviewed-by: Fabiano Rosas Signed-off-by: Peter Xu --- migration/postcopy-ram.c | 15 ++- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git

[PATCH v3 03/14] migration/postcopy: Drop all atomic ops in blocktime feature

2025-06-13 Thread Peter Xu
Now with the mutex protection it's not needed anymore. Reviewed-by: Fabiano Rosas Signed-off-by: Peter Xu --- migration/postcopy-ram.c | 23 ++- 1 file changed, 10 insertions(+), 13 deletions(-) diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c

[PATCH v3 11/14] migration/postcopy: Cleanup the total blocktime accounting

2025-06-13 Thread Peter Xu
e variable at all: since nobody is touching smp_cpus_down except ourselves, we can safely do the calculation at the end before decrementing smp_cpus_down. Hopefully this makes the logic easier to read, side benefit is we drop one temp var. Reviewed-by: Fabiano Rosas Signed-off-by: Peter Xu ---

[PATCH v3 04/14] migration/postcopy: Make all blocktime vars 64bits

2025-06-13 Thread Peter Xu
cktime_start. - Rename vcpu_blocktime to vcpu_blocktime_total. - Touch up the trace-events to not dump blocktime ctx pointer Reviewed-by: Fabiano Rosas Signed-off-by: Peter Xu --- migration/postcopy-ram.c | 50 migration/trace-events | 4 ++-- 2 files chan

[PATCH v3 13/14] migration/postcopy: blocktime allows track / report non-vCPU faults

2025-06-13 Thread Peter Xu
ed". Cc: Markus Armbruster Cc: Dr. David Alan Gilbert Reviewed-by: Fabiano Rosas Tested-by: Mario Casquero Signed-off-by: Peter Xu --- qapi/migration.json | 12 - migration/migration-hmp-cmds.c| 5 +++ migration/postcopy-ram.c | 64 ++

[PATCH v3 08/14] migration/postcopy: Report fault latencies in blocktime

2025-06-13 Thread Peter Xu
get_vcpu_blocktime_list() so we don't need to walk the list twice, meanwhile add the entry checks in qtests for all postcopy tests. Cc: Markus Armbruster Cc: Dr. David Alan Gilbert Reviewed-by: Fabiano Rosas Tested-by: Mario Casquero Signed-off-by: Peter Xu --- qapi/migration

[PATCH v3 10/14] migration/postcopy: Cache the tid->vcpu mapping for blocktime

2025-06-13 Thread Peter Xu
access a guest page that was missing. These kind of faults are not accounted by blocktime so far. Reviewed-by: Fabiano Rosas Signed-off-by: Peter Xu --- migration/postcopy-ram.c | 68 ++-- migration/trace-events | 3 +- 2 files changed, 59 insertions(+), 12 d

[PATCH v3 06/14] migration/postcopy: Bring blocktime layer to ns level

2025-06-13 Thread Peter Xu
With 64-bit fields, it is trivial. The caution is when exposing any values in QMP, it was still declared with milliseconds (ms). Hence it's needed to do the convertion when exporting the values to existing QMP queries. Reviewed-by: Fabiano Rosas Signed-off-by: Peter Xu --- migration/pos

[PATCH v3 14/14] migration/postcopy: Add latency distribution report for blocktime

2025-06-13 Thread Peter Xu
sec -8 sec ]: 0 [8 sec - 16 sec ]: 0 Cc: Markus Armbruster Acked-by: Dr. David Alan Gilbert Reviewed-by: Fabiano Rosas Signed-off-by: Peter Xu --- qapi/migration.json | 12 ++- migration/migration-hmp-cmds.c| 32 +++

[PATCH v3 04/11] migration/bg-snapshot: Do not check for SKIP in iterator

2025-06-13 Thread Peter Xu
It's not possible to happen in bg-snapshot case. Reviewed-by: Juraj Marcin Signed-off-by: Peter Xu --- migration/migration.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/migration/migration.c b/migration/migration.c index 4098870bce..e33e39ac74 100644

[PATCH v3 10/11] migration: Rewrite the migration complete detect logic

2025-06-13 Thread Peter Xu
ses (precopy v.s. postcopy). (4) Trivial touch up on threshold_size comparision Which changes: "(!pending_size || pending_size < s->threshold_size)" into: "(pending_size <= s->threshold_size)" Reviewed-by: Juraj Marcin Signed-off-by: Peter

[PATCH v3 00/11] migration: Some enhancements and cleanups for 10.1

2025-06-13 Thread Peter Xu
welcomed, thanks. Peter Xu (11): migration/hmp: Reorg "info migrate" once more migration/hmp: Fix postcopy-blocktime per-vCPU results migration/docs: Move docs for postcopy blocktime feature migration/bg-snapshot: Do not check for SKIP in iterator migration: Drop save_live_complete

[PATCH v3 01/11] migration/hmp: Reorg "info migrate" once more

2025-06-13 Thread Peter Xu
n Gilbert Tested-by: Li Zhijian Reviewed-by: Li Zhijian Acked-by: Dr. David Alan Gilbert Reviewed-by: Juraj Marcin Tested-by: Mario Casquero Signed-off-by: Peter Xu --- migration/migration-hmp-cmds.c | 59 ++ 1 file changed, 31 insertions(+), 28 deletions(-)

[PATCH v3 03/11] migration/docs: Move docs for postcopy blocktime feature

2025-06-13 Thread Peter Xu
Move it out of vanilla postcopy session, but instead a standalone feature. When at it, removing the NOTE because it's incorrect now after introduction of max-postcopy-bandwidth, which can control the throughput even for postcopy phase. Reviewed-by: Juraj Marcin Signed-off-by: Peter Xu ---

[PATCH v3 09/11] migration/ram: Add tracepoints for ram_save_complete()

2025-06-13 Thread Peter Xu
Take notes on start/end state of dirty pages for the whole system. Reviewed-by: Juraj Marcin Signed-off-by: Peter Xu --- migration/ram.c| 5 + migration/trace-events | 1 + 2 files changed, 6 insertions(+) diff --git a/migration/ram.c b/migration/ram.c index c66ad3cf8b..a1d0e8ada2

[PATCH v3 06/11] migration: Rename save_live_complete_precopy to save_complete

2025-06-13 Thread Peter Xu
Signed-off-by: Peter Xu --- include/migration/register.h | 4 ++-- hw/ppc/spapr.c | 2 +- hw/s390x/s390-stattrib.c | 2 +- hw/vfio/migration.c| 2 +- migration/block-dirty-bitmap.c | 2 +- migration/ram.c| 2 +- migration/savevm.c

[PATCH v3 05/11] migration: Drop save_live_complete_postcopy hook

2025-06-13 Thread Peter Xu
The hook is only defined in two vmstate users ("ram" and "block dirty bitmap"), meanwhile both of them define the hook exactly the same as the precopy version. Hence, this postcopy version isn't needed. No functional change intended. Signed-off-by: Peter Xu --- inc

[PATCH v3 07/11] migration: qemu_savevm_complete*() helpers

2025-06-13 Thread Peter Xu
Since we use the same save_complete() hook for both precopy and postcopy, add a set of helpers to invoke the hook() to dedup the code. Signed-off-by: Peter Xu --- migration/savevm.c | 78 ++ 1 file changed, 44 insertions(+), 34 deletions(-) diff

[PATCH v3 08/11] migration/ram: One less indent for ram_find_and_save_block()

2025-06-13 Thread Peter Xu
The check over PAGE_DIRTY_FOUND isn't necessary. We could indent one less and assert that instead. Reviewed-by: Juraj Marcin Signed-off-by: Peter Xu --- migration/ram.c | 20 +++- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/migration/ram.c b/migration/

[PATCH v3 02/11] migration/hmp: Fix postcopy-blocktime per-vCPU results

2025-06-13 Thread Peter Xu
52, 50, 38, 40, 37, 40, 49, 40, 35, 35, 35, 81, 19, 18, 19, 18, 30, 22, 3, 0, 0, 0, 0, 0, 0, 0, 0] Cc: Dr. David Alan Gilbert Cc: Alexey Perevalov Cc: Markus Armbruster Tested-by: Mario Casquero Signed-off-by: Peter Xu --- migration/migration-hmp-cmds.c | 22 +- 1 fi

Re: [PATCH 10/21] migration: Use QAPI_CLONE_MEMBERS in query_migrate_parameters

2025-06-12 Thread Peter Xu
On Thu, Jun 12, 2025 at 05:58:14PM -0300, Fabiano Rosas wrote: > Fabiano Rosas writes: > > > Peter Xu writes: > > > >> On Mon, Jun 02, 2025 at 10:37:59PM -0300, Fabiano Rosas wrote: > >>> QAPI_CLONE_MEMBERS is a better option than copying parameters one

Re: live-migration performance regression when using pmem

2025-06-12 Thread Peter Xu
x1.local/ But I guess I got the use case wrong; looks like it worked for you. In that case please switch to memory_region_is_nonvolatile(), and add proper Fixes, and copy stable. Thanks, -- Peter Xu

Re: [PATCH v4 2/2] migration: Support fd-based socket address in cpr_transfer_input

2025-06-12 Thread Peter Xu
fixed both issues and queued the series, thanks all. For the other patch I did null-initialize for the var. https://gitlab.com/peterx/qemu/-/commits/migration-staging -- Peter Xu

Re: [PATCH v2 00/11] migration: Some enhancements and cleanups for 10.1

2025-06-12 Thread Peter Xu
en revert the last patch of 1st series, as "postcopy-latency' metrics wasn't around before applying the 2nd series, or you'll need to use some kernel tracepoints. This is definitely an awkward series to test when having the two mangled. Again, feel free to skip that, just FYI! Thanks, -- Peter Xu

Re: [PATCH v2 00/11] migration: Some enhancements and cleanups for 10.1

2025-06-11 Thread Peter Xu
On Mon, Jun 09, 2025 at 12:18:44PM -0400, Peter Xu wrote: > v2: > - Collected R-bs > - Avoid using "\b" in HMP dumps [Markus, Dave] > > The series is based on a small patch from Yanfei Xu here: > > Based-on: <20250514115827.3216082-1-yanfei...@bytedanc

Re: [PATCH V5 00/38] Live update: vfio and iommufd

2025-06-11 Thread Peter Xu
On Wed, Jun 11, 2025 at 04:25:13PM +0200, Cédric Le Goater wrote: > Peter, Fabiano, > > The first 2 patches are migration patches. Do you agree if I take them > through the VFIO queue ? Yep, please go ahead. -- Peter Xu

Re: [PATCH] ui/vnc: Update display update interval when VM state changes to RUNNING

2025-06-11 Thread Peter Xu
On Wed, Jun 11, 2025 at 04:41:21PM +0400, Marc-André Lureau wrote: > Hi > > On Wed, Jun 11, 2025 at 4:34 PM Peter Xu wrote: > > > On Wed, May 21, 2025 at 05:16:13PM +0200, Juraj Marcin wrote: > > > From: Juraj Marcin > > > > > > If a virtual machi

Re: [PATCH v2 00/11] migration: Some enhancements and cleanups for 10.1

2025-06-11 Thread Peter Xu
meat. Note that this is not a request to have you test everything! Just to mention the bits from test perspective, so just take it as FYI. I appreciate your help already to test on the recv side! Thanks, -- Peter Xu

Re: [PATCH] ui/vnc: Update display update interval when VM state changes to RUNNING

2025-06-11 Thread Peter Xu
nges > and changing the update interval when the VM state changes to RUNNING. > The update_displaychangelistener() function updates the internal timer, > and the display is refreshed immediately if the timer is expired. > > Signed-off-by: Juraj Marcin [add Dan] Ping - anyone is willing to pick this one up? Thanks, -- Peter Xu

Re: [PATCH 19/21] migration: Allow migrate commands to provide the migration config

2025-06-10 Thread Peter Xu
t; g) config not touching the parameter, value is reset to default: > > { 'execute': 'migrate', 'arguments': { ..., 'config': { 'multifd': true } } } > => "multifd-channels": 2, HMP: multifd-channels: 2 > > h) config overrides all: > > { 'execute': 'migrate', 'arguments': { ..., 'config': {'multifd-channels': > 16 } } } > => QMP: "multifd-channels": 16, HMP: multifd-channels: 16 > > I'll update the variable names and code comments to be more > precise. Sorry for the noise. Good to know it's even working. Thanks for digging it. -- Peter Xu

Re: [PATCH v2 08/13] migration/postcopy: Report fault latencies in blocktime

2025-06-10 Thread Peter Xu
On Tue, Jun 10, 2025 at 01:53:40PM +, Dr. David Alan Gilbert wrote: > * Peter Xu (pet...@redhat.com) wrote: > > On Tue, Jun 10, 2025 at 12:08:23AM +, Dr. David Alan Gilbert wrote: > > > > diff --git a/qapi/migration.json b/qapi/migration.json > > > > in

Re: [PATCH v2] migration: Setup pre-listened cpr.sock to remove race-condition.

2025-06-10 Thread Peter Xu
ick comment while we can wait for others to look at the details: when it involves both qemu and tests changes, please consider splitting that into two patches. The test patch can be prefixed with "tests/migration:". Thanks, -- Peter Xu

Re: [PATCH v2 08/13] migration/postcopy: Report fault latencies in blocktime

2025-06-10 Thread Peter Xu
es in the report, maybe I should also mark all of these fields experimental from the start? So we don't necessarily need to maintain the ABI - the expectation is even if a mgmt would like to fetch those they should only fetch and dump it into log so that human can read later only for debugging purposes. -- Peter Xu

[PATCH] migration/postcopy: Add latency distribution report for blocktime

2025-06-09 Thread Peter Xu
sec -8 sec ]: 0 [8 sec - 16 sec ]: 0 Cc: Dr. David Alan Gilbert Cc: Markus Armbruster Signed-off-by: Peter Xu --- This patch is based on: [PATCH v2 00/13] migration/postcopy: Blocktime tracking overhaul Based-on: <20250609191259.9053-1-pet...@redhat.com>

Re: [PATCH v2 08/13] migration/postcopy: Report fault latencies in blocktime

2025-06-09 Thread Peter Xu
On Mon, Jun 09, 2025 at 06:05:16PM -0400, Peter Xu wrote: > On Mon, Jun 09, 2025 at 03:12:54PM -0400, Peter Xu wrote: > > +static void migration_dump_blocktime(Monitor *mon, MigrationInfo *info) > > +{ > > +if (info->has_postcopy_blocktime) { > > +

[PATCH v2 10/13] migration/postcopy: Cache the tid->vcpu mapping for blocktime

2025-06-09 Thread Peter Xu
access a guest page that was missing. These kind of faults are not accounted by blocktime so far. Reviewed-by: Fabiano Rosas Signed-off-by: Peter Xu --- migration/postcopy-ram.c | 68 ++-- migration/trace-events | 3 +- 2 files changed, 59 insertions(+), 12 d

Re: [PATCH v2 08/13] migration/postcopy: Report fault latencies in blocktime

2025-06-09 Thread Peter Xu
On Mon, Jun 09, 2025 at 03:12:54PM -0400, Peter Xu wrote: > +static void migration_dump_blocktime(Monitor *mon, MigrationInfo *info) > +{ > +if (info->has_postcopy_blocktime) { > +monitor_printf(mon, "Postcopy Blocktime (ms): %" PRIu32 "\n&qu

Re: [PATCH 19/21] migration: Allow migrate commands to provide the migration config

2025-06-09 Thread Peter Xu
On Mon, Jun 09, 2025 at 04:41:06PM -0300, Fabiano Rosas wrote: > Peter Xu writes: > > > On Mon, Jun 09, 2025 at 03:02:06PM -0300, Fabiano Rosas wrote: > >> Peter Xu writes: > >> > >> > On Mon, Jun 09, 2025 at 11:37:02AM -0300, Fab

[PATCH v2 09/13] migration/postcopy: Initialize blocktime context only until listen

2025-06-09 Thread Peter Xu
;t created instead this time; the old "if" trick isn't needed when we're sure it will only happen once now. Reviewed-by: Fabiano Rosas Signed-off-by: Peter Xu --- migration/postcopy-ram.c | 15 ++- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git

[PATCH v2 12/13] migration/postcopy: Optimize blocktime fault tracking with hashtable

2025-06-09 Thread Peter Xu
. One tracepoint is added when walking all the fault entries. Reviewed-by: Fabiano Rosas Signed-off-by: Peter Xu --- migration/postcopy-ram.c | 259 --- migration/trace-events | 5 +- 2 files changed, 216 insertions(+), 48 deletions(-) diff --git a/migration/p

[PATCH v2 00/13] migration/postcopy: Blocktime tracking overhaul

2025-06-09 Thread Peter Xu
statistics for the latency report, e.g. how many faults were resolved in (2^N, 2^(N+1)) us window, etc. Comments welcomed, thanks. Peter Xu (13): migration: Add option to set postcopy-blocktime migration/postcopy: Push blocktime start/end into page req mutex migration/postcopy

[PATCH v2 01/13] migration: Add option to set postcopy-blocktime

2025-06-09 Thread Peter Xu
Add a global property to allow enabling postcopy-blocktime feature. Reviewed-by: Fabiano Rosas Signed-off-by: Peter Xu --- migration/options.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/migration/options.c b/migration/options.c index 162c72cda4..4e923a2e07 100644 --- a/migration

[PATCH v2 07/13] migration/postcopy: Add blocktime fault counts per-vcpu

2025-06-09 Thread Peter Xu
Add a field to count how many remote faults one vCPU has taken. So far it's still not used, but will be soon. Reviewed-by: Fabiano Rosas Signed-off-by: Peter Xu --- migration/postcopy-ram.c | 5 + 1 file changed, 5 insertions(+) diff --git a/migration/postcopy-ram.c b/migration/pos

[PATCH v2 08/13] migration/postcopy: Report fault latencies in blocktime

2025-06-09 Thread Peter Xu
get_vcpu_blocktime_list() so we don't need to walk the list twice, meanwhile add the entry checks in qtests for all postcopy tests. Cc: Markus Armbruster Cc: Dr. David Alan Gilbert Reviewed-by: Fabiano Rosas Signed-off-by: Peter Xu --- qapi/migration.json

[PATCH v2 06/13] migration/postcopy: Bring blocktime layer to us level

2025-06-09 Thread Peter Xu
With 64-bit fields, it is trivial. The caution is when exposing any values in QMP, it was still declared with milliseconds (ms). Hence it's needed to do the convertion when exporting the values to existing QMP queries. Reviewed-by: Fabiano Rosas Signed-off-by: Peter Xu --- migration/pos

[PATCH v2 05/13] migration/postcopy: Drop PostcopyBlocktimeContext.start_time

2025-06-09 Thread Peter Xu
Now with 64bits, the offseting using start_time is not needed anymore, because the array can always remember the whole timestamp. Then drop the unused parameter in get_low_time_offset() altogether. Reviewed-by: Fabiano Rosas Signed-off-by: Peter Xu --- migration/postcopy-ram.c | 10

[PATCH v2 11/13] migration/postcopy: Cleanup the total blocktime accounting

2025-06-09 Thread Peter Xu
e variable at all: since nobody is touching smp_cpus_down except ourselves, we can safely do the calculation at the end before decrementing smp_cpus_down. Hopefully this makes the logic easier to read, side benefit is we drop one temp var. Reviewed-by: Fabiano Rosas Signed-off-by: Peter Xu ---

[PATCH v2 03/13] migration/postcopy: Drop all atomic ops in blocktime feature

2025-06-09 Thread Peter Xu
Now with the mutex protection it's not needed anymore. Reviewed-by: Fabiano Rosas Signed-off-by: Peter Xu --- migration/postcopy-ram.c | 23 ++- 1 file changed, 10 insertions(+), 13 deletions(-) diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c

[PATCH v2 13/13] migration/postcopy: blocktime allows track / report non-vCPU faults

2025-06-09 Thread Peter Xu
ed". Cc: Markus Armbruster Cc: Dr. David Alan Gilbert Reviewed-by: Fabiano Rosas Signed-off-by: Peter Xu --- qapi/migration.json | 7 +++ migration/migration-hmp-cmds.c| 5 +++ migration/postcopy-ram.c | 64 +-- tests/qtest/mig

[PATCH v2 02/13] migration/postcopy: Push blocktime start/end into page req mutex

2025-06-09 Thread Peter Xu
ayer. Note that we need a stub for mark_postcopy_blocktime_begin() for Windows builds. Reviewed-by: Fabiano Rosas Signed-off-by: Peter Xu --- migration/migration.h| 2 +- migration/postcopy-ram.h | 2 ++ migration/migration.c| 24 ++--- migration/postcopy-ram.c | 56 ++

[PATCH v2 04/13] migration/postcopy: Make all blocktime vars 64bits

2025-06-09 Thread Peter Xu
cktime_start. - Rename vcpu_blocktime to vcpu_blocktime_total. - Touch up the trace-events to not dump blocktime ctx pointer Reviewed-by: Fabiano Rosas Signed-off-by: Peter Xu --- migration/postcopy-ram.c | 50 migration/trace-events | 4 ++-- 2 files chan

  1   2   3   4   5   6   7   8   9   10   >