[RFC 3/3] vdagent: Drop blocker to support migration

2025-03-27 Thread yong . huang
From: Hyman Huang Signed-off-by: Hyman Huang --- ui/vdagent.c | 11 --- 1 file changed, 11 deletions(-) diff --git a/ui/vdagent.c b/ui/vdagent.c index 4635e8fa56..36b0568135 100644 --- a/ui/vdagent.c +++ b/ui/vdagent.c @@ -6,7 +6,6 @@ #include "qemu/option.h" #include "qemu/units.h"

[RFC 1/3] vdagent: Wrap vdagent_register_to_qemu_clipboard function

2025-03-24 Thread yong . huang
From: Hyman Huang For direct use in the upcoming commit, wrap the vdagent registry logic as vdagent_register_to_qemu_clipboard. Meanwhile, add a trace event for vdagent_recv_caps. Signed-off-by: Hyman Huang --- ui/trace-events | 1 + ui/vdagent.c| 23 --- 2 files chan

Re: [RFC 0/3] Support live migration for qemu-vdagent chardev

2025-03-21 Thread Yong Huang
On Fri, Mar 21, 2025 at 2:42 PM Marc-André Lureau < marcandre.lur...@redhat.com> wrote: > Hi > > On Fri, Mar 21, 2025 at 7:40 AM wrote: > > > > From: Hyman Huang > > > > Our goal is to migrate VMs that are configured with qemu-vdagent-typed > > chardev while allowing the agent to continue workin

[RFC 2/3] vdagent: Set up mouse and clipboard after live migration

2025-03-20 Thread yong . huang
From: Hyman Huang The struct VDAgentChardev's caps, last_serial, and cbpending fields need to be migrated in order to allow live migration for vdagent. And the clipboard and mouse should be configured to correspond with the previously negotiated caps on the destination. Signed-off-by: Hyman Huan

[RFC 0/3] Support live migration for qemu-vdagent chardev

2025-03-20 Thread yong . huang
From: Hyman Huang Our goal is to migrate VMs that are configured with qemu-vdagent-typed chardev while allowing the agent to continue working without having to restart the service in guest. Let's justify which fields should be taken into account for struct VDAgentChardev. struct VDAgentChardev

Re: [RFC] migration: Introduce migration throttle event

2025-02-18 Thread Yong Huang
On Wed, Feb 19, 2025 at 4:24 AM Markus Armbruster wrote: > Peter Xu writes: > > > On Tue, Feb 18, 2025 at 10:39:55AM +0800, yong.hu...@smartx.com wrote: > >> From: Hyman Huang > >> > >> When the developer is examining the time distribution of > >> the migration, it is useful to record the migra

Re: [RFC] migration: Introduce migration throttle event

2025-02-17 Thread Yong Huang
On Tue, Feb 18, 2025 at 1:44 PM Markus Armbruster wrote: > yong.hu...@smartx.com writes: > > > From: Hyman Huang > > > > When the developer is examining the time distribution of > > the migration, it is useful to record the migration throttle > > timestamp. Consequently, include the migration th

[RFC] migration: Introduce migration throttle event

2025-02-17 Thread yong . huang
From: Hyman Huang When the developer is examining the time distribution of the migration, it is useful to record the migration throttle timestamp. Consequently, include the migration throttle event. Signed-off-by: Hyman Huang --- migration/ram.c | 1 + qapi/migration.json | 15 +++

[PATCH v4 RESEND 2/4] guestperf: Nitpick the inconsistent parameters

2025-02-14 Thread yong . huang
From: Hyman Huang Signed-off-by: Hyman Huang Reviewed-by: Fabiano Rosas Reviewed-by: Daniel P. Berrangé --- tests/migration-stress/guestperf/comparison.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/migration-stress/guestperf/comparison.py b/tests/migration-stre

[PATCH v4 RESEND 0/4] Guestperf: miscellaneous refinement and enrichment

2025-02-14 Thread yong . huang
From: Hyman Huang v4: 1. rebase v3 patchset on master 2. build initrd-stress.img manually like before as suggested by Peter and Fabiano 3. drop the [PATCH v3 1/5], [PATCH v3 2/5] patches 4. add an extra patch: [PATCH v4 4/4] guestperf: Add test result data into report Please review, thanks Yon

[PATCH v4 RESEND 4/4] guestperf: Add test result data into report

2025-02-14 Thread yong . huang
From: Hyman Huang The migration result data is not included in the guestperf report information; include the result as a report entry so the developer can check whether the migration was successful after running guestperf. Signed-off-by: Hyman Huang --- tests/migration-stress/guestperf/engine.

[PATCH v4 RESEND 1/4] guestperf: Support deferred migration for multifd

2025-02-14 Thread yong . huang
From: Hyman Huang The way to enable multifd migration has been changed by commit, 82137e6c8c (migration: enforce multifd and postcopy preempt to be set before incoming), and guestperf has not made the necessary changes. If multifd migration had been enabled in the previous manner, the following e

[PATCH v4 RESEND 3/4] guestperf: Introduce multifd compression option

2025-02-14 Thread yong . huang
From: Hyman Huang Guestperf tool does not cover the multifd compression option currently, it is worth supporting so that developers can analysis the migration performance with different compression algorithms. Multifd support 4 compression algorithms currently: zlib, zstd, qpl, uadk To request

[PATCH v4 1/4] guestperf: Support deferred migration for multifd

2025-02-13 Thread yong . huang
From: Hyman Huang The way to enable multifd migration has been changed by commit, 82137e6c8c (migration: enforce multifd and postcopy preempt to be set before incoming), and guestperf has not made the necessary changes. If multifd migration had been enabled in the previous manner, the following e

[PATCH v4 2/4] guestperf: Nitpick the inconsistent parameters

2025-02-13 Thread yong . huang
From: Hyman Huang Signed-off-by: Hyman Huang Reviewed-by: Fabiano Rosas Reviewed-by: Daniel P. Berrangé --- tests/migration-stress/guestperf/comparison.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/migration-stress/guestperf/comparison.py b/tests/migration-stre

[PATCH v4 0/4] Guestperf: miscellaneous refinement and enrichment

2025-02-13 Thread yong . huang
From: Hyman Huang Please review, thanks Yong v3: 1. Remove the two redundant assignments in [PATCH v2 2/5] suggested by Daniel v2: 1. Update the MAINTAINERS section suggested by Fabiano Rosas 2. Ensure the dependencies when build the initrd-stress.img suggested by Daniel 3. Fix some bugs v1: T

[PATCH v4 3/4] guestperf: Introduce multifd compression option

2025-02-13 Thread yong . huang
From: Hyman Huang Guestperf tool does not cover the multifd compression option currently, it is worth supporting so that developers can analysis the migration performance with different compression algorithms. Multifd support 4 compression algorithms currently: zlib, zstd, qpl, uadk To request

[PATCH v4 4/4] guestperf: Add test result data into report

2025-02-13 Thread yong . huang
From: Hyman Huang The migration result data is not included in the guestperf report information; include the result as a report entry so the developer can check whether the migration was successful after running guestperf. Signed-off-by: Hyman Huang --- tests/migration-stress/guestperf/engine.

Re: [PATCH 2/2] improve precision of throttle_pct

2025-01-21 Thread Yong Huang
On Tue, Dec 31, 2024 at 9:56 AM fuqiang wang wrote: > Using the current algorithm, there are issues with precision not being > handled correctly during division operations. (Even though double type > casting is used in the function, it does not seem to have any effect.) > Refer to the results of

Re: [PATCH 1/2] optimize the dirtylimit_throttle_pct trace event

2025-01-02 Thread Yong Huang
On Tue, Dec 31, 2024 at 9:56 AM fuqiang wang wrote: > The current dirtylimit_throttle_pct trace event is triggered when the > throttle time is adjusted linearly. Modify the trace event so that it > can record non-linear adjustments. Please target the optimization mentioned above in a single pat

Re: [PATCH v2 2/2] qxl: Fix race on accessing cursor in struct SimpleSpiceDisplay

2025-01-02 Thread Yong Huang
- Ping On Fri, Dec 20, 2024 at 7:46 PM Hyman Huang wrote: > Both the spice server and the qemu (iothread/main loop) context > can access the cursor field in struct SimpleSpiceDisplay. > > Add the mutex lock before accessing cursor in > qxl_spice_reset_cursor(). > > Signed-off-by: Hyman Huang >

Re: [PATCH 7/9] system/dirtylimit: Don't use migration_is_active()

2024-12-17 Thread Yong Huang
On Tue, Dec 17, 2024 at 5:47 PM Avihai Horon wrote: > > On 17/12/2024 5:01, Yong Huang wrote: > > *External email: Use caution opening links or attachments* > > > > > > > > > > On Mon, Dec 16, 2024 at 5:47 PM Avihai Horon wrote: >

Re: [PATCH 7/9] system/dirtylimit: Don't use migration_is_active()

2024-12-16 Thread Yong Huang
On Mon, Dec 16, 2024 at 5:47 PM Avihai Horon wrote: > vcpu_dirty_rate_stat_collect() uses migration_is_active() to detect > whether migration is running or not, in order to get the correct dirty > rate period value. > > However, recently there has been an effort to simplify the migration > status

Re: [PATCH v3 0/5] Guestperf: miscellaneous refinement and enrichment

2024-12-08 Thread Yong Huang
On Wed, Dec 4, 2024 at 1:40 AM Peter Xu wrote: > On Tue, Dec 03, 2024 at 10:15:57AM -0300, Fabiano Rosas wrote: > > We shouldn't be adding warnings to the build like that. When building > > static binaries, I'd assume the person at least knows there's a -static > > in there somewhere. If you're j

Re: [PATCH v3 0/5] Guestperf: miscellaneous refinement and enrichment

2024-11-30 Thread Yong Huang
On Fri, Nov 29, 2024 at 9:44 PM Fabiano Rosas wrote: > Fabiano Rosas writes: > > > Yong Huang writes: > > > >> On Wed, Oct 23, 2024 at 11:06 PM Fabiano Rosas wrote: > >> > >>> yong.hu...@smartx.com writes: > >>> > >>

Re: [PATCH v3 0/5] Guestperf: miscellaneous refinement and enrichment

2024-11-29 Thread Yong Huang
On Wed, Oct 23, 2024 at 11:06 PM Fabiano Rosas wrote: > yong.hu...@smartx.com writes: > > > From: Hyman Huang > > > > v3: > > 1. Remove the two redundant assignments in [PATCH v2 2/5] suggested by > Daniel > > > > Please review, thanks > > Yong > > > > v2: > > 1. Update the MAINTAINERS section s

Re: [PATCH v1 2/2] migration: Do not perform RAMBlock dirty sync during the first iteration

2024-11-11 Thread Yong Huang
On Mon, Nov 11, 2024 at 6:42 PM David Hildenbrand wrote: > On 11.11.24 11:08, Yong Huang wrote: > > > > > > On Mon, Nov 11, 2024 at 5:27 PM David Hildenbrand > <mailto:da...@redhat.com>> wrote: > > > > On 09.11.24 05:59, Hyman Huang wrote: &g

Re: [PATCH v1 2/2] migration: Do not perform RAMBlock dirty sync during the first iteration

2024-11-11 Thread Yong Huang
On Mon, Nov 11, 2024 at 6:42 PM David Hildenbrand wrote: > On 11.11.24 11:08, Yong Huang wrote: > > > > > > On Mon, Nov 11, 2024 at 5:27 PM David Hildenbrand > <mailto:da...@redhat.com>> wrote: > > > > On 09.11.24 05:59, Hyman Huang wrote: &g

Re: [PATCH v1 2/2] migration: Do not perform RAMBlock dirty sync during the first iteration

2024-11-11 Thread Yong Huang
On Mon, Nov 11, 2024 at 5:07 PM Wang, Wei W wrote: > On Saturday, November 9, 2024 1:00 PM, Hyman Huang wrote: > > The first iteration's RAMBlock dirty sync can be omitted because QEMU > > always initializes the RAMBlock's bmap to all 1s by default. > > > > Signed-off-by: Hyman Huang > > --- > >

Re: [PATCH v1 2/2] migration: Do not perform RAMBlock dirty sync during the first iteration

2024-11-11 Thread Yong Huang
On Mon, Nov 11, 2024 at 5:27 PM David Hildenbrand wrote: > On 09.11.24 05:59, Hyman Huang wrote: > > The first iteration's RAMBlock dirty sync can be omitted because QEMU > > always initializes the RAMBlock's bmap to all 1s by default. > > > > Signed-off-by: Hyman Huang > > --- > > migration/c

Re: [PATCH] migration: Do not perform RAMBlock dirty sync during the first iteration

2024-11-08 Thread Yong Huang
On Fri, Nov 8, 2024 at 9:50 PM Peter Xu wrote: > On Fri, Nov 08, 2024 at 02:03:47PM +0800, Yong Huang wrote: > > On Fri, Nov 8, 2024 at 12:28 AM Peter Xu wrote: > > > > > On Thu, Nov 07, 2024 at 05:56:50PM +0800, yong.hu...@smartx.com wrote: > > > > From: Hy

Re: [PATCH] migration: Do not perform RAMBlock dirty sync during the first iteration

2024-11-07 Thread Yong Huang
On Fri, Nov 8, 2024 at 12:28 AM Peter Xu wrote: > On Thu, Nov 07, 2024 at 05:56:50PM +0800, yong.hu...@smartx.com wrote: > > From: Hyman Huang > > > > The first iteration's RAMBlock dirty sync can be omitted because QEMU > > always initializes the RAMBlock's bmap to all 1s by default. > > > > Si

[PATCH] migration: Do not perform RAMBlock dirty sync during the first iteration

2024-11-07 Thread yong . huang
From: Hyman Huang The first iteration's RAMBlock dirty sync can be omitted because QEMU always initializes the RAMBlock's bmap to all 1s by default. Signed-off-by: Hyman Huang --- migration/cpu-throttle.c | 2 +- migration/ram.c | 19 --- 2 files changed, 17 insertion

Re: [PATCH RFC 2/2] migration: Avoid doing RAMBlock dirty sync in the initial iteration

2024-10-30 Thread Yong Huang
On Thu, Oct 31, 2024 at 3:43 AM Peter Xu wrote: > On Wed, Oct 30, 2024 at 10:09:38AM +0800, Yong Huang wrote: > > On Wed, Oct 30, 2024 at 12:21 AM Peter Xu wrote: > > > > > On Wed, Oct 23, 2024 at 10:09:51AM +0800, yong.hu...@smartx.com wrote: > > > > Fr

Re: [PATCH RFC 2/2] migration: Avoid doing RAMBlock dirty sync in the initial iteration

2024-10-29 Thread Yong Huang
On Wed, Oct 30, 2024 at 12:21 AM Peter Xu wrote: > On Wed, Oct 23, 2024 at 10:09:51AM +0800, yong.hu...@smartx.com wrote: > > From: Hyman Huang > > > > KVM always returns 1 when userspace retrieves a dirty bitmap for > > the first time when KVM_DIRTY_LOG_INITIALLY_SET is enabled; in such > > sce

Re: [PATCH v3 0/5] Guestperf: miscellaneous refinement and enrichment

2024-10-23 Thread Yong Huang
On Wed, Oct 23, 2024 at 11:06 PM Fabiano Rosas wrote: > yong.hu...@smartx.com writes: > > > From: Hyman Huang > > > > v3: > > 1. Remove the two redundant assignments in [PATCH v2 2/5] suggested by > Daniel > > > > Please review, thanks > > Yong > > > > v2: > > 1. Update the MAINTAINERS section s

[PATCH RFC 1/2] accel/kvm: Introduce kvm_dirty_log_manual_enabled

2024-10-22 Thread yong . huang
From: Hyman Huang Introudce kvm_dirty_log_manual_enabled to indicate if dirty logging manually was enabled in KVM. The kvm_dirty_log_manual_enabled will be used in the next commit. Signed-off-by: Hyman Huang --- accel/kvm/kvm-all.c | 2 ++ include/sysemu/kvm.h | 8 2 files changed,

[PATCH RFC 0/2] migration: Skip sync in ram_init_bitmaps()

2024-10-22 Thread yong . huang
From: Hyman Huang As discussed in: https://lore.kernel.org/qemu-devel/ZvbQ0RQx-zxOeo4Y@x1n/ RAMBlock dirty sync in ram_init_bitmaps() appears to be unnecessary; this patchset attempts to eliminate it and asks for comments on how to do so. Please review, thanks Yong Hyman Huang (2): accel/kvm

[PATCH RFC 2/2] migration: Avoid doing RAMBlock dirty sync in the initial iteration

2024-10-22 Thread yong . huang
From: Hyman Huang KVM always returns 1 when userspace retrieves a dirty bitmap for the first time when KVM_DIRTY_LOG_INITIALLY_SET is enabled; in such scenario, the RAMBlock dirty sync of the initial iteration can be skipped. Signed-off-by: Hyman Huang --- migration/cpu-throttle.c | 3 ++- mi

Re: [PATCH v2] migration/dirtyrate: Silence warning about strcpy() on OpenBSD

2024-10-21 Thread Yong Huang
On Tue, Oct 22, 2024 at 2:34 PM Thomas Huth wrote: > The linker on OpenBSD complains: > > ld: warning: dirtyrate.c:447 (../src/migration/dirtyrate.c:447)(...): > warning: strcpy() is almost always misused, please use strlcpy() > > It's currently not a real problem in this case since both arrays

[PATCH v3 3/5] guestperf: Support deferred migration for multifd

2024-10-21 Thread yong . huang
From: Hyman Huang The way to enable multifd migration has been changed by commit, 82137e6c8c (migration: enforce multifd and postcopy preempt to be set before incoming), and guestperf has not made the necessary changes. If multifd migration had been enabled in the previous manner, the following e

[PATCH v3 5/5] guestperf: Introduce multifd compression option

2024-10-21 Thread yong . huang
From: Hyman Huang Guestperf tool does not cover the multifd compression option currently, it is worth supporting so that developers can analysis the migration performance with different compression algorithms. Multifd support 4 compression algorithms currently: zlib, zstd, qpl, uadk To request

[PATCH v3 1/5] tests/migration: Move the guestperf tool to scripts directory

2024-10-21 Thread yong . huang
From: Hyman Huang Guestperf was designed to test the performance of migration, with a loose connection to the fundamental test cases of QEMU. To improve the repository's structure, move it to the scripts directory. Add myself as a maintainer for the guestperf so that I can help to fix bugs. Si

[PATCH v3 4/5] guestperf: Nitpick the inconsistent parameters

2024-10-21 Thread yong . huang
From: Hyman Huang Signed-off-by: Hyman Huang Reviewed-by: Fabiano Rosas Reviewed-by: Daniel P. Berrangé --- scripts/migration/guestperf/comparison.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/migration/guestperf/comparison.py b/scripts/migration/guestperf/co

[PATCH v3 0/5] Guestperf: miscellaneous refinement and enrichment

2024-10-21 Thread yong . huang
From: Hyman Huang v3: 1. Remove the two redundant assignments in [PATCH v2 2/5] suggested by Daniel Please review, thanks Yong v2: 1. Update the MAINTAINERS section suggested by Fabiano Rosas 2. Ensure the dependencies when build the initrd-stress.img suggested by Daniel 3. Fix some bugs v1:

[PATCH v3 2/5] tests/migration: Make initrd-stress.img built by default

2024-10-21 Thread yong . huang
From: Hyman Huang The initrd-stress.img was compiled by specifying the target, to make it easier for developers to play the guestperf tool, make it built when dependencies suffices. Signed-off-by: Hyman Huang Reviewed-by: Daniel P. Berrangé --- tests/migration/meson.build | 30 +++

Re: [PATCH v4 5/6] migration: Support periodic RAMBlock dirty bitmap sync

2024-10-17 Thread Yong Huang
On Fri, Oct 18, 2024 at 3:33 AM Peter Xu wrote: > On Thu, Oct 17, 2024 at 02:42:54PM +0800, yong.hu...@smartx.com wrote: > > +void cpu_throttle_dirty_sync_timer_tick(void *opaque) > > +{ > > +static uint64_t prev_sync_cnt; > > We may need to reset this in case migration got cancelled and invo

Re: [PATCH] migration/dirtyrate: Silence warning about strcpy() on OpenBSD

2024-10-17 Thread Yong Huang
On Thu, Oct 17, 2024 at 1:40 PM Thomas Huth wrote: > On 16/10/2024 18.22, Daniel P. Berrangé wrote: > > On Wed, Oct 16, 2024 at 06:07:12PM +0200, Thomas Huth wrote: > >> The linker on OpenBSD complains: > >> > >> ld: warning: dirtyrate.c:447 (../src/migration/dirtyrate.c:447)(...): > >> warni

[PATCH v4 2/6] migration: Stop CPU throttling conditionally

2024-10-16 Thread yong . huang
From: Hyman Huang Since CPU throttling only occurs when auto-converge is on, stop it conditionally. Signed-off-by: Hyman Huang --- migration/migration.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/migration/migration.c b/migration/migration.c index 021faee2f3..37a200

[PATCH v4 3/6] migration: Move cpu-throttole.c from system to migration

2024-10-16 Thread yong . huang
From: Hyman Huang Move cpu-throttle.c from system to migration since it's only used for migration; this makes us avoid exporting the util functions and variables in misc.h but export them in migration.h when implementing the periodic ramblock dirty sync feature in the upcoming commits. Since CPU

[PATCH v4 1/6] accel/tcg/icount-common: Remove the reference to the unused header file

2024-10-16 Thread yong . huang
From: Hyman Huang Signed-off-by: Hyman Huang --- accel/tcg/icount-common.c | 1 - 1 file changed, 1 deletion(-) diff --git a/accel/tcg/icount-common.c b/accel/tcg/icount-common.c index 8d3d3a7e9d..30bf8500dc 100644 --- a/accel/tcg/icount-common.c +++ b/accel/tcg/icount-common.c @@ -36,7 +36,6

[PATCH v4 5/6] migration: Support periodic RAMBlock dirty bitmap sync

2024-10-16 Thread yong . huang
From: Hyman Huang When VM is configured with huge memory, the current throttle logic doesn't look like to scale, because migration_trigger_throttle() is only called for each iteration, so it won't be invoked for a long time if one iteration can take a long time. The periodic dirty sync aims to f

[PATCH v4 6/6] tests/migration: Add case for periodic ramblock dirty sync

2024-10-16 Thread yong . huang
From: Hyman Huang Signed-off-by: Hyman Huang Reviewed-by: Peter Xu --- tests/qtest/migration-test.c | 32 1 file changed, 32 insertions(+) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index 95e45b5029..e6a2803e71 100644 --- a/tests/

[PATCH v4 4/6] migration: Remove "rs" parameter in migration_bitmap_sync_precopy

2024-10-16 Thread yong . huang
From: Hyman Huang The global static variable ram_state in fact is referred to by the "rs" parameter in migration_bitmap_sync_precopy. For ease of calling by the callees, use the global variable directly in migration_bitmap_sync_precopy and remove "rs" parameter. The migration_bitmap_sync_precopy

[PATCH v4 0/6] migration: auto-converge refinements for huge VM

2024-10-16 Thread yong . huang
From: Hyman Huang v4: 1. split the [PATCH v3 1/6] into smaller patches. 2. refine some comment and commit message 3. fix race issue for the throttle_dirty_sync_timer 4. refine the util function cpu_throttle_dirty_sync_timer Thanks Peter for the suggestions. Please review. Yong v3: 1. drop the

Re: [PATCH v3 3/4] migration: Support periodic ramblock dirty sync

2024-10-16 Thread Yong Huang
On Thu, Oct 17, 2024 at 2:49 AM Peter Xu wrote: > On Wed, Oct 16, 2024 at 03:56:44PM +0800, yong.hu...@smartx.com wrote: > > From: Hyman Huang > > > > When VM is configured with huge memory, the current throttle logic > > doesn't look like to scale, because migration_trigger_throttle() > > is on

Re: [PATCH v3 1/4] migration: Move cpu-throttole.c from system to migration

2024-10-16 Thread Yong Huang
On Wed, Oct 16, 2024 at 11:50 PM Peter Xu wrote: > On Wed, Oct 16, 2024 at 03:56:42PM +0800, yong.hu...@smartx.com wrote: > > From: Hyman Huang > > > > Move cpu-throttle.c from system to migration since it's > > only used for migration; this makes us avoid exporting the > > util functions and va

[PATCH v3 2/4] migration: Remove "rs" parameter in migration_bitmap_sync_precopy

2024-10-16 Thread yong . huang
From: Hyman Huang The global static variable ram_state in fact is referred to by the "rs" parameter in migration_bitmap_sync_precopy. For ease of calling by the callees, use the global variable directly in migration_bitmap_sync_precopy and remove "rs" parameter. The migration_bitmap_sync_precopy

[PATCH v3 1/4] migration: Move cpu-throttole.c from system to migration

2024-10-16 Thread yong . huang
From: Hyman Huang Move cpu-throttle.c from system to migration since it's only used for migration; this makes us avoid exporting the util functions and variables in misc.h but export them in migration.h when implementing the background ramblock dirty sync feature in the upcoming commits. Additio

[PATCH v3 4/4] tests/migration: Add case for periodic ramblock dirty sync

2024-10-16 Thread yong . huang
From: Hyman Huang Signed-off-by: Hyman Huang --- tests/qtest/migration-test.c | 32 1 file changed, 32 insertions(+) diff --git a/tests/qtest/migration-test.c b/tests/qtest/migration-test.c index 95e45b5029..e6a2803e71 100644 --- a/tests/qtest/migration-test.c

[PATCH v3 0/4] migration: auto-converge refinements for huge VM

2024-10-16 Thread yong . huang
From: Hyman Huang v3: 1. drop the responsive throttle patchset 2. rename background sync to periodic ramblock dirty sync 3. move the cpu-throttle.* from system to migration 4. remove "rs" parameter in migration_bitmap_sync_precopy 5. implement periodic ramblock dirty sync in cpu-throttle.c 6. mov

[PATCH v3 3/4] migration: Support periodic ramblock dirty sync

2024-10-16 Thread yong . huang
From: Hyman Huang When VM is configured with huge memory, the current throttle logic doesn't look like to scale, because migration_trigger_throttle() is only called for each iteration, so it won't be invoked for a long time if one iteration can take a long time. The periodic dirty sync aims to f

Re: [PATCH] migration: Put thread names together with macros

2024-10-12 Thread Yong Huang
On Sat, Oct 12, 2024 at 1:29 AM Fabiano Rosas wrote: > Peter Xu writes: > > > Keep migration thread names together, so it's easier to see a list of all > > possible migration threads. > > > > Still two functional changes below besides the macro defintions: > > > > - There's one dirty rate thre

Re: [PATCH v2 3/3] migration: Support responsive CPU throttle

2024-10-07 Thread Yong Huang
On Tue, Oct 1, 2024 at 11:37 PM Peter Xu wrote: > On Tue, Oct 01, 2024 at 10:18:54AM +0800, Yong Huang wrote: > > On Tue, Oct 1, 2024 at 4:47 AM Peter Xu wrote: > > > > > On Mon, Sep 30, 2024 at 01:14:28AM +0800, yong.hu...@smartx.com wrote: > > > > From: Hy

Re: [PATCH v2 1/3] migration: Support background ramblock dirty sync

2024-10-07 Thread Yong Huang
On Tue, Oct 1, 2024 at 11:28 PM Peter Xu wrote: > On Tue, Oct 01, 2024 at 10:02:53AM +0800, Yong Huang wrote: > > On Tue, Oct 1, 2024 at 4:41 AM Peter Xu wrote: > > > > > On Mon, Sep 30, 2024 at 01:14:26AM +0800, yong.hu...@smartx.com wrote: > > > > From: Hym

Re: [PATCH v2 3/3] migration: Support responsive CPU throttle

2024-10-01 Thread Yong Huang
On Tue, Oct 1, 2024 at 4:47 AM Peter Xu wrote: > On Mon, Sep 30, 2024 at 01:14:28AM +0800, yong.hu...@smartx.com wrote: > > From: Hyman Huang > > > > Currently, the convergence algorithm determines that the migration > > cannot converge according to the following principle: > > The dirty pages g

Re: [PATCH v2 1/3] migration: Support background ramblock dirty sync

2024-09-30 Thread Yong Huang
On Tue, Oct 1, 2024 at 4:41 AM Peter Xu wrote: > On Mon, Sep 30, 2024 at 01:14:26AM +0800, yong.hu...@smartx.com wrote: > > From: Hyman Huang > > > > When VM is configured with huge memory, the current throttle logic > > doesn't look like to scale, because migration_trigger_throttle() > > is onl

[PATCH v2 2/3] qapi/migration: Introduce cpu-throttle-responsive parameter

2024-09-29 Thread yong . huang
From: Hyman Huang To enable the responsive throttle that will be implemented in the next commit, introduce the cpu-responsive-throttle parameter. Signed-off-by: Hyman Huang --- migration/migration-hmp-cmds.c | 8 migration/options.c| 20 migration/opt

[PATCH v2 3/3] migration: Support responsive CPU throttle

2024-09-29 Thread yong . huang
From: Hyman Huang Currently, the convergence algorithm determines that the migration cannot converge according to the following principle: The dirty pages generated in current iteration exceed a specific percentage (throttle-trigger-threshold, 50 by default) of the number of transmissions. Let's

[PATCH v2 0/3] migration: auto-converge refinements for huge VM

2024-09-29 Thread yong . huang
From: Hyman Huang v2: 1. background sync - Throw out the idea of "not updating the bitmap" when the RAMBlock of the RAM list is iterated during migration; re-implement the background RAM dirty sync using Peter's updated method. 2. responsive throttle - Rename the "cpu-responsive throttl

[PATCH v2 1/3] migration: Support background ramblock dirty sync

2024-09-29 Thread yong . huang
From: Hyman Huang When VM is configured with huge memory, the current throttle logic doesn't look like to scale, because migration_trigger_throttle() is only called for each iteration, so it won't be invoked for a long time if one iteration can take a long time. The background dirty sync aim to

Re: [PATCH v1 1/7] migration: Introduce structs for background sync

2024-09-27 Thread Yong Huang
On Fri, Sep 27, 2024 at 11:35 PM Peter Xu wrote: > On Fri, Sep 27, 2024 at 10:50:01AM +0800, Yong Huang wrote: > > On Fri, Sep 27, 2024 at 3:55 AM Peter Xu wrote: > > > > > On Fri, Sep 27, 2024 at 02:13:47AM +0800, Yong Huang wrote: > > > > On Thu, Sep

Re: [PATCH v1 1/7] migration: Introduce structs for background sync

2024-09-26 Thread Yong Huang
On Fri, Sep 27, 2024 at 3:55 AM Peter Xu wrote: > On Fri, Sep 27, 2024 at 02:13:47AM +0800, Yong Huang wrote: > > On Thu, Sep 26, 2024 at 3:17 AM Peter Xu wrote: > > > > > On Fri, Sep 20, 2024 at 10:43:31AM +0800, Yong Huang wrote: > > > > Yes, invok

Re: [PATCH v1 1/7] migration: Introduce structs for background sync

2024-09-26 Thread Yong Huang
On Thu, Sep 26, 2024 at 3:17 AM Peter Xu wrote: > On Fri, Sep 20, 2024 at 10:43:31AM +0800, Yong Huang wrote: > > Yes, invoke migration_bitmap_sync_precopy more frequently is also my > > first idea but it involves bitmap updating and interfere with the > behavior > >

Re: [PATCH v1 1/7] migration: Introduce structs for background sync

2024-09-19 Thread Yong Huang
On Fri, Sep 20, 2024 at 2:45 AM Peter Xu wrote: > On Tue, Sep 17, 2024 at 02:48:03PM +0800, Yong Huang wrote: > > On Tue, Sep 17, 2024 at 5:11 AM Fabiano Rosas wrote: > > > > > Hyman Huang writes: > > > > > > > shadow_bmap, iter_bmap and iter_dirt

Re: [PATCH v1 1/7] migration: Introduce structs for background sync

2024-09-19 Thread Yong Huang
On Fri, Sep 20, 2024 at 2:45 AM Peter Xu wrote: > On Tue, Sep 17, 2024 at 02:48:03PM +0800, Yong Huang wrote: > > On Tue, Sep 17, 2024 at 5:11 AM Fabiano Rosas wrote: > > > > > Hyman Huang writes: > > > > > > > shadow_bmap, iter_bmap and iter_dirt

Re: [PATCH v1 1/7] migration: Introduce structs for background sync

2024-09-19 Thread Yong Huang
On Fri, Sep 20, 2024 at 2:45 AM Peter Xu wrote: > On Tue, Sep 17, 2024 at 02:48:03PM +0800, Yong Huang wrote: > > On Tue, Sep 17, 2024 at 5:11 AM Fabiano Rosas wrote: > > > > > Hyman Huang writes: > > > > > > > shadow_bmap, iter_bmap and iter_dirt

Re: [PATCH v1 3/7] qapi/migration: Introduce the iteration-count

2024-09-18 Thread Yong Huang
On Tue, Sep 17, 2024 at 4:35 AM Fabiano Rosas wrote: > Hyman Huang writes: > > > The original migration information dirty-sync-count could > > no longer reflect iteration count due to the introduction > > of background synchronization in the next commit; > > add the iteration count to compensate

Re: [PATCH v1 6/7] qapi/migration: Introduce cpu-responsive-throttle parameter

2024-09-16 Thread Yong Huang
On Tue, Sep 17, 2024 at 4:55 AM Fabiano Rosas wrote: > Hyman Huang writes: > > > To enable the responsive throttle that will be implemented > > in the next commit, introduce the cpu-responsive-throttle > > parameter. > > > > Signed-off-by: Hyman Huang > > --- > > migration/migration-hmp-cmds.c

Re: [PATCH v1 3/7] qapi/migration: Introduce the iteration-count

2024-09-16 Thread Yong Huang
On Tue, Sep 17, 2024 at 4:35 AM Fabiano Rosas wrote: > Hyman Huang writes: > > > The original migration information dirty-sync-count could > > no longer reflect iteration count due to the introduction > > of background synchronization in the next commit; > > add the iteration count to compensate

Re: [PATCH v1 1/7] migration: Introduce structs for background sync

2024-09-16 Thread Yong Huang
On Tue, Sep 17, 2024 at 5:11 AM Fabiano Rosas wrote: > Hyman Huang writes: > > > shadow_bmap, iter_bmap and iter_dirty_pages are introduced > > to satisfy the need for background sync. > > > > Meanwhile, introduce enumeration of sync method. > > > > Signed-off-by: Hyman Huang > > --- > > inclu

Re: [PATCH RESEND RFC 08/10] migration: Introduce cpu-responsive-throttle parameter

2024-09-09 Thread Yong Huang
In order to reduce the maintenance work of QMP. Given that the focus of this patchset is huge VM migration, is it possible to enable or disable these two features with a single parameter, such as "cpu-aggressive-throttle"? Yong On Mon, Sep 9, 2024 at 10:26 PM Hyman Huang wrote: > To enable the

Re: [PATCH RESEND RFC 03/10] qapi/migration: Introduce periodic CPU throttling parameters

2024-09-09 Thread Yong Huang
On Tue, Sep 10, 2024 at 5:30 AM Peter Xu wrote: > On Mon, Sep 09, 2024 at 10:25:36PM +0800, Hyman Huang wrote: > > To activate the periodic CPU throttleing feature, introduce > > the cpu-periodic-throttle. > > > > To control the frequency of throttling, introduce the > > cpu-periodic-throttle-int

Re: [PATCH RFC 10/10] tests/migration-tests: Add test case for responsive CPU throttle

2024-09-09 Thread Yong Huang
On Mon, Sep 9, 2024 at 10:03 PM Peter Maydell wrote: > On Mon, 9 Sept 2024 at 14:51, Hyman Huang wrote: > > > > Despite the fact that the responsive CPU throttle is enabled, > > the dirty sync count may not always increase because this is > > an optimization that might not happen in any situatio

Re: [PATCH v1 2/5] tests/migration: Make initrd-stress.img built by default

2024-08-09 Thread Yong Huang
On Fri, Aug 9, 2024 at 4:34 PM Daniel P. Berrangé wrote: > On Fri, Aug 09, 2024 at 03:43:33PM +0800, yong.hu...@smartx.com wrote: > > From: Hyman Huang > > > > The initrd-stress.img was compiled by specifying the > > target, to make it easier for developers to play the > > guestperf tool, make i

[PATCH v1 5/5] guestperf: Introduce multifd compression option

2024-08-09 Thread yong . huang
From: Hyman Huang Guestperf tool does not cover the multifd compression option currently, it is worth supporting so that developers can analysis the migration performance with different compression algorithms. Multifd support 4 compression algorithms currently: zlib, zstd, qpl, uadk To request

[PATCH v1 3/5] guestperf: Support deferred migration for multifd

2024-08-09 Thread yong . huang
From: Hyman Huang The way to enable multifd migration has been changed by commit, 82137e6c8c (migration: enforce multifd and postcopy preempt to be set before incoming), and guestperf has not made the necessary changes. If multifd migration had been enabled in the previous manner, the following e

[PATCH v1 1/5] tests/migration: Move the guestperf tool to scripts directory

2024-08-09 Thread yong . huang
From: Hyman Huang Guestperf was designed to test the performance of migration, with a loose connection to the fundamental test cases of QEMU. To improve the repository's structure, move it to the scripts directory. Signed-off-by: Hyman Huang --- MAINTAINERS

[PATCH v1 2/5] tests/migration: Make initrd-stress.img built by default

2024-08-09 Thread yong . huang
From: Hyman Huang The initrd-stress.img was compiled by specifying the target, to make it easier for developers to play the guestperf tool, make it built by default. Signed-off-by: Hyman Huang --- tests/migration/meson.build | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --gi

[PATCH v1 4/5] guestperf: Nitpick the inconsistent parameters

2024-08-09 Thread yong . huang
From: Hyman Huang Signed-off-by: Hyman Huang Reviewed-by: Fabiano Rosas --- scripts/migration/guestperf/comparison.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/migration/guestperf/comparison.py b/scripts/migration/guestperf/comparison.py index 42cc0372d1..40e

[PATCH v1 0/5] Guestperf: miscellaneous refinement and enrichment

2024-08-09 Thread yong . huang
From: Hyman Huang The previous patchset: https://lore.kernel.org/qemu-devel/cover.1722957352.git.yong.hu...@smartx.com/ does not made the necessary changes and tests for the upstream version. This patchset works for that: 1. Move the guestperf to scripts directory suggested by Fabiano Rosas 2. M

Re: [PATCH] migration: Refine the convergence detection

2024-08-07 Thread Yong Huang
On Wed, Aug 7, 2024 at 8:59 PM Peter Xu wrote: > On Wed, Aug 07, 2024 at 02:46:29PM +0800, Yong Huang wrote: > > I still think dropping the dirty_rate_high_cnt is a refinement. If > migration > > has a precise detection of convergence, what do you think of it? > > Maybe;

Re: [PATCH] migration: Refine the convergence detection

2024-08-06 Thread Yong Huang
On Wed, Aug 7, 2024 at 12:45 AM Peter Xu wrote: > On Mon, Aug 05, 2024 at 03:03:27PM +0800, Yong Huang wrote: > > Sorry for the late reply. > > > > On Wed, Jul 31, 2024 at 4:01 AM Peter Xu wrote: > > > > > On Wed, Jul 24, 2024 at 07:39:29PM +0800, Hyman

Re: [PATCH 2/2] tests/migration: Introduce multifd compression into guestperf

2024-08-06 Thread Yong Huang
On Wed, Aug 7, 2024 at 3:22 AM Fabiano Rosas wrote: > yong.hu...@smartx.com writes: > > > From: Hyman Huang > > > > Guestperf tool does not cover the multifd compression option > > currently, it is worth supporting so that developers can > > analysis the migration performance with different > >

[PATCH 2/2] tests/migration: Introduce multifd compression into guestperf

2024-08-06 Thread yong . huang
From: Hyman Huang Guestperf tool does not cover the multifd compression option currently, it is worth supporting so that developers can analysis the migration performance with different compression algorithms. Multifd support 4 compression algorithms currently: zlib, zstd, qpl, uadk To request

[PATCH 0/2] Support multifd compression test for guestperf

2024-08-06 Thread yong . huang
From: Hyman Huang The first patch just nitpick the inconsistent parameters for multifd test. The second patch support multifd compression test. Please review, thanks, Yong Hyman Huang (2): tests/migration: Nitpick the inconsistent parameters tests/migration: Introduce multifd compression

[PATCH 1/2] tests/migration: Nitpick the inconsistent parameters

2024-08-06 Thread yong . huang
From: Hyman Huang Signed-off-by: Hyman Huang --- tests/migration/guestperf/comparison.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/migration/guestperf/comparison.py b/tests/migration/guestperf/comparison.py index 42cc0372d1..40e9d2eb1d 100644 --- a/tests/migrati

Re: [PATCH] migration: Refine the convergence detection

2024-08-05 Thread Yong Huang
Sorry for the late reply. On Wed, Jul 31, 2024 at 4:01 AM Peter Xu wrote: > On Wed, Jul 24, 2024 at 07:39:29PM +0800, Hyman Huang wrote: > > Currently, the convergence algorithm determines that the migration > > cannot converge according to the following principle: > > The dirty pages generated

Re: [PATCH] e1000: Fix the unexpected assumption that the receive buffer is full

2024-07-16 Thread Yong Huang
On Wed, Jul 17, 2024 at 9:29 AM Jason Wang wrote: > On Wed, Jul 17, 2024 at 9:24 AM Yong Huang wrote: > > > > > > > > On Fri, Jul 12, 2024 at 10:01 AM Jason Wang wrote: > >> > >> On Wed, Jul 10, 2024 at 5:05 PM Yong Huang > wrote: > >&g

Re: [PATCH] e1000: Fix the unexpected assumption that the receive buffer is full

2024-07-16 Thread Yong Huang
On Fri, Jul 12, 2024 at 10:01 AM Jason Wang wrote: > On Wed, Jul 10, 2024 at 5:05 PM Yong Huang wrote: > > > > > > > > On Wed, Jul 10, 2024 at 3:36 PM Jason Wang wrote: > >> > >> On Wed, Jul 10, 2024 at 2:26 PM Yong Huang > wrote: > >>

  1   2   >