ping ... also had received some bounce back from few individual email-ids, so consider this one as resend.
Thanks, - Jitendra On 5/30/2016 4:19 PM, Jitendra Kolhe wrote: > ping... > for entire v3 version of the patchset. > http://patchwork.ozlabs.org/project/qemu-devel/list/?submitter=68462 > > - Jitendra > > On Wed, May 18, 2016 at 4:50 PM, Jitendra Kolhe <[email protected]> > wrote: >> While measuring live migration performance for qemu/kvm guest, it was >> observed >> that the qemu doesn’t maintain any intelligence for the guest ram pages >> released >> by the guest balloon driver and treat such pages as any other >> normal guest ram pages. This has direct impact on overall migration time for >> the guest which has released (ballooned out) memory to the host. >> >> In case of large systems, where we can configure large guests with 1TB and >> with considerable amount of memory released by balloon driver to the host, >> the migration time gets worse. >> >> The solution proposed below is local to qemu (and does not require any >> modification to Linux kernel or any guest driver). We have verified the fix >> for large guests =1TB on HPE Superdome X (which can support up to 240 cores >> and 12TB of memory). >> >> During live migration, as part of first iteration in ram_save_iterate() -> >> ram_find_and_save_block () will try to migrate ram pages even if they are >> released by vitrio-balloon driver (balloon inflate). Although these pages >> which are returned to the host by virtio-balloon driver are zero pages, >> the migration algorithm will still end up scanning the entire page >> ram_find_and_save_block() -> ram_save_page()/ram_save_compressed_page() -> >> save_zero_page() -> is_zero_range(). We also end-up sending header >> information >> over network for these pages during migration. This adds to the total >> migration time. >> >> The solution creates a balloon bitmap ramblock as a part of virtio-balloon >> device initialization. The bits in the balloon bitmap represent a guest ram >> page of size 1UL << VIRTIO_BALLOON_PFN_SHIFT or 4K. If TARGET_PAGE_BITS <= >> VIRTIO_BALLOON_PFN_SHIFT, ram_addr offset for the dirty page which is used by >> dirty page bitmap during migration is checked against the balloon bitmap as >> is, if the bit is set in the balloon bitmap, the corresponding ram page will >> be >> excluded from scanning and sending header information during migration. In >> case >> TARGET_PAGE_BITS > VIRTIO_BALLOON_PFN_SHIFT for a given dirty page ram_addr, >> all sub-pages of 1UL << VIRTIO_BALLOON_PFN_SHIFT size should be ballooned out >> to avoid zero page scan and sending header information. >> >> The bitmap represents entire guest ram memory till max configured memory. >> Guest ram pages claimed by the virtio-balloon driver will be represented by 1 >> in the bitmap. Since the bitmap is maintained as a ramblock, it’s migrated to >> target as part migration’s ram iterative and ram complete phase. So that >> substituent migrations from the target can continue to use optimization. >> >> A new migration capability called skip-balloon is introduced. The user can >> disable the capability in cases where user does not expect much benefit or in >> case the migration is from an older version. >> >> During live migration setup the optimization can be set to disabled state if >> . no virtio-balloon device is initialized. >> . skip-balloon migration capability is disabled. >> . If the guest virtio-balloon driver has not set >> VIRTIO_BALLOON_F_MUST_TELL_HOST >> flag. Which means the guest may start using a ram pages freed by guest >> balloon >> driver, even before the host/qemu is aware of it. In such case, the >> optimization is disabled so that the ram pages that are being used by the >> guest will continue to be scanned and migrated. >> >> Balloon bitmap ramblock size is set to zero if the optimization is disabled, >> to avoid overhead of migrating the bitmap. If the bitmap is not migrated to >> the target, the destination starts with a fresh bitmap and tracks the >> ballooning operation thereafter. >> >> Jitendra Kolhe (4): >> balloon: maintain bitmap for pages released by guest balloon driver. >> balloon: add balloon bitmap migration capability and setup bitmap >> migration status. >> balloon: reset balloon bitmap ramblock size on source and target. >> migration: skip scanning and migrating ram pages released by >> virtio-balloon driver. >> >> Changed in v2: >> - Resolved compilation issue for qemu-user binaries in exec.c >> - Localize balloon bitmap test to save_zero_page(). >> - Updated version string for newly added migration capability to 2.7. >> - Made minor modifications to patch commit text. >> >> Changed in v3: >> - Add balloon bitmap to RAMBlock. >> - Resolve bitmap offset calculation by translating host addr back to a >> RAMBlock and ram_addr >> - Add balloon bitmap support for case if TARGET_PAGE_BITS >> > VIRTIO_BALLOON_PFN_SHIFT. >> - Remove dependency of skip-balloon migration capability on postcopy >> migration. >> - Disable optimization if the guest balloon driver does not support >> VIRTIO_BALLOON_F_MUST_TELL_HOST feature. >> - Split single patch into 4 small patches. >> >> balloon.c | 196 >> ++++++++++++++++++++++++++++++++++++- >> exec.c | 6 ++ >> hw/virtio/virtio-balloon.c | 42 +++++++- >> include/hw/virtio/virtio-balloon.h | 1 + >> include/migration/migration.h | 1 + >> include/sysemu/balloon.h | 13 ++- >> migration/migration.c | 9 ++ >> migration/ram.c | 9 +- >> qapi-schema.json | 5 +- >> 9 files changed, 276 insertions(+), 6 deletions(-) >> >> -- >> 1.8.3.1 >> >>
