Re: [PATCH 2/2] migration/multifd: Fix rb->receivedmap cleanup race

2024-09-20 Thread Elena Ufimtseva
mu_loadvm_state_cleanup(); > cpu_synchronize_all_post_init(); > Hi Fabiano I have a question. By removing qemu_loadvm_state_cleanup() here, the failure path that ends up with exit(EXIT_FAILURE) in process_incoming_migration_co() end up not calling the qemu_loadvm_state_cleanup(). I am not sure how this is important since there is exit, but the vfio, for example, will not call the VF reset. Another more general question is why destination Qemu has to terminate there if there was an error detected during live migration? Could just failing the migration and leave destination running be a more expected scenario? Thank you! return ret; > -- > 2.35.3 > > > -- Elena

Re: [PATCH RFC 2/2] migration: abort on destination if switchover limit exceeded

2024-07-26 Thread Elena Ufimtseva
starting point and be named as ram-downtime-limit and everything else would have switchover allowance of downtime_limit - ram-downtime-limit? The correct value of ram-downtime-limit would take few iterations after dirty sync to get established. I think that is somewhat similar to what Joao is think

[PATCH RFC 2/2] migration: abort on destination if switchover limit exceeded

2024-06-21 Thread Elena Ufimtseva
During live migration, receive current downtime from source and start a downtime timer. When the destination dowtime and added source downtime exceeds downtime limit for more than switchover limit, abort live migration on destination. Signed-off-by: Elena Ufimtseva --- migration/migration.c

[PATCH RFC 1/2] migration: abort when switchover limit exceeded

2024-06-21 Thread Elena Ufimtseva
Introduce capability switchover_abort and migration parameter switchover_limit to allow for live migration abort when the source downtime exceeded by switchover_limit. Signed-off-by: Elena Ufimtseva --- hw/core/machine.c | 1 + include/migration/client-options.h | 1

[PATCH RFC 0/2] migration: introduce strict SLA

2024-06-21 Thread Elena Ufimtseva
(switchover-limit) and total downtime would not be more than 310ms. Please send your comments and recommendations. The patchset idea originally comes from Joao Martins . Elena Ufimtseva (2): migration: abort when switchover limit exceeded migration: abort on destination if switchover limit exceeded

Re: [PATCH v2 4/7] migration/multifd: Enable zero page checking from multifd threads.

2024-02-21 Thread Elena Ufimtseva
> -return ram_save_page(rs, pss); > +return ram_save_multifd_page(block, offset); > } > > /* Should be called before sending a host page */ > @@ -2984,7 +3006,12 @@ static int ram_save_setup(QEMUFile *f, void *opaque) > } > > migration_ops = g_malloc0(sizeof(MigrationOps)); > -migration_ops->ram_save_target_page = ram_save_target_page_legacy; > + > +if (migrate_multifd()) { > +migration_ops->ram_save_target_page = > ram_save_target_page_multifd; > +} else { > +migration_ops->ram_save_target_page = ram_save_target_page_legacy; > +} > > bql_unlock(); > ret = multifd_send_sync_main(); > -- > 2.30.2 > > > -- Elena

Re: [PATCH v2 3/7] migration/multifd: Zero page transmission on the multifd thread.

2024-02-21 Thread Elena Ufimtseva
ts.normal_pages, pages->normal_num); > stat64_add(&mig_stats.zero_pages, pages->zero_num); > > multifd_pages_reset(p->pages); > @@ -1256,7 +1285,8 @@ static void *multifd_recv_thread(void *opaque) > p->flags &= ~MULTIFD_FLAG_SYNC; > qemu_mutex_unlock(&p->mutex); > > -if (p->normal_num) { > +if (p->normal_num + p->zero_num) { > +assert(!(flags & MULTIFD_FLAG_SYNC)); > This assertion seems to be not relevant to this patch. Could you post it separately and explain why it's needed here? > ret = multifd_recv_state->ops->recv_pages(p, &local_err); > if (ret != 0) { > break; > diff --git a/migration/multifd.h b/migration/multifd.h > index 9822ff298a..125f0bbe60 100644 > --- a/migration/multifd.h > +++ b/migration/multifd.h > @@ -53,6 +53,11 @@ typedef struct { > uint32_t unused32[1];/* Reserved for future use */ > uint64_t unused64[3];/* Reserved for future use */ > char ramblock[256]; > +/* > + * This array contains the pointers to: > + * - normal pages (initial normal_pages entries) > + * - zero pages (following zero_pages entries) > + */ > uint64_t offset[]; > } __attribute__((packed)) MultiFDPacket_t; > > @@ -224,6 +229,8 @@ typedef struct { > > void multifd_register_ops(int method, MultiFDMethods *ops); > void multifd_send_fill_packet(MultiFDSendParams *p); > +void multifd_zero_page_check_send(MultiFDSendParams *p); > +void multifd_zero_page_check_recv(MultiFDRecvParams *p); > > static inline void multifd_send_prepare_header(MultiFDSendParams *p) > { > diff --git a/qapi/migration.json b/qapi/migration.json > index 99843a8e95..e2450b92d4 100644 > --- a/qapi/migration.json > +++ b/qapi/migration.json > @@ -660,9 +660,11 @@ > # > # @none: Do not perform zero page checking. > # > +# @multifd: Perform zero page checking on the multifd sender thread. > (since 9.0) > +# > ## > { 'enum': 'ZeroPageDetection', > - 'data': [ 'legacy', 'none' ] } > + 'data': [ 'legacy', 'none', 'multifd' ] } > > ## > # @BitmapMigrationBitmapAliasTransform: > -- > 2.30.2 > > > -- Elena

Re: [PATCH v2 2/7] migration/multifd: Support for zero pages transmission in multifd format.

2024-02-21 Thread Elena Ufimtseva
64 " pages %" PRIu64 > +multifd_recv_thread_end(uint8_t id, uint64_t packets, uint64_t > normal_pages, uint64_t zero_pages) "channel %u packets %" PRIu64 " normal > pages %" PRIu64 " zero pages %" PRIu64 > multifd_recv_thread_start(uint8_t id) "%u" > -multifd_send(uint8_t id, uint64_t packet_num, uint32_t normal, uint32_t > flags, uint32_t next_packet_size) "channel %u packet_num %" PRIu64 " normal > pages %u flags 0x%x next packet size %u" > +multifd_send(uint8_t id, uint64_t packet_num, uint32_t normal_pages, > uint32_t zero_pages, uint32_t flags, uint32_t next_packet_size) "channel %u > packet_num %" PRIu64 " normal pages %u zero pages %u flags 0x%x next packet > size %u" > multifd_send_error(uint8_t id) "channel %u" > multifd_send_sync_main(long packet_num) "packet num %ld" > multifd_send_sync_main_signal(uint8_t id) "channel %u" > multifd_send_sync_main_wait(uint8_t id) "channel %u" > multifd_send_terminate_threads(void) "" > -multifd_send_thread_end(uint8_t id, uint64_t packets, uint64_t > normal_pages) "channel %u packets %" PRIu64 " normal pages %" PRIu64 > +multifd_send_thread_end(uint8_t id, uint64_t packets, uint64_t > normal_pages, uint64_t zero_pages) "channel %u packets %" PRIu64 " normal > pages %" PRIu64 " zero pages %" PRIu64 > multifd_send_thread_start(uint8_t id) "%u" > multifd_tls_outgoing_handshake_start(void *ioc, void *tioc, const char > *hostname) "ioc=%p tioc=%p hostname=%s" > multifd_tls_outgoing_handshake_error(void *ioc, const char *err) "ioc=%p > err=%s" > -- > 2.30.2 > > > -- Elena

Re: [PATCH v2 1/7] migration/multifd: Add new migration option zero-page-detection.

2024-02-21 Thread Elena Ufimtseva
zero-page-detection'] } > > ## > # @MigrateSetParameters: > @@ -1066,6 +1081,10 @@ > # @mode: Migration mode. See description in @MigMode. Default is 'normal'. > #(Since 8.2) > # > +# @zero-page-detection: See description in @ZeroPageDetection. > +# Default is 'legacy'. (Since 9.0) > +# > +# > # Features: > # > # @deprecated: Member @block-incremental is deprecated. Use > @@ -1119,7 +1138,8 @@ > '*x-vcpu-dirty-limit-period': { 'type': 'uint64', > 'features': [ 'unstable' ] }, > '*vcpu-dirty-limit': 'uint64', > -'*mode': 'MigMode'} } > +'*mode': 'MigMode', > +'*zero-page-detection': 'ZeroPageDetection'} } > > ## > # @migrate-set-parameters: > @@ -1294,6 +1314,9 @@ > # @mode: Migration mode. See description in @MigMode. Default is 'normal'. > #(Since 8.2) > # > +# @zero-page-detection: See description in @ZeroPageDetection. > +# Default is 'legacy'. (Since 9.0) > +# > # Features: > # > # @deprecated: Member @block-incremental is deprecated. Use > @@ -1344,7 +1367,8 @@ > '*x-vcpu-dirty-limit-period': { 'type': 'uint64', > 'features': [ 'unstable' ] }, > '*vcpu-dirty-limit': 'uint64', > -'*mode': 'MigMode'} } > +'*mode': 'MigMode', > +'*zero-page-detection': 'ZeroPageDetection'} } > > ## > # @query-migrate-parameters: > -- > 2.30.2 > > > -- Elena

Re: [PATCH v3 3/6] util/bufferiszero: remove AVX512 variant

2024-02-06 Thread Elena Ufimtseva
OPT > -{ CPUINFO_AVX512F, buffer_zero_avx512 }, > -#endif > #ifdef CONFIG_AVX2_OPT > { CPUINFO_AVX2,buffer_zero_avx2 }, > #endif > @@ -191,7 +159,7 @@ static unsigned used_accel > = 0; > #endif > > -#if defined(CONFIG_AVX512F_OPT) || defined(CONFIG_AVX2_OPT) > +#if defined(CONFIG_AVX2_OPT) > static void __attribute__((constructor)) init_accel(void) > { > used_accel = select_accel_cpuinfo(cpuinfo_init()); > -- > 2.32.0 > > > -- Elena

Re: [PATCH v2 00/20] Use Intel DSA accelerator to offload zero page checking in multifd live migration.

2023-11-15 Thread Elena Ufimtseva
> linux-headers/linux/idxd.h | 356 ++ > meson.build|2 + > meson_options.txt |2 + > migration/migration-hmp-cmds.c | 22 + > migration/multifd-zlib.c |8 +- > migration/multifd-zstd.c | 8 +- > migration/multifd.c| 203 +- > migration/multifd.h| 28 +- > migration/options.c| 107 +++ > migration/options.h|4 + > migration/ram.c| 45 +- > migration/trace-events |8 +- > qapi/migration.json| 53 +- > scripts/meson-buildoptions.sh |3 + > tests/qtest/migration-test.c | 77 ++- > tests/unit/meson.build |6 + > tests/unit/test-dsa.c | 466 + > util/dsa.c | 1132 > util/meson.build |1 + > 20 files changed, 2612 insertions(+), 38 deletions(-) > create mode 100644 include/qemu/dsa.h > create mode 100644 linux-headers/linux/idxd.h > create mode 100644 tests/unit/test-dsa.c > create mode 100644 util/dsa.c > > -- > 2.30.2 > > -- Elena

[PATCH v2 2/4] multifd: document packet_len, next_packet_size

2023-10-11 Thread Elena Ufimtseva
used to indicate the size of the compressed buffer on source and destination. Will be it helpful to rename it as data_size or dirty_data_size? Signed-off-by: Elena Ufimtseva --- migration/multifd.h | 35 ++- 1 file changed, 30 insertions(+), 5 deletions(-) diff

[PATCH v2 0/4] multifd: various fixes

2023-10-11 Thread Elena Ufimtseva
and moved discussion of the other issues under "---". Thank you in advance and looking forward for your feedback. Elena Ufimtseva (4): migration: check for rate_limit_max for RATE_LIMIT_DISABLED multifd: document packet_len, next_packet_size multifd: fix counters in multifd_s

[PATCH v2 1/4] migration: check for rate_limit_max for RATE_LIMIT_DISABLED

2023-10-11 Thread Elena Ufimtseva
calling qemu_fflush() from migration_transferred_bytes() if the migration rate is not exceeded. This should be fine since migration thread calls in the loop migration_update_counters from migration_rate_limit() that calls the migration_transferred_bytes() and flushes there. Signed-off-by: Elena

[PATCH v2 4/4] multifd: reset next_packet_len after sending pages

2023-10-11 Thread Elena Ufimtseva
Sometimes multifd sends just sync packet with no pages (normal_num is 0). In this case the old value is being preserved and being accounted for while only packet_len is being transferred. Reset it to 0 after sending and accounting for. Signed-off-by: Elena Ufimtseva Reviewed-by: Fabiano Rosas

[PATCH v2 3/4] multifd: fix counters in multifd_send_thread

2023-10-11 Thread Elena Ufimtseva
t as iov[0]. The packet_len part should be added to account for the size of MultiFDPacket and the array of the offsets. Signed-off-by: Elena Ufimtseva Reviewed-by: Fabiano Rosas --- migration/multifd.c | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/migration/m

Re: [PATCH 1/4] multifd: wait for channels_ready before sending sync

2023-09-22 Thread Elena Ufimtseva
On Fri, Sep 22, 2023 at 01:06:53PM -0300, Fabiano Rosas wrote: > Elena Ufimtseva writes: > > > In multifd_send_sync_main we need to wait for channels_ready > > before submitting sync packet as the threads may still be sending > > their previous pages. > > Ther

Re: [RFC PATCH 1/3] migration/multifd: Move channels_ready semaphore

2023-09-22 Thread Elena Ufimtseva
]; > > -qemu_sem_wait(&multifd_send_state->channels_ready); > trace_multifd_send_sync_main_wait(p->id); > qemu_sem_wait(&p->sem_sync); > > @@ -763,8 +763,8 @@ out: > * who pay attention to me. > */

[PATCH 4/4] multifd: reset next_packet_len after sending pages

2023-09-21 Thread Elena Ufimtseva
multifd_send 136054.792 pid=18602 id=0x1 packet_num=0x574 normal=0x0 flags=0x0 next_packet_size=0x0 If there is a suggestion how to fix this properly, I will be glad to use it. Signed-off-by: Elena Ufimtseva --- migration/multifd.c | 1 + 1 file changed, 1 insertion(+) diff --git a/migration

[PATCH 0/4] multifd: various fixes

2023-09-21 Thread Elena Ufimtseva
/4, but fixes the over-accounting in case of sync only packet. Thank you in advance and looking forward for your feedback. Elena Elena Ufimtseva (4): multifd: wait for channels_ready before sending sync migration: check for rate_limit_max for RATE_LIMIT_DISABLED multifd: fix counters in

[PATCH 3/4] multifd: fix counters in multifd_send_thread

2023-09-21 Thread Elena Ufimtseva
t as iov[0]. The packet_len part should be added to account for the size of MultiFDPacket and the array of the offsets. Signed-off-by: Elena Ufimtseva --- migration/multifd.c | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/migration/multifd.c b/migration/multifd.c index

[PATCH 2/4] migration: check for rate_limit_max for RATE_LIMIT_DISABLED

2023-09-21 Thread Elena Ufimtseva
In migration rate limiting atomic operations are used to read the rate limit variables and transferred bytes and they are expensive. Check first if rate_limit_max is equal to RATE_LIMIT_DISABLED and return false immediately if so. Signed-off-by: Elena Ufimtseva --- migration/migration-stats.c

[PATCH 1/4] multifd: wait for channels_ready before sending sync

2023-09-21 Thread Elena Ufimtseva
start with waiting for channels_ready semaphore. Changes to commit 90b3cec351996dd8ef4eb847ad38607812c5e7f5 ("multifd: Fix the number of channels ready") Signed-off-by: Elena Ufimtseva --- migration/multifd.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/migration/m

Re: Call for agenda for 2023-09-19 QEMU developers call

2023-09-18 Thread Elena Ufimtseva
On Tue, Sep 19, 2023 at 02:02:49AM +0200, Juan Quintela wrote: > Elena Ufimtseva wrote: > > Hello Juan, > > > > Not sure if this is worth its own topic, would be it possible to hear > > the community thoughts on the live migration series review/pull > > progres

Re: Call for agenda for 2023-09-19 QEMU developers call

2023-09-18 Thread Elena Ufimtseva
; j...@nvidia.com ; c...@nvidia.com ; David Edmondson ; Elena Ufimtseva ; Konrad Wilk ; Alessandro Di Federico ; a...@rev.ng ; shameerali.kolothum.th...@huawei.com ; wei.w.w...@intel.com ; chao.p.p...@linux.intel.com ; qemu-devel@nongnu.org ; Mark Burton Subject: Call for agenda for 2023-09

Re: ioregionfd with io_uring IORING_OP_URING_CMD

2022-06-07 Thread Elena
On Mon, Jun 06, 2022 at 03:57:47PM +0100, Stefan Hajnoczi wrote: > Hi, > During Elena Afanasova's Outreachy project we discussed whether > ioregionfd should be a custom struct file_operations (anon inode) or a > userspace-provided file (socketpair, UNIX domain socket, etc)

Re: [RFC 4/8] ioregionfd: Introduce IORegionDFObject type

2022-02-15 Thread Elena
On Fri, Feb 11, 2022 at 02:46:47PM +0100, Markus Armbruster wrote: > Elena Ufimtseva writes: > > > Signed-off-by: Elena Ufimtseva > > [...] > > > diff --git a/qapi/qom.json b/qapi/qom.json > > index eeb5395ff3..439fb94c93 100644 > > --- a/qapi/qom.jso

Re: [RFC 4/8] ioregionfd: Introduce IORegionDFObject type

2022-02-15 Thread Elena
On Mon, Feb 14, 2022 at 02:37:21PM +, Stefan Hajnoczi wrote: > On Mon, Feb 07, 2022 at 11:22:18PM -0800, Elena Ufimtseva wrote: > > Signed-off-by: Elena Ufimtseva > > --- > > meson.build| 15 ++- > > qapi/qom.json | 32 ++

Re: [RFC 0/8] ioregionfd introduction

2022-02-15 Thread Elena
On Mon, Feb 14, 2022 at 02:52:29PM +, Stefan Hajnoczi wrote: > On Mon, Feb 07, 2022 at 11:22:14PM -0800, Elena Ufimtseva wrote: > > This patchset is an RFC version for the ioregionfd implementation > > in QEMU. The kernel patches are to be posted with some fixes as a v4. &g

[RFC 5/8] multiprocess: prepare ioregionfds for remote device

2022-02-08 Thread Elena Ufimtseva
Signed-off-by: Elena Ufimtseva --- include/hw/remote/ioregionfd.h | 1 + include/hw/remote/remote.h | 1 + hw/remote/ioregionfd.c | 26 ++ hw/remote/remote-obj.c | 93 ++ 4 files changed, 121 insertions(+) diff --git a/include/hw

[RFC 6/8] multiprocess: add MPQEMU_CMD_BAR_INFO

2022-02-08 Thread Elena Ufimtseva
This command is used to request the bar type info from remote device. Signed-off-by: Elena Ufimtseva --- include/hw/remote/ioregionfd.h | 2 ++ include/hw/remote/machine.h | 1 + include/hw/remote/mpqemu-link.h | 2 ++ hw/remote/ioregionfd.c | 28 hw

[RFC 4/8] ioregionfd: Introduce IORegionDFObject type

2022-02-08 Thread Elena Ufimtseva
Signed-off-by: Elena Ufimtseva --- meson.build| 15 ++- qapi/qom.json | 32 +- include/hw/remote/ioregionfd.h | 40 +++ hw/remote/ioregionfd.c | 196 + Kconfig.host | 3 + MAINTAINERS

[RFC 3/8] ioregionfd: introduce memory API functions

2022-02-08 Thread Elena Ufimtseva
Similar to ioeventfd, introduce the ioregionfd functions to add and delete ioregionfds. Signed-off-by: Elena Ufimtseva --- softmmu/memory.c | 207 +++ 1 file changed, 207 insertions(+) diff --git a/softmmu/memory.c b/softmmu/memory.c index 7340e19ff5

[RFC 8/8] multiprocess: handle ioregionfd commands

2022-02-08 Thread Elena Ufimtseva
Signed-off-by: Elena Ufimtseva --- include/hw/remote/ioregionfd.h | 2 + include/hw/remote/remote.h | 2 + linux-headers/ioregionfd.h | 30 + hw/remote/ioregionfd.c | 111 + hw/remote/remote-obj.c | 44 + 5 files

[RFC 0/8] ioregionfd introduction

2022-02-08 Thread Elena Ufimtseva
uential write read IOPS read BW write IOPS write BW no ioregionfd367k 1434MiB/s 76k 297MiB/s ioregionfd 374k 1459MiB/s 77.3k302MiB/s Please review and send your feedback. Thank you! Elena Elena Ufimtseva (8): ioregionfd:

[RFC 7/8] multiprocess: add ioregionfd memory region in proxy

2022-02-08 Thread Elena Ufimtseva
Signed-off-by: Elena Ufimtseva --- include/hw/remote/proxy.h | 1 + hw/remote/proxy.c | 66 --- 2 files changed, 63 insertions(+), 4 deletions(-) diff --git a/include/hw/remote/proxy.h b/include/hw/remote/proxy.h index 741def71f1..9efef0b935 100644

[RFC 2/8] multiprocess: place RemoteObject definition in a header file

2022-02-08 Thread Elena Ufimtseva
This will be needed later. No functional changes. Signed-off-by: Elena Ufimtseva --- include/hw/remote/remote.h | 28 hw/remote/remote-obj.c | 16 +--- MAINTAINERS| 1 + 3 files changed, 30 insertions(+), 15 deletions(-) create mode

[RFC 1/8] ioregionfd: introduce a syscall and memory API

2022-02-08 Thread Elena Ufimtseva
Signed-off-by: Elena Ufimtseva --- include/exec/memory.h | 50 +++ include/sysemu/kvm.h | 15 + linux-headers/linux/kvm.h | 25 accel/kvm/kvm-all.c | 132 ++ accel/stubs/kvm-stub.c| 1 + 5 files changed, 223

Re: MMIO/PIO dispatch file descriptors (ioregionfd) design discussion

2021-10-25 Thread Elena
On Mon, Oct 25, 2021 at 01:42:56PM +0100, Stefan Hajnoczi wrote: > On Mon, Oct 11, 2021 at 10:34:29PM -0700, elena wrote: > > On Wed, Nov 25, 2020 at 12:44:07PM -0800, Elena Afanasova wrote: > > > Hello, > > > > > > > Hi > > > > Sorry for top-po

Re: MMIO/PIO dispatch file descriptors (ioregionfd) design discussion

2021-10-11 Thread elena
On Wed, Nov 25, 2020 at 12:44:07PM -0800, Elena Afanasova wrote: > Hello, > Hi Sorry for top-posting, just wanted to provide a quik update. We are currently working on the support for ioregionfd in Qemu and will be posting the patches soon. Plus the KVM patches will be posted based of the

[PATCH RFC v2 16/16] vfio-user: migration support

2021-08-16 Thread Elena Ufimtseva
From: John Johnson Signed-off-by: John G Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman --- hw/vfio/user-protocol.h | 18 + hw/vfio/user.h | 3 +++ hw/vfio/common.c| 23 - hw/vfio/migration.c | 34

[PATCH RFC v2 12/16] vfio-user: proxy container connect/disconnect

2021-08-16 Thread Elena Ufimtseva
From: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- include/hw/vfio/vfio-common.h | 3 ++ hw/vfio/common.c | 84 +++ hw/vfio/pci.c | 22 + 3 files changed

[PATCH RFC v2 13/16] vfio-user: dma map/unmap operations

2021-08-16 Thread Elena Ufimtseva
From: John Johnson Signed-off-by: Jagannathan Raman Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson --- hw/vfio/user-protocol.h | 32 + hw/vfio/user.h| 6 ++ include/hw/vfio/vfio-common.h | 1 + hw/vfio/common.c | 71

[PATCH RFC v2 01/16] vfio-user: introduce vfio-user protocol specification

2021-08-16 Thread Elena Ufimtseva
From: Thanos Makatos This patch introduces the vfio-user protocol specification (formerly known as VFIO-over-socket), which is designed to allow devices to be emulated outside QEMU, in a separate process. vfio-user reuses the existing VFIO defines, structs and concepts. This patch is sourced fro

[PATCH RFC v2 10/16] vfio-user: pci_user_realize PCI setup

2021-08-16 Thread Elena Ufimtseva
From: John Johnson PCI BARs read from remote device PCI config reads/writes sent to remote server Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- hw/vfio/pci.c | 210 +- 1 file changed, 175

[PATCH RFC v2 02/16] vfio-user: add VFIO base abstract class

2021-08-16 Thread Elena Ufimtseva
From: John Johnson Add an abstract base class both the kernel driver and user socket implementations can use to share code. Signed-off-by: John G Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman --- hw/vfio/pci.h | 16 +++-- hw/vfio/pci.c | 63

[PATCH RFC v2 08/16] vfio-user: get region info

2021-08-16 Thread Elena Ufimtseva
From: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- hw/vfio/user-protocol.h | 14 +++ hw/vfio/user.h| 2 + include/hw/vfio/vfio-common.h | 3 ++ hw/vfio/common.c | 76

[PATCH RFC v2 14/16] vfio-user: dma read/write operations

2021-08-16 Thread Elena Ufimtseva
From: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- hw/vfio/user-protocol.h | 11 +++ hw/vfio/user.h | 1 + hw/vfio/pci.c | 63 - hw/vfio/user.c | 7

[PATCH RFC v2 15/16] vfio-user: pci reset

2021-08-16 Thread Elena Ufimtseva
From: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- hw/vfio/user.h | 1 + hw/vfio/pci.c | 29 ++--- hw/vfio/user.c | 12 3 files changed, 39 insertions(+), 3 deletions(-) diff --git a/hw

[PATCH RFC v2 11/16] vfio-user: get and set IRQs

2021-08-16 Thread Elena Ufimtseva
From: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- hw/vfio/user-protocol.h | 25 ++ hw/vfio/user.h | 2 + hw/vfio/common.c| 26 -- hw/vfio/pci.c | 31 ++-- hw/vfio/user.c

[PATCH RFC v2 06/16] vfio-user: negotiate version with remote server

2021-08-16 Thread Elena Ufimtseva
From: John Johnson Signed-off-by: Jagannathan Raman Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson --- hw/vfio/user-protocol.h | 23 hw/vfio/user.h | 1 + hw/vfio/pci.c | 9 ++ hw/vfio/user.c | 267

[PATCH RFC v2 07/16] vfio-user: get device info

2021-08-16 Thread Elena Ufimtseva
From: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- hw/vfio/user-protocol.h | 13 + hw/vfio/user.h | 1 + hw/vfio/pci.c | 13 + hw/vfio/user.c | 20 4

[PATCH RFC v2 05/16] vfio-user: define VFIO Proxy and communication functions

2021-08-16 Thread Elena Ufimtseva
From: John Johnson Signed-off-by: John G Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman --- hw/vfio/user-protocol.h | 62 + hw/vfio/user.h | 8 ++ hw/vfio/pci.c | 6 + hw/vfio/user.c | 289

[PATCH RFC v2 03/16] vfio-user: Define type vfio_user_pci_dev_info

2021-08-16 Thread Elena Ufimtseva
From: John Johnson New class for vfio-user with its class and instance constructors and destructors, and its pci ops. Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- hw/vfio/pci.h | 9 ++ hw/vfio/pci.c | 86

[PATCH RFC v2 04/16] vfio-user: connect vfio proxy to remote server

2021-08-16 Thread Elena Ufimtseva
From: John Johnson Signed-off-by: John G Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman --- hw/vfio/user.h| 66 ++ include/hw/vfio/vfio-common.h | 2 + hw/vfio/pci.c | 29 ++ hw/vfio/user.c| 160

[PATCH RFC v2 00/16] vfio-user implementation

2021-08-16 Thread Elena Ufimtseva
also worked on implementing a client and will be sending this patch series shortly. Contributors: John G Johnson John Levon Thanos Makatos Elena Ufimtseva Jagannathan Raman Changes in v2: - combine some patches with relevant functionality. - use SocketAddress with idea to modify late

[PATCH RFC v2 09/16] vfio-user: region read/write

2021-08-16 Thread Elena Ufimtseva
From: John Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- hw/vfio/user-protocol.h | 12 hw/vfio/user.h | 4 hw/vfio/common.c| 16 +-- hw/vfio/user.c | 43

[PATCH RFC 18/19] vfio-user: migration support

2021-07-18 Thread Elena Ufimtseva
From: John G Johnson Send migration region operations to remote server. Send VFIO_USER_USER_DIRTY_PAGES to get remote dirty bitmap. Signed-off-by: John G Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman --- hw/vfio/user.h | 17 +++ hw/vfio/common.c

[PATCH RFC 13/19] vfio-user: respond to remote DMA read/write requests

2021-07-18 Thread Elena Ufimtseva
From: John G Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- hw/vfio/user.h | 16 hw/vfio/pci.c | 67 ++ hw/vfio/user.c | 21 +++- 3 files changed, 103

[PATCH RFC 16/19] vfio-user: pci reset

2021-07-18 Thread Elena Ufimtseva
From: John G Johnson Send VFIO_USER_DEVICE_RESET to reset remote device. Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- hw/vfio/user.h | 1 + hw/vfio/pci.c | 29 ++--- hw/vfio/user.c | 12 3 files

[PATCH RFC 19/19] vfio-user: add migration cli options and version negotiation

2021-07-18 Thread Elena Ufimtseva
From: John G Johnson Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman Signed-off-by: Elena Ufimtseva --- hw/vfio/user.h | 4 hw/vfio/pci.c | 5 + hw/vfio/user.c | 33 + 3 files changed, 42 insertions(+) diff --git a/hw/vfio/user.h b

[PATCH RFC 11/19] vfio-user: get region and DMA map/unmap operations

2021-07-18 Thread Elena Ufimtseva
From: John G Johnson Send VFIO_USER_DEVICE_GET_REGION_INFO to get device regions and VFIO_USER_DMA_MAP/UNMAP to tell remote server the DMA addresses it can access. Signed-off-by: Jagannathan Raman Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson --- hw/vfio/user.h

[PATCH RFC 17/19] vfio-user: probe remote device ROM BAR

2021-07-18 Thread Elena Ufimtseva
From: John G Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- hw/vfio/pci.c | 38 ++ 1 file changed, 30 insertions(+), 8 deletions(-) diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index a6c28dac03

[PATCH RFC 00/19] vfio-user implementation

2021-07-18 Thread Elena Ufimtseva
ll be sending this patch series shortly. Contributors: John G Johnson John Levon Thanos Makatos Elena Ufimtseva Jagannathan Raman Please send your comments and questions! Thank you. References: [1] https://wiki.qemu.org/Features/MultiProcessQEMU [2] https://patchwork.kernel.org/project/qemu-

[PATCH RFC 01/19] vfio-user: introduce vfio-user protocol specification

2021-07-18 Thread Elena Ufimtseva
From: Thanos Makatos This patch introduces the vfio-user protocol specification (formerly known as VFIO-over-socket), which is designed to allow devices to be emulated outside QEMU, in a separate process. vfio-user reuses the existing VFIO defines, structs and concepts. This patch is sourced fro

[PATCH RFC 09/19] vfio-user: get device info and get irq info

2021-07-18 Thread Elena Ufimtseva
From: John G Johnson Send VFIO_USER_DEVICE_GET_INFO and VFIO_USER_DEVICE_GET_IRQ_INFO commands. Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- hw/vfio/user.h | 27 +++ hw/vfio/pci.c | 32

[PATCH RFC 12/19] vfio-user: probe remote device's BARs

2021-07-18 Thread Elena Ufimtseva
From: John G Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- hw/vfio/pci.c | 32 +++- 1 file changed, 27 insertions(+), 5 deletions(-) diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index 054e673552

[PATCH RFC 14/19] vfio_user: setup MSI/X interrupts and PCI config operations

2021-07-18 Thread Elena Ufimtseva
From: John G Johnson Send VFIO_USER_DEVICE_SET_IRQS to setup interrup configuration. vfio_pci_write_config/vfio_pci_read_config iforms the remote server of PCI config space reads and writes. Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- hw

[PATCH RFC 15/19] vfio-user: vfio user device realize

2021-07-18 Thread Elena Ufimtseva
From: John G Johnson Setup INTx interrupts and a device region info cache for remote device info. Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman Signed-off-by: Elena Ufimtseva --- include/hw/vfio/vfio-common.h | 1 + hw/vfio/common.c | 33

[PATCH RFC 06/19] vfio-user: negotiate protocol with remote server

2021-07-18 Thread Elena Ufimtseva
From: John G Johnson Send version and capabilities and validate reply. Signed-off-by: Jagannathan Raman Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson --- hw/vfio/user.h | 8 ++ hw/vfio/pci.c | 10 +++ hw/vfio/user.c | 223

[PATCH RFC 02/19] vfio-user: add VFIO base abstract class

2021-07-18 Thread Elena Ufimtseva
From: John G Johnson Add an abstract base class both the kernel driver and user socket implementations can use to share code. Signed-off-by: John G Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman --- hw/vfio/pci.h | 25 ++-- hw/vfio/pci.c | 63

[PATCH RFC 05/19] vfio-user: connect vfio proxy to remote server

2021-07-18 Thread Elena Ufimtseva
From: John G Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- hw/vfio/user.h | 2 ++ hw/vfio/pci.c | 16 ++ hw/vfio/user.c | 87 ++ 3 files changed, 105 insertions(+) diff

[PATCH RFC 10/19] vfio-user: device region read/write

2021-07-18 Thread Elena Ufimtseva
From: John G Johnson Send VFIO_REGION_READ and VFIO_REGION_WRITE commands. Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- hw/vfio/user.h | 16 hw/vfio/common.c | 17 +++-- hw/vfio/pci.c| 13

[PATCH RFC 07/19] vfio-user: define vfio-user pci ops

2021-07-18 Thread Elena Ufimtseva
From: John G Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- hw/vfio/pci.c | 31 +++ 1 file changed, 31 insertions(+) diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index 8ca1431cca..388b7d82d7 100644 --- a/hw

[PATCH RFC 03/19] vfio-user: define VFIO Proxy and communication functions

2021-07-18 Thread Elena Ufimtseva
From: John G Johnson Add user.c and user.h files for vfio-user with the basic send and receive functions. Signed-off-by: John G Johnson Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman --- hw/vfio/user.h| 120 ++ include/hw/vfio/vfio-common.h | 2

[PATCH RFC 08/19] vfio-user: VFIO container setup & teardown

2021-07-18 Thread Elena Ufimtseva
From: John G Johnson Create SW-emulated containers and groups for vfio-user in lieu of the host IOMMU based ones used by the kernel driver VFIO implementation. Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- include/hw/vfio/vfio-common.h

[PATCH RFC 04/19] vfio-user: Define type vfio_user_pci_dev_info

2021-07-18 Thread Elena Ufimtseva
From: John G Johnson New class for vfio-user with its class and instance constructors and destructors. Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- hw/vfio/pci.c | 49 + 1 file changed, 49

Re: [PATCH v21 00/20] Initial support for multi-process Qemu

2021-03-12 Thread elena
have posted 20 > > versions on the qemu-devel mailing list. You can find them by following > > the links below ([1] - [20]). Following people contributed to the design and > > implementation of this project: > > Jagannathan Raman > > Elena Ufimtseva > > John G John

[PATCH RFC] hw/misc/pc-testdev: add support for ioregionfd testing

2021-03-01 Thread Elena Afanasova
Signed-off-by: Elena Afanasova --- hw/misc/pc-testdev.c | 74 +++ include/sysemu/kvm.h | 4 +-- linux-headers/linux/kvm.h | 24 + 3 files changed, 100 insertions(+), 2 deletions(-) diff --git a/hw/misc/pc-testdev.c b/hw/misc/pc

Re: [PULL v2 00/27] Block patches

2021-02-04 Thread elena
mpile, clang: > > ../../hw/remote/mpqemu-link.c:40:29: error: suggest braces around > initialization of subobject [-Werror,-Wmissing-braces] > struct iovec send[2] = {0}; > ^ > {} Stefan, should we make changes for the patch? Please let us know. Thank

Re: [PATCH v21 00/20] Initial support for multi-process Qemu

2021-02-04 Thread elena
On Thu, Feb 04, 2021 at 09:51:45AM +, Daniel P. Berrangé wrote: > On Wed, Feb 03, 2021 at 12:02:05PM -0800, Elena Ufimtseva wrote: > > On Wed, Feb 03, 2021 at 04:11:50PM +, Stefan Hajnoczi wrote: > > > On Fri, Jan 29, 2021 at 11:46:01AM -0500, Jagannathan Raman wrote: &

Re: [PATCH v21 00/20] Initial support for multi-process Qemu

2021-02-03 Thread Elena Ufimtseva
ubmission. > Thanks, applied to my block tree: > https://gitlab.com/stefanha/qemu/commits/block > Thank you Stefan! Elena > Stefan

Re: [PATCH] multi-process: Acceptance test for multiprocess QEMU

2020-12-23 Thread Elena Ufimtseva
On Wed, Dec 23, 2020 at 03:01:24PM +0400, Marc-André Lureau wrote: > Hi > > On Wed, Dec 23, 2020 at 10:45 AM wrote: > > > From: Jagannathan Raman > > > > Runs the Avocado acceptance test to check if a > > remote lsi53c895a device gets identified by the

[PATCH v15 18/20] multi-process: create IOHUB object to handle irq

2020-12-22 Thread elena . ufimtseva
-off-by: John G Johnson Signed-off-by: Jagannathan Raman Signed-off-by: Elena Ufimtseva Reviewed-by: Stefan Hajnoczi --- include/hw/pci/pci_ids.h| 3 + include/hw/remote/iohub.h | 42 +++ include/hw/remote/machine.h | 3 + include/hw/remote/mpqemu-link.h | 1

[PATCH v15 19/20] multi-process: Retrieve PCI info from remote process

2020-12-22 Thread elena . ufimtseva
From: Jagannathan Raman Retrieve PCI configuration info about the remote device and configure the Proxy PCI object based on the returned information Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman Reviewed-by: Stefan Hajnoczi --- hw/remote

[PATCH v15 16/20] multi-process: PCI BAR read/write handling for proxy & remote endpoints

2020-12-22 Thread elena . ufimtseva
. Signed-off-by: Jagannathan Raman Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Reviewed-by: Stefan Hajnoczi --- include/hw/remote/mpqemu-link.h | 10 include/hw/remote/proxy.h | 10 hw/remote/message.c | 83 + hw/remote

[PATCH v15 08/20] multi-process: add qio channel read function

2020-12-22 Thread elena . ufimtseva
From: Elena Ufimtseva Adds qio_channel_readv_full_all() to read both data and FDs. Refactors existing code to use this function. Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- include/io/channel.h | 25 + io/channel.c

[PATCH v15 06/20] multi-process: setup a machine object for remote device process

2020-12-22 Thread elena . ufimtseva
From: Jagannathan Raman x-remote-machine object sets up various subsystems of the remote device process. Instantiate PCI host bridge object and initialize RAM, IO & PCI memory regions. Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman Signed-off-by: Elena Ufimtseva Reviewe

[PATCH] multi-process: Acceptance test for multiprocess QEMU

2020-12-22 Thread elena . ufimtseva
From: Jagannathan Raman Runs the Avocado acceptance test to check if a remote lsi53c895a device gets identified by the guest. Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman --- tests/acceptance/multiprocess.py | 104

[PATCH v15 07/20] multi-process: add qio channel write function

2020-12-22 Thread elena . ufimtseva
From: Elena Ufimtseva Adds qio_channel_writev_full_all() to transmit both data and FDs. Refactors existing code to use this function. Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman Reviewed-by: Stefan Hajnoczi --- include/io/channel.h | 25

[PATCH v15 20/20] multi-process: perform device reset in the remote process

2020-12-22 Thread elena . ufimtseva
From: Elena Ufimtseva Perform device reset in the remote process when QEMU performs device reset. This is required to reset the internal state (like registers, etc...) of emulated devices Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman Reviewed

[PATCH v15 09/20] multi-process: define MPQemuMsg format and transmission functions

2020-12-22 Thread elena . ufimtseva
From: Elena Ufimtseva Defines MPQemuMsg, which is the message that is sent to the remote process. This message is sent over QIOChannel and is used to command the remote process to perform various tasks. Define transmission functions used by proxy and by remote. Signed-off-by: Jagannathan Raman

[PATCH v15 12/20] multi-process: setup memory manager for remote device

2020-12-22 Thread elena . ufimtseva
. Signed-off-by: Jagannathan Raman Signed-off-by: John G Johnson Signed-off-by: Elena Ufimtseva Reviewed-by: Stefan Hajnoczi --- include/hw/remote/memory.h | 19 ++ include/hw/remote/mpqemu-link.h | 10 + hw/remote/memory.c | 65 + hw

[PATCH v15 17/20] multi-process: Synchronize remote memory

2020-12-22 Thread elena . ufimtseva
changes to memory, the remote process receives the message and processes it in the handler for SYNC_SYSMEM message. Signed-off-by: Jagannathan Raman Signed-off-by: John G Johnson Signed-off-by: Elena Ufimtseva Reviewed-by: Stefan Hajnoczi --- include/hw/remote/proxy-memory-listener.h | 28

[PATCH v15 10/20] multi-process: Initialize message handler in remote device

2020-12-22 Thread elena . ufimtseva
From: Jagannathan Raman Initializes the message handler function in the remote process. It is called whenever there's an event pending on QIOChannel that registers this function. Signed-off-by: Elena Ufimtseva Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman Reviewed-by: S

[PATCH v15 15/20] multi-process: Forward PCI config space acceses to the remote process

2020-12-22 Thread elena . ufimtseva
From: Elena Ufimtseva The Proxy Object sends the PCI config space accesses as messages to the remote process over the communication channel Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman Signed-off-by: John G Johnson Reviewed-by: Stefan Hajnoczi --- include/hw/remote

[PATCH v15 13/20] multi-process: introduce proxy object

2020-12-22 Thread elena . ufimtseva
From: Elena Ufimtseva Defines a PCI Device proxy object as a child of TYPE_PCI_DEVICE. Signed-off-by: Elena Ufimtseva Signed-off-by: Jagannathan Raman Signed-off-by: John G Johnson Reviewed-by: Stefan Hajnoczi --- include/hw/remote/proxy.h | 36 ++ hw/remote/proxy.c

[PATCH v15 00/20] Initial support for multi-process Qemu

2020-12-22 Thread elena . ufimtseva
From: Elena Ufimtseva Hi This is the v15 of the patchset. Thank you for your time reviewing v14. This version has the following changes: - [PATCH v15 08/20] multi-process: add qio channel read function Prevent memory leaks by closing file descriptors and freeing memory for fd array on

[PATCH v15 03/20] memory: alloc RAM from file at offset

2020-12-22 Thread elena . ufimtseva
From: Jagannathan Raman Allow RAM MemoryRegion to be created from an offset in a file, instead of allocating at offset of 0 by default. This is needed to synchronize RAM between QEMU & remote process. Signed-off-by: Jagannathan Raman Signed-off-by: John G Johnson Signed-off-by: E

[PATCH v15 04/20] multi-process: Add config option for multi-process QEMU

2020-12-22 Thread elena . ufimtseva
From: Jagannathan Raman Add configuration options to enable or disable multiprocess QEMU code Signed-off-by: John G Johnson Signed-off-by: Jagannathan Raman Signed-off-by: Elena Ufimtseva Reviewed-by: Stefan Hajnoczi --- configure | 10 ++ meson.build | 2

  1   2   3   4   >