Unlike the kernel, vhost-user backend knows nothing about QEMU's userspace addresses. We can pass GPA instead and nothing changes. Generally, vhost-user servers need these addresses only to calculate offsets inside memory regions. Still, some servers (QEMU's internal is one example) may do checks for passed addresses to be "userspace addresses", for example check for non-zero. That's why we need additional negotiation for the feature.
The benefit: this opens the doors for further implementation of local migration (live-update) with passing open vhost-related FDs through UNIX domain socket. This way the connection with backend is kept live and untouched. Without this change, we would have to communicate with backend to inform it about UVA address changes, but it's better to simply use more stable GPA numbers, which don't change after migration. Additionally, the current implementation exposes QEMU's process address space by passing UVA, which breaks ASLR. New protocol feature avoids that. Note, that we do nothing with backend messages and replies. Frontends have to work with backends userspace addresses anyway, because they come from userfaultfd. Signed-off-by: Vladimir Sementsov-Ogievskiy <[email protected]> --- docs/interop/vhost-user.rst | 21 +++++++++++++++++---- hw/virtio/vhost-user.c | 22 +++++++++++++++++----- include/hw/virtio/vhost-user.h | 1 + 3 files changed, 35 insertions(+), 9 deletions(-) diff --git a/docs/interop/vhost-user.rst b/docs/interop/vhost-user.rst index 137c9f3669..8dcb030c53 100644 --- a/docs/interop/vhost-user.rst +++ b/docs/interop/vhost-user.rst @@ -164,8 +164,16 @@ A vring address description :log: a 64-bit guest address for logging -Note that a ring address is an IOVA if ``VIRTIO_F_IOMMU_PLATFORM`` has -been negotiated. Otherwise it is a user address. +.. Note:: + When ``VIRTIO_F_IOMMU_PLATFORM`` is negotiated, ring addresses are IOVAs. + + Otherwise, when ``VHOST_USER_PROTOCOL_F_GPA_ADDRESSES`` is negotiated, the + ring addresses are guest physical addresses for frontend messages. That + does not apply to backend replies. + + Finally, when neither ``VIRTIO_F_IOMMU_PLATFORM`` nor + ``VHOST_USER_PROTOCOL_F_GPA_ADDRESSES`` features are negotiated, ring + addresses are user virtual addresses. .. _memory_region_description: @@ -180,7 +188,9 @@ Memory region description :size: a 64-bit size -:user address: a 64-bit user address +:user address: a 64-bit user address. When ``VHOST_USER_PROTOCOL_F_GPA_ADDRESSES`` + is negotiated, this field contain guest physical address instead and must + duplicate ``guest address`` field. :mmap offset: a 64-bit offset where region starts in the mapped memory @@ -252,7 +262,9 @@ An IOTLB message :size: a 64-bit size -:user address: a 64-bit user address +:user address: a 64-bit user address. When ``VHOST_USER_PROTOCOL_F_GPA_ADDRESSES`` + is negotiated, this field contain guest physical address instead, except for + ``VHOST_USER_BACKEND_IOTLB_MSG``, where it's user address anyway. :permissions flags: an 8-bit value: - 0: No access @@ -1063,6 +1075,7 @@ Protocol features #define VHOST_USER_PROTOCOL_F_SHARED_OBJECT 18 #define VHOST_USER_PROTOCOL_F_DEVICE_STATE 19 #define VHOST_USER_PROTOCOL_F_GET_VRING_BASE_INFLIGHT 20 + #define VHOST_USER_PROTOCOL_F_GPA_ADDRESSES 21 Front-end message types ----------------------- diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c index 2e58f38ac5..125b170c5b 100644 --- a/hw/virtio/vhost-user.c +++ b/hw/virtio/vhost-user.c @@ -568,12 +568,22 @@ static MemoryRegion *vhost_user_get_mr_data(uint64_t addr, ram_addr_t *offset, return mr; } -static void vhost_user_fill_msg_region(VhostUserMemoryRegion *dst, +static bool vhost_user_gpa_addresses(struct vhost_dev *dev) +{ + return vhost_user_has_protocol_feature( + dev, VHOST_USER_PROTOCOL_F_GPA_ADDRESSES); +} + +static void vhost_user_fill_msg_region(struct vhost_dev *dev, + VhostUserMemoryRegion *dst, struct vhost_memory_region *src, uint64_t mmap_offset) { + bool use_phys = vhost_user_gpa_addresses(dev); + assert(src != NULL && dst != NULL); - dst->userspace_addr = src->userspace_addr; + + dst->userspace_addr = use_phys ? src->guest_phys_addr : src->userspace_addr; dst->memory_size = src->memory_size; dst->guest_phys_addr = src->guest_phys_addr; dst->mmap_offset = mmap_offset; @@ -611,7 +621,7 @@ static int vhost_user_fill_set_mem_table_msg(struct vhost_user *u, error_report("Failed preparing vhost-user memory table msg"); return -ENOBUFS; } - vhost_user_fill_msg_region(®ion_buffer, reg, offset); + vhost_user_fill_msg_region(dev, ®ion_buffer, reg, offset); msg->payload.memory.regions[*fd_num] = region_buffer; fds[(*fd_num)++] = fd; } else if (track_ramblocks) { @@ -757,7 +767,7 @@ static int send_remove_regions(struct vhost_dev *dev, if (fd > 0) { msg->hdr.request = VHOST_USER_REM_MEM_REG; - vhost_user_fill_msg_region(®ion_buffer, shadow_reg, 0); + vhost_user_fill_msg_region(dev, ®ion_buffer, shadow_reg, 0); msg->payload.mem_reg.region = region_buffer; ret = vhost_user_write(dev, msg, NULL, 0); @@ -818,7 +828,7 @@ static int send_add_regions(struct vhost_dev *dev, u->region_rb[reg_idx] = mr->ram_block; } msg->hdr.request = VHOST_USER_ADD_MEM_REG; - vhost_user_fill_msg_region(®ion_buffer, reg, offset); + vhost_user_fill_msg_region(dev, ®ion_buffer, reg, offset); msg->payload.mem_reg.region = region_buffer; ret = vhost_user_write(dev, msg, &fd, 1); @@ -3154,4 +3164,6 @@ const VhostOps user_ops = { .vhost_supports_device_state = vhost_user_supports_device_state, .vhost_set_device_state_fd = vhost_user_set_device_state_fd, .vhost_check_device_state = vhost_user_check_device_state, + .vhost_phys_vring_addr = vhost_user_gpa_addresses, + .vhost_phys_iotlb_msg = vhost_user_gpa_addresses, }; diff --git a/include/hw/virtio/vhost-user.h b/include/hw/virtio/vhost-user.h index 0e576fc538..5ab70dab7e 100644 --- a/include/hw/virtio/vhost-user.h +++ b/include/hw/virtio/vhost-user.h @@ -34,6 +34,7 @@ enum VhostUserProtocolFeature { VHOST_USER_PROTOCOL_F_SHARED_OBJECT = 18, VHOST_USER_PROTOCOL_F_DEVICE_STATE = 19, VHOST_USER_PROTOCOL_F_GET_VRING_BASE_INFLIGHT = 20, + VHOST_USER_PROTOCOL_F_GPA_ADDRESSES = 21, VHOST_USER_PROTOCOL_F_MAX }; -- 2.52.0
