Hello Paolo,
On Tue, Mar 4, 2014 at 8:29 PM, Paolo Bonzini <[email protected]> wrote: > Il 04/03/2014 19:22, Antonios Motakis ha scritto: > > In this patch series we would like to introduce our approach for putting a >> virtio-net backend in an external userspace process. Our eventual target >> is to >> run the network backend in the Snabbswitch ethernet switch, while >> receiving >> traffic from a guest inside QEMU/KVM which runs an unmodified virtio-net >> implementation. >> >> For this, we are working into extending vhost to allow equivalent >> functionality >> for userspace. Vhost already passes control of the data plane of >> virtio-net to >> the host kernel; we want to realize a similar model, but for userspace. >> >> In this patch series the concept of a vhost-backend is introduced. >> >> We define two vhost backend types - vhost-kernel and vhost-user. The >> former is >> the interface to the current kernel module implementation. Its control >> plane is >> ioctl based. The data plane is realized by the kernel directly accessing >> the >> QEMU allocated, guest memory. >> >> In the new vhost-user backend, the control plane is based on communication >> between QEMU and another userspace process using a unix domain socket. >> This >> allows to implement a virtio backend for a guest running in QEMU, inside >> the >> other userspace process. For this communication we use a chardev with a >> Unix >> domain socket backend. Vhost-user is client/server agnostic regarding the >> chardev, however it does not support the 'nowait' and 'telnet' options. >> >> A reconnection in 'server' mode is supported, but the backend's exposed >> virtio >> features need to be compatible with the first connected slave. >> >> We change -mem-path to QemuOpts and add prealloc and share as properties >> to it. HugeTLBFS is required for this option to work. >> >> The data path is realized by directly accessing the vrings and the buffer >> data >> off the guest's memory. >> >> The current user of vhost-user is only vhost-net. We add new netdev >> backend >> that is intended to initialize vhost-net with vhost-user backend. >> >> Example usage: >> >> qemu -m 1024 -mem-path /hugetlbfs,share=on \ >> -chardev socket,id=chr0,path=/path/to/ >> socket \ >> -netdev type=vhost-user,id=net0,chardev=chr0 \ >> -device virtio-net-pci,netdev=net0 >> >> On non-MSIX guests the vhost feture can be forced using a special option: >> >> ... >> -netdev type=vhost-user,id=net0,chardev=chr0,vhostforce >> ... >> >> In order to use ioeventfds, kvm should be enabled. >> >> This code can be pulled from [email protected]:virtualopensystems/qemu.git >> vhost-user-v9 >> A simple functional test is available in tests/vhost-user-test.c >> >> A reference vhost-user slave for testing is also available from >> [email protected]:virtualopensystems/vapp.git >> > > Hi, > > did you see the series I posted today? It would be great if you tested > vhost-user on top of it. I am preparing a new version of this patch series and would like to test (or even rebase?) it on top of your patch series. What is its current status? Is there a tree that I can track and use for rebases? > The replacement for the above -mem-path incantation should be the > following: > > -object memory-file,id=mem,path=/hugetlbfs,share=on \ > -numa node,memdev=mem > > Is this command line update still valid with the latest development? > Do the ideas behind vhost-user work if you have multiple hugetlbfs files, > one per node? > > Paolo > > > Changes from v8: >> - Removed prealloc property from the -mem-path refactoring >> - Added and use new function - kvm_eventfds_enabled >> - Add virtio_queue_get_avail_idx used in vhost_virtqueue_stop to >> get a sane value in case of VHOST_GET_VRING_BASE failure >> - vhost user uses kvm_eventfds_enabled to check whether the ioeventfd >> capability of KVM is available >> - Added flag VHOST_USER_VRING_NOFD_MASK to be set when KICK, CALL or ERR >> file >> descriptor is invalid or ioeventfd is not available >> >> Changes from v7: >> - Slave reconnection when using chardev in server mode >> - qtest vhost-user-test added >> - New qemu_chr_fe_get_msgfds for reading multiple fds from the chardev >> - Mandatory features in vhost_dev, used on reconnect to verify for >> conflicts >> - Add vhostforce parameter to -netdev vhost-user (for non-MSIX guests) >> - Extend libqemustub.a to support qemu-char.c >> >> Changes from v6: >> - Remove the 'unlink' property of '-mem-path' >> - Extend qemu-char: blocking read, send fds, monitor for connection close >> - Vhost-user uses chardev as a backend >> - Poll and reconnect removed (no VHOST_USER_ECHO). >> - Disconnect is deteced by the chardev (G_IO_HUP event) >> - vhost-backend.c split to vhost-user.c >> >> Changes from v5: >> - Split -mem-path unlink option to a separate patch >> - Fds are passed only in the ancillary data >> - Stricter message size checks on receive/send >> - Netdev vhost-user now includes path and poll_time options >> - The connection probing interval is configurable >> >> Changes from v4: >> - Use error_report for errors >> - VhostUserMsg has new field `size` indicating the following payload >> length. >> Field `flags` now has version and reply bits. The structure is packed. >> - Send data is of variable length (`size` field in message) >> - Receive in 2 steps, header and payload >> - Add new message type VHOST_USER_ECHO, to check connection status >> >> Changes from v3: >> - Convert -mem-path to QemuOpts with prealloc, share and unlink >> properties >> - Set 1 sec timeout when read/write to the unix domain socket >> - Fix file descriptor leak >> >> Changes from v2: >> - Reconnect when the backend disappears >> >> Changes from v1: >> - Implementation of vhost-user netdev backend >> - Code improvements >> >> Antonios Motakis (20): >> Convert -mem-path to QemuOpts and add share property >> Add kvm_eventfds_enabled function >> Add chardev API qemu_chr_fe_read_all >> Add chardev API qemu_chr_fe_set_msgfds >> Add chardev API qemu_chr_fe_get_msgfds >> Add G_IO_HUP handler for socket chardev >> vhost_net should call the poll callback only when it is set >> Refactor virtio-net to use generic get_vhost_net >> Add new virtio API virtio_queue_get_avail_idx >> Gracefully handle ioctl failure in vhost_virtqueue_stop >> vhost_net_init will use VhostNetOptions to get all its arguments >> Add vhost_ops to vhost_dev struct and replace all relevant ioctls >> Add mandatory_features to vhost_dev >> Add vhost-backend and VhostBackendType >> Add vhost-user as a vhost backend. >> Add new vhost-user netdev backend >> Add the vhost-user netdev backend to the command line >> Add vhost-user protocol documentation >> libqemustub: add stubs to be able to use qemu-char.c >> Add qtest for vhost-user >> >> docs/specs/vhost-user.txt | 261 ++++++++++++++++++++++++++++ >> exec.c | 21 ++- >> hmp-commands.hx | 4 +- >> hw/net/vhost_net.c | 141 ++++++++++----- >> hw/net/virtio-net.c | 29 +--- >> hw/scsi/vhost-scsi.c | 20 ++- >> hw/virtio/Makefile.objs | 2 +- >> hw/virtio/vhost-backend.c | 71 ++++++++ >> hw/virtio/vhost-user.c | 356 ++++++++++++++++++++++++++++++ >> ++++++++ >> hw/virtio/vhost.c | 58 +++---- >> hw/virtio/virtio.c | 5 + >> include/exec/cpu-all.h | 1 - >> include/hw/virtio/vhost-backend.h | 38 ++++ >> include/hw/virtio/vhost.h | 9 +- >> include/hw/virtio/virtio.h | 1 + >> include/net/vhost-user.h | 17 ++ >> include/net/vhost_net.h | 13 +- >> include/sysemu/char.h | 43 ++++- >> include/sysemu/kvm.h | 11 ++ >> kvm-all.c | 4 + >> kvm-stub.c | 1 + >> net/Makefile.objs | 2 +- >> net/clients.h | 3 + >> net/hub.c | 1 + >> net/net.c | 25 +-- >> net/tap.c | 18 +- >> net/vhost-user.c | 273 +++++++++++++++++++++++++++++ >> qapi-schema.json | 19 +- >> qemu-char.c | 272 +++++++++++++++++++++++++---- >> qemu-options.hx | 25 ++- >> stubs/Makefile.objs | 8 + >> stubs/bdrv-commit-all.c | 7 + >> stubs/chr-msmouse.c | 7 + >> stubs/get-next-serial.c | 3 + >> stubs/is-daemonized.c | 7 + >> stubs/machine-init-done.c | 6 + >> stubs/monitor-init.c | 6 + >> stubs/notify-event.c | 6 + >> stubs/vc-init.c | 7 + >> tests/Makefile | 4 + >> tests/vhost-user-test.c | 309 ++++++++++++++++++++++++++++++ >> +++ >> vl.c | 23 ++- >> 42 files changed, 1979 insertions(+), 158 deletions(-) >> create mode 100644 docs/specs/vhost-user.txt >> create mode 100644 hw/virtio/vhost-backend.c >> create mode 100644 hw/virtio/vhost-user.c >> create mode 100644 include/hw/virtio/vhost-backend.h >> create mode 100644 include/net/vhost-user.h >> create mode 100644 net/vhost-user.c >> create mode 100644 stubs/bdrv-commit-all.c >> create mode 100644 stubs/chr-msmouse.c >> create mode 100644 stubs/get-next-serial.c >> create mode 100644 stubs/is-daemonized.c >> create mode 100644 stubs/machine-init-done.c >> create mode 100644 stubs/monitor-init.c >> create mode 100644 stubs/notify-event.c >> create mode 100644 stubs/vc-init.c >> create mode 100644 tests/vhost-user-test.c >> >> > regards, Nikolay Nikolaev
