In migration all data is copied to a static buffer in QEMUFile, this hurts our network bandwidth and CPU usage especially with large guests. We switched to iovec for storing different buffers to send (even a byte field is considered as a buffer) and use writev to send the iovec. writev was chosen (as apposed to sendmsg) because it supprts non socket fds. Guest memory pages are not copied by calling a new function qemu_put_buffer_no_copy. The page header data and device state data are still copied into the static buffer. This data consists of a lot of bytes and integer fields and the static buffer is used to store it during batching. Another improvement is changing qemu_putbe64/32/16 to create a single buffer instead of several byte sized buffer.
git repository: git://github.com/oritwas/qemu.git sendv_v2 Changes from v1: Use iov_send for socket. Make writev_buffer optional and if it is not implemented use put_buffer Future work: Make number of iovec changeable Orit Wasserman (8): Add QemuFileWritevBuffer QemuFileOps Add socket_writev_buffer function Update bytes_xfer in qemu_put_byte Store the data to send also in iovec Use writev ops if available More optimized qemu_put_be64/32/16 Add qemu_put_buffer_no_copy Use qemu_put_buffer_no_copy for guest memory pages arch_init.c | 2 +- include/migration/qemu-file.h | 12 +++++ savevm.c | 113 +++++++++++++++++++++++++++++------------- 3 files changed, 92 insertions(+), 35 deletions(-) -- 1.7.11.7
