On 08/24/2015 12:19 PM, Santosh R wrote: > Hi, > > Earlier I was seeing lower throughput in VMs using VxLan as GRO was > not happening in VM. > Tom Herbert suggested to use "vxlan: GRO support at tunnel layer" patch > series. > With today's net-next (4.2.0-rc7) in host and VM, I could see GRO > happening for vxlan, macvtap and virtual interface in VM. > The throughput is still low between VMs (around 4Gbps compared to > 9Gbps without VxLAN). > Looks like the packet is getting segmented in Host and then GROed in VM. > Is this an expected behaviour?
Currently yes. I am working on adding GSO_TUNNEL and related checksum support to virtio to eliminate this segmentation. -vlad > Is my below configuration correct? > > Here is the configuration. > eth (VM) - macvtap - vxlan - phy iface <-> phy iface - vxlan - > macvtap - (VM) eth > > VM is started with > # qemu-system-x86_64 -m 4096 -smp 4 -boot c -device > virtio-net-pci,netdev=hostnet0,id=net0,mac=C2:B2:CA:6F:BC:A4 -device > e1000,netdev=tap0,mac=DE:AD:BE:EF:96:32 -netdev tap,id=hostnet0,fd=3 > 3<>/dev/tap18 -netdev tap,id=tap0,script=no -drive > file=/root/vdisk_rhel65.img > > Here is the skb_segment count for 10 sec iperf receive test. > host # ./funccount skb_segment > Tracing "skb_segment"... Ctrl-C to end. > ^C > FUNC COUNT > skb_segment 58604 > > # ./functrace skb_segment > ... > <idle>-0 [006] ..s. 17632.030126: skb_segment <-tcp_gso_segment > ksoftirqd/6-38 [006] ..s. 17632.030177: skb_segment <-tcp_gso_segment > ksoftirqd/6-38 [006] ..s. 17632.030223: skb_segment <-tcp_gso_segment > ksoftirqd/6-38 [006] ..s. 17632.030269: skb_segment <-tcp_gso_segment > ksoftirqd/6-38 [006] ..s. 17632.030298: skb_segment <-tcp_gso_segment > qemu-system-x86-5932 [006] ..s. 17632.030489: skb_segment <-tcp_gso_segment > qemu-system-x86-5932 [006] ..s. 17632.030507: skb_segment <-tcp_gso_segment > qemu-system-x86-5932 [006] ..s. 17632.030528: skb_segment <-tcp_gso_segment > qemu-system-x86-5932 [006] ..s. 17632.030550: skb_segment <-tcp_gso_segment > qemu-system-x86-5932 [006] ..s. 17632.030576: skb_segment <-tcp_gso_segment > qemu-system-x86-5932 [006] ..s1 17632.030759: skb_segment <-tcp_gso_segment > qemu-system-x86-5932 [006] ..s1 17632.030814: skb_segment <-tcp_gso_segment > .. > > # Physical interface > 21:32:49.749263 IP 102.22.22.14.39561 > 102.22.22.12.otv: UDP, length 2870 > 21:32:49.749278 IP 102.22.22.14.39561 > 102.22.22.12.otv: UDP, length 9860 > 21:32:49.749326 IP 102.22.22.12.44214 > 102.22.22.14.otv: UDP, length 74 > 21:32:49.749333 IP 102.22.22.12.44214 > 102.22.22.14.otv: UDP, length 74 > 21:32:49.749340 IP 102.22.22.12.44214 > 102.22.22.14.otv: UDP, length 74 > 21:32:49.749405 IP 102.22.22.14.39561 > 102.22.22.12.otv: UDP, length 2870 > 21:32:49.749425 IP 102.22.22.14.39561 > 102.22.22.12.otv: UDP, length 11258 > > # VxLAN > 21:32:49.749268 IP 102.44.44.14.60616 > 102.44.44.12.commplex-link: > Flags [.], seq 25:2821, ack 1, win 111, options [nop,nop,TS val > 15632994 ecr 13334931], length 2796 > 21:32:49.749281 IP 102.44.44.14.60616 > 102.44.44.12.commplex-link: > Flags [.], seq 2821:12607, ack 1, win 111, options [nop,nop,TS val > 15632994 ecr 13334931], length 9786 > 21:32:49.749322 IP 102.44.44.12.commplex-link > 102.44.44.14.60616: > Flags [.], ack 2821, win 270, options [nop,nop,TS val 13334931 ecr > 15632994], length 0 > 21:32:49.749331 IP 102.44.44.12.commplex-link > 102.44.44.14.60616: > Flags [.], ack 7015, win 336, options [nop,nop,TS val 13334931 ecr > 15632994], length 0 > 21:32:49.749336 IP 102.44.44.12.commplex-link > 102.44.44.14.60616: > Flags [.], ack 12607, win 423, options [nop,nop,TS val 13334931 ecr > 15632994], length 0 > 21:32:49.749411 IP 102.44.44.14.60616 > 102.44.44.12.commplex-link: > Flags [.], seq 12607:15403, ack 1, win 111, options [nop,nop,TS val > 15632994 ecr 13334931], length 2796 > 21:32:49.749429 IP 102.44.44.14.60616 > 102.44.44.12.commplex-link: > Flags [P.], seq 15403:26587, ack 1, win 111, options [nop,nop,TS val > 15632994 ecr 13334931], length 11184 > > # macvtap > 2.44.44.14.60616 > 102.44.44.12.commplex-link: Flags [.], seq 25:2821, > ack 1, win 111, options [nop,nop,TS val 15632994 ecr 13334931], length > 2796 > 21:32:49.749281 IP 102.44.44.14.60616 > 102.44.44.12.commplex-link: > Flags [.], seq 2821:12607, ack 1, win 111, options [nop,nop,TS val > 15632994 ecr 13334931], length 9786 > 21:32:49.749321 IP 102.44.44.12.commplex-link > 102.44.44.14.60616: > Flags [.], ack 2821, win 270, options [nop,nop,TS val 13334931 ecr > 15632994], length 0 > 21:32:49.749330 IP 102.44.44.12.commplex-link > 102.44.44.14.60616: > Flags [.], ack 7015, win 336, options [nop,nop,TS val 13334931 ecr > 15632994], length 0 > 21:32:49.749335 IP 102.44.44.12.commplex-link > 102.44.44.14.60616: > Flags [.], ack 12607, win 423, options [nop,nop,TS val 13334931 ecr > 15632994], length 0 > 21:32:49.749411 IP 102.44.44.14.60616 > 102.44.44.12.commplex-link: > Flags [.], seq 12607:15403, ack 1, win 111, options [nop,nop,TS val > 15632994 ecr 13334931], length 2796 > 21:32:49.749429 IP 102.44.44.14.60616 > 102.44.44.12.commplex-link: > Flags [P.], seq 15403:26587, ack 1, win 111, options [nop,nop,TS val > 15632994 ecr 13334931], length 11184 > > # VM interface > 2:02:48.126327 IP 102.44.44.14.60616 > 102.44.44.12.commplex-link: > Flags [.], seq 25:2821, ack 1, win 111, options [nop,nop,TS val > 15632994 ecr 13334931], length 2796 > 12:02:48.126332 IP 102.44.44.12.commplex-link > 102.44.44.14.60616: > Flags [.], ack 2821, win 270, options [nop,nop,TS val 13334931 ecr > 15632994], length 0 > 12:02:48.126352 IP 102.44.44.14.60616 > 102.44.44.12.commplex-link: > Flags [.], seq 2821:7015, ack 1, win 111, options [nop,nop,TS val > 15632994 ecr 13334931], length 4194 > 12:02:48.126357 IP 102.44.44.12.commplex-link > 102.44.44.14.60616: > Flags [.], ack 7015, win 336, options [nop,nop,TS val 13334931 ecr > 15632994], length 0 > 12:02:48.126361 IP 102.44.44.14.60616 > 102.44.44.12.commplex-link: > Flags [.], seq 7015:12607, ack 1, win 111, options [nop,nop,TS val > 15632994 ecr 13334931], length 5592 > 12:02:48.126366 IP 102.44.44.12.commplex-link > 102.44.44.14.60616: > Flags [.], ack 12607, win 423, options [nop,nop,TS val 13334931 ecr > 15632994], length 0 > 12:02:48.126468 IP 102.44.44.14.60616 > 102.44.44.12.commplex-link: > Flags [.], seq 12607:15403, ack 1, win 111, options [nop,nop,TS val > 15632994 ecr 13334931], length 2796 > > Regards > -Santosh > -- > To unsubscribe from this list: send the line "unsubscribe netdev" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
