After moving to 3.5 kernel I haven't seen it, even at 3x traffic which used to cause it On Mar 25, 2014 9:06 PM, "Dewey McDonnell" <de...@poslinker.com> wrote:
> My web search reveals this problem has been around for years and most > posts conclude that the problem continues. After solving my tap0 TX > packets dropped problem, I find that my VM network has random freezes a > few times each day. Many bloggers on this subject say this VM network > freeze is difficult to reproduce. Not for me! I can cause a network > freeze on my VM in a heartbeat. Like many others, all I need to do is > start a data transfer over the bridge (FTP 2Kb file or larger). When the > freeze happens, ifconfig tap0 says overruns:1. This is QEMU version > 1.6.2 without KVM on CentOS 6.5 version 2.6.32-431. > > -- > You received this bug notification because you are subscribed to the bug > report. > https://bugs.launchpad.net/bugs/997978 > > Title: > KVM images lose connectivity with bridged network > > Status in OpenStack Compute (Nova): > Invalid > Status in "qemu-kvm" package in Ubuntu: > Fix Released > Status in "qemu-kvm" source package in Precise: > Fix Released > > Bug description: > ========================================= > SRU Justification: > 1. Impact: networking breaks after awhile in kvm guests using virtio > networking > 2. Development fix: The bug was fixed upstream and the fix picked up in > a new > merge. > 3. Stable fix: 3 virtio patches are cherrypicked from upstream: > a821ce5 virtio: order index/descriptor reads > 92045d8 virtio: add missing mb() on enable notification > a281ebc virtio: add missing mb() on notification > 4. Test case: Create a bridge enslaving the real NIC, and use that as > the bridge > for a kvm instance with virtio networking. See comment #44 for > specific test > case. > 5. Regression potential: Should be low as several people have tested the > fixed > package under heavy load. > ========================================= > > System: > ----------- > Dell R410 Dual processor 2.4Ghz w/16G RAM > Distributor ID: Ubuntu > Description: Ubuntu 12.04 LTS > Release: 12.04 > Codename: precise > > Setup: > --------- > We're running 3 KVM guests, all Ubuntu 12.04 LTS using bridged > networking. > > From the host: > # cat /etc/network/interfaces > auto br0 > iface br0 inet static > address 212.XX.239.98 > netmask 255.255.255.240 > gateway 212.XX.239.97 > bridge_ports eth0 > bridge_fd 9 > bridge_hello 2 > bridge_maxage 12 > bridge_stp off > > # ifconfig eth0 > eth0 Link encap:Ethernet HWaddr d4:ae:52:84:2d:5a > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > RX packets:11278363 errors:0 dropped:3128 overruns:0 frame:0 > TX packets:14437384 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:1000 > RX bytes:4115980743 (4.1 GB) TX bytes:5451961979 (5.4 GB) > Interrupt:36 Memory:da000000-da012800 > > # ifconfig br0 > br0 Link encap:Ethernet HWaddr d4:ae:52:84:2d:5a > inet addr:212.XX.239.98 Bcast:212.XX.239.111 > Mask:255.255.255.240 > inet6 addr: fe80::d6ae:52ff:fe84:2d5a/64 Scope:Link > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > RX packets:1720861 errors:0 dropped:0 overruns:0 frame:0 > TX packets:1708622 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:0 > RX bytes:210152198 (210.1 MB) TX bytes:300858508 (300.8 MB) > > # brctl show > bridge name bridge id STP enabled interfaces > br0 8000.d4ae52842d5a no eth0 > > I have no default network configured to autostart in libvirt as we're > using bridged networking: > # virsh net-list --all > Name State Autostart > ----------------------------------------- > default inactive no > > # arp > Address HWtype HWaddress Flags Mask > Iface > mailer03.xxxx.com ether 52:54:00:82:5f:0f C > br0 > mailer01.xxxx.com ether 52:54:00:d2:f7:31 C > br0 > mailer02.xxxx.com ether 52:54:00:d3:8f:91 C > br0 > dxi-gw2.xxxx.com ether 00:1a:30:2a:b1:c0 C > br0 > > From one of the guests: > <domain type='kvm' id='4'> > <name>mailer01</name> > <uuid>d41d1355-84e8-ae23-e84e-227bc0231b97</uuid> > <memory>2097152</memory> > <currentMemory>2097152</currentMemory> > <vcpu>1</vcpu> > <os> > <type arch='x86_64' machine='pc-1.0'>hvm</type> > <boot dev='hd'/> > </os> > <features> > <acpi/> > </features> > <clock offset='utc'/> > <on_poweroff>destroy</on_poweroff> > <on_reboot>restart</on_reboot> > <on_crash>destroy</on_crash> > <devices> > <emulator>/usr/bin/kvm</emulator> > <disk type='file' device='disk'> > <driver name='qemu' type='raw'/> > <source file='/dev/mapper/vg_main-mailer01--root'/> > <target dev='hda' bus='ide'/> > <alias name='ide0-0-0'/> > <address type='drive' controller='0' bus='0' unit='0'/> > </disk> > <disk type='file' device='disk'> > <driver name='qemu' type='raw'/> > <source file='/dev/mapper/vg_main-mailer01--swap'/> > <target dev='hdb' bus='ide'/> > <alias name='ide0-0-1'/> > <address type='drive' controller='0' bus='0' unit='1'/> > </disk> > <controller type='ide' index='0'> > <alias name='ide0'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x01' > function='0x1'/> > </controller> > <interface type='bridge'> > <mac address='52:54:00:d2:f7:31'/> > <source bridge='br0'/> > <target dev='vnet0'/> > <model type='virtio'/> > <alias name='net0'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x03' > function='0x0'/> > </interface> > <serial type='pty'> > <source path='/dev/pts/0'/> > <target port='0'/> > <alias name='serial0'/> > </serial> > <console type='pty' tty='/dev/pts/0'> > <source path='/dev/pts/0'/> > <target type='serial' port='0'/> > <alias name='serial0'/> > </console> > <input type='mouse' bus='ps2'/> > <graphics type='vnc' port='5900' autoport='yes' listen='127.0.0.1'> > <listen type='address' address='127.0.0.1'/> > </graphics> > <video> > <model type='cirrus' vram='9216' heads='1'/> > <alias name='video0'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x02' > function='0x0'/> > </video> > <memballoon model='virtio'> > <alias name='balloon0'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x04' > function='0x0'/> > </memballoon> > </devices> > <seclabel type='dynamic' model='apparmor' relabel='yes'> > <label>libvirt-d41d1355-84e8-ae23-e84e-227bc0231b97</label> > <imagelabel>libvirt-d41d1355-84e8-ae23-e84e-227bc0231b97</imagelabel> > </seclabel> > </domain> > > From within the guest: > # cat /etc/network/interfaces > # The primary network interface > auto eth0 > iface eth0 inet static > address 212.XX.239.100 > netmask 255.255.255.240 > network 212.XX.239.96 > broadcast 212.XX.239.111 > gateway 212.XX.239.97 > > # ifconfig > eth0 Link encap:Ethernet HWaddr 52:54:00:d2:f7:31 > inet addr:212.XX.239.100 Bcast:212.XX.239.111 > Mask:255.255.255.240 > inet6 addr: fe80::5054:ff:fed2:f731/64 Scope:Link > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > RX packets:5631830 errors:0 dropped:0 overruns:0 frame:0 > TX packets:6683416 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:1000 > RX bytes:2027322829 (2.0 GB) TX bytes:2076698690 (2.0 GB) > > A commandline which starts the KVM guest: > /usr/bin/kvm -S -M pc-1.0 -enable-kvm -m 2048 -smp > 1,sockets=1,cores=1,threads=1 -name mailer01 -uuid > d41d1355-84e8-ae23-e84e-227bc0231b97 -nodefconfig -nodefaults -chardev > socket,id=charmonitor,path=/var/lib/libvirt/qemu/mailer01.monitor,server,nowait > -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown > -drive > file=/dev/mapper/vg_main-mailer01--root,if=none,id=drive-ide0-0-0,format=raw > -device > ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 > -drive > file=/dev/mapper/vg_main-mailer01--swap,if=none,id=drive-ide0-0-1,format=raw > -device ide-drive,bus=ide.0,unit=1,drive=drive-ide0-0-1,id=ide0-0-1 -netdev > tap,fd=18,id=hostnet0 -device > virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:d2:f7:31,bus=pci.0,addr=0x3 > -chardev pty,id=charserial0 -device > isa-serial,chardev=charserial0,id=serial0 -usb -vnc 127.0.0.1:0 -vga > cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4 > > Problem: > ------------ > Periodically (at least once a day), one or more of the guests lose > network connectivity. Ping responds with 'host unreachable', even from the > dom host. Logging in via the serial console shows no problems: eth0 is up, > can ping the local host, but no outside connectivity. Restart the network > (/etc/init.d/networking restart) does nothing. Reboot the machine and it > comes alive again. > > I've verified there's no arp games going on on the primary host (the > arp tables remain the same before - when it had connectivity - and > after - when it doesn't. > > This is a critical issue affecting production services on the latest > LTS release of Ubuntu. It's similar to an issue which was 'resolved' > in 10.04 but appears to have risen its ugly head again. > > To manage notifications about this bug go to: > https://bugs.launchpad.net/nova/+bug/997978/+subscriptions > -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/997978 Title: KVM images lose connectivity with bridged network To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/997978/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
