This happened because guest used lots of memory on NUMA machine. The KSM might merge similar memory pages of different VMs, sitting on different NUMA memory regions, causing the affected processes to crawl.
Disabled KSM merge_across_nodes: echo 2 > /sys/kernel/mm/ksm/run && sleep 300 && cat /sys/kernel/mm/ksm/pages_shared If there are no pages shared : echo 0 > /sys/kernel/mm/ksm/merge_across_nodes && echo 1 > /sys/kernel/mm/ksm/run make sure to set merge_across_nodes in /etc/sysctl.d to stay across reboots. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/503138 Title: Lucid & Natty, KVM, After kernel message hrtimer: interrupt too slow.... the SMP kvm guest becomes slow. Status in linux package in Ubuntu: Incomplete Status in qemu package in Ubuntu: Incomplete Bug description: KVM Host is running a clean 9.10 server install with just qemu-kvm and virt-manager kernel=2.6.31-17-generic uname -a: Linux VMMASTER 2.6.31-17-generic #54-Ubuntu SMP Thu Dec 10 16:20:31 UTC 2009 i686 GNU/Linux version signature: Ubuntu 2.6.31-17.54-generic KVM Guest is running a clean 9.10 server install with some userland services (apache/postfix/whatnot) kernel=2.6.31-16-generic-pae (from the linux-image-virtual package) uname -a: Linux VM1 2.6.31-16-generic-pae #53-Ubuntu SMP Tue Dec 8 05:20:21 UTC 2009 i686 GNU/Linux version signature: Ubuntu 2.6.31-16.53-generic-pae KVM guest startup command (as invoked by virt-manager): /usr/bin/kvm -S -M pc-0.11 -m 1024 -smp 2 -name lexx -uuid [UUID] -monitor unix:/var/run/libvirt/qemu/VM1.monitor,server,nowait -boot c -drive file=,if=ide,media=cdrom,index=2 -drive file=/var/lib/libvirt/images/VM1.img,if=virtio,index=0,boot=on -net nic,macaddr=[MAC],vlan=0,model=virtio,name=virtio.0 -net tap,fd=16,vlan=0,name=tap.0 -serial none -parallel none -usb -vnc 127.0.0.1:0 -k en-us -vga cirrus Problem description: After a while (and high network IO) I see this pop up in my guest dmesg: hrtimer: interrupt too slow, forcing clock min delta to 215997540 ns after that the guest becomes very slow to respond, sometimes taking seconds to echo my ssh input back, on a local lan. Only a reboot of the kvm guest fixes this, until the dreaded hrtimer message pops up After a lot of googling and trying a lot of things I found this discussion on the patchwork kernel mailinglist, which contains a possible solution: http://patchwork.kernel.org/patch/51561/ Please look into it, perhaps this solves a lot of kvm-users' problems I'd like to patch my kvm guests' kernel myself to test this hrtimer patch, do you have the correct procedure for me so i can create a custom kernel? To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/503138/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
