On 07/22/2018 04:03 PM, Karel Hendrych wrote:
Hi, I am seeing frequent libvirtd hangs (clients not responding) after last
CentOS6-Xen update :
xen-devel is not the best place to seek help with downstream issues,
particularly libvirt ones :-). You would have better luck contacting the CentOS6
maintainers.
Regards,
Jim
libvirt-libs-4.1.0-2.xen46.el6.x86_64
libvirt-daemon-4.1.0-2.xen46.el6.x86_64
libvirt-daemon-driver-network-4.1.0-2.xen46.el6.x86_64
libvirt-daemon-driver-nwfilter-4.1.0-2.xen46.el6.x86_64
libgcc-4.4.7-18.el6_9.2.x86_64
2:qemu-img-0.12.1.2-2.503.el6_9.5.x86_64
libvirt-daemon-driver-storage-core-4.1.0-2.xen46.el6.x86_64
libvirt-daemon-driver-secret-4.1.0-2.xen46.el6.x86_64
libvirt-daemon-driver-interface-4.1.0-2.xen46.el6.x86_64
libvirt-daemon-driver-nodedev-4.1.0-2.xen46.el6.x86_64
10:centos-release-xen-common-8-4.el6.x86_64
xen-licenses-4.6.6-12.el6.x86_64
xen-libs-4.6.6-12.el6.x86_64
libvirt-daemon-driver-libxl-4.1.0-2.xen46.el6.x86_64
libvirt-daemon-driver-xen-4.1.0-2.xen46.el6.x86_64
libvirt-daemon-driver-qemu-4.1.0-2.xen46.el6.x86_64
libvirt-daemon-driver-storage-gluster-4.1.0-2.xen46.el6.x86_64
libvirt-daemon-driver-storage-logical-4.1.0-2.xen46.el6.x86_64
libvirt-daemon-driver-storage-mpath-4.1.0-2.xen46.el6.x86_64
libvirt-daemon-driver-storage-disk-4.1.0-2.xen46.el6.x86_64
libvirt-daemon-driver-storage-scsi-4.1.0-2.xen46.el6.x86_64
libvirt-daemon-driver-storage-iscsi-4.1.0-2.xen46.el6.x86_64
libvirt-daemon-driver-storage-4.1.0-2.xen46.el6.x86_64
libstdc++-4.4.7-18.el6_9.2.x86_64
libvirt-daemon-config-nwfilter-4.1.0-2.xen46.el6.x86_64
libvirt-daemon-config-network-4.1.0-2.xen46.el6.x86_64
libvirt-daemon-driver-lxc-4.1.0-2.xen46.el6.x86_64
libvirt-client-4.1.0-2.xen46.el6.x86_64
linux-firmware-20171215-82.git2451bb22.el6.noarch
12:dhcp-common-4.1.1-53.P1.el6.centos.4.x86_64
12:dhclient-4.1.1-53.P1.el6.centos.4.x86_64
libvirt-4.1.0-2.xen46.el6.x86_64
10:centos-release-xen-46-8-4.el6.x86_64
10:centos-release-xen-44-8-4.el6.x86_64
tzdata-2018e-3.el6.noarch
libgomp-4.4.7-18.el6_9.2.x86_64
kernel-4.9.86-30.el6.x86_64
xen-hypervisor-4.6.6-12.el6.x86_64
xen-runtime-4.6.6-12.el6.x86_64
xen-4.6.6-12.el6.x86_64
libvirt-daemon-xen-4.1.0-2.xen46.el6.x86_64
Remedy is to kill -9 libvirtd and start again. Issue can be replicated within
few domU starts. Usually libvirtd hangs when domU is bringing up xen drivers or
something around udev, like:
xen_netfront: Initialising Xen virtual ethernet driver
I've been looking into libvirtd strace and debug logs, so far most suspicious in
libvirtd debug log is this:
libvirtd.log:2018-05-22 08:32:44.760+0000: 25455: debug :
udevRemoveOneDevice:1289 : Failed to find device to remove that has udev name
'/sys/devices/vif-24-0/net/vif24.0/queues/tx-7'
libvirtd.log:2018-05-22 08:32:44.761+0000: 25455: debug :
udevRemoveOneDevice:1289 : Failed to find device to remove that has udev name
'/sys/devices/vif-24-0/net/vif24.0/queues/tx-6'
libvirtd.log:2018-05-22 08:32:44.761+0000: 25455: debug :
udevRemoveOneDevice:1289 : Failed to find device to remove that has udev name
'/sys/devices/vif-24-0/net/vif24.0/queues/tx-4'
libvirtd.log:2018-05-22 08:32:44.762+0000: 25455: debug :
udevRemoveOneDevice:1289 : Failed to find device to remove that has udev name
'/sys/devices/vif-24-0/net/vif24.0/queues/tx-5'
libvirtd.log:2018-05-22 08:32:44.763+0000: 25455: debug :
udevRemoveOneDevice:1289 : Failed to find device to remove that has udev name
'/sys/devices/vif-24-0/net/vif24.0/queues/tx-2'
libvirtd.log:2018-05-22 08:32:44.764+0000: 25455: debug :
udevRemoveOneDevice:1289 : Failed to find device to remove that has udev name
'/sys/devices/vif-24-0/net/vif24.0/queues/tx-3'
libvirtd.log:2018-05-22 08:32:44.765+0000: 25455: debug :
udevRemoveOneDevice:1289 : Failed to find device to remove that has udev name
'/sys/devices/vif-24-0/net/vif24.0/queues/rx-6'
libvirtd.log:2018-05-22 08:32:44.766+0000: 25455: debug :
udevRemoveOneDevice:1289 : Failed to find device to remove that has udev name
'/sys/devices/vif-24-0/net/vif24.0/queues/rx-5'
libvirtd.log:2018-05-22 08:32:44.767+0000: 25455: debug :
udevRemoveOneDevice:1289 : Failed to find device to remove that has udev name
'/sys/devices/vif-24-0/net/vif24.0/queues/rx-4'
libvirtd.log:2018-05-22 08:32:44.767+0000: 25455: debug :
udevRemoveOneDevice:1289 : Failed to find device to remove that has udev name
'/sys/devices/vif-24-0/net/vif24.0/queues/rx-7'
libvirtd.log:2018-05-22 08:32:44.768+0000: 25455: debug :
udevRemoveOneDevice:1289 : Failed to find device to remove that has udev name
'/sys/devices/vif-24-0/net/vif24.0/queues/rx-2'
libvirtd.log:2018-05-22 08:32:44.769+0000: 25455: debug :
udevRemoveOneDevice:1289 : Failed to find device to remove that has udev name
'/sys/devices/vif-24-0/net/vif24.0/queues/rx-3'
I could not get rid of this by reducing amount of driver queues (not sure if
that applies to PV)
Is someone out there seeing similar issues? Anyone perhaps interested in
reviewing full debug log / strace ?
Cheers
Karel
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
