Hi,
Seeing failures when trying to do PCI passthrough of Intel XL710 40G
interface to KVM vm.
0a:00.1 Ethernet controller: Intel Corporation Ethernet Controller
XL710 for 40GbE QSFP+ (rev 01)
>From dmesg on host:
[80326.559674] kvm: zapping shadow pages for mmio generation wraparound
[80327.271191] kvm [175994]: vcpu0 unhandled rdmsr: 0x1c9
[80327.271689] kvm [175994]: vcpu0 unhandled rdmsr: 0x1a6
[80327.272201] kvm [175994]: vcpu0 unhandled rdmsr: 0x1a7
[80327.272681] kvm [175994]: vcpu0 unhandled rdmsr: 0x3f6
[80327.376186] kvm [175994]: vcpu0 unhandled rdmsr: 0x606
The pci device is still available in the VM but stat transfer fails.
With the i40e driver, the data transfer fails.
Relevant dmesg output:
[ 11.544088] i40e 0000:00:05.0 eth1: NIC Link is Up 40 Gbps Full Duplex,
Flow Control: None
[ 11.689178] i40e 0000:00:06.0 eth2: NIC Link is Up 40 Gbps Full Duplex,
Flow Control: None
[ 16.704071] ------------[ cut here ]------------
[ 16.705053] WARNING: CPU: 1 PID: 0 at net/sched/sch_generic.c:303
dev_watchdog+0x23e/0x250()
[ 16.705053] NETDEV WATCHDOG: eth1 (i40e): transmit queue 1 timed out
[ 16.705053] Modules linked in: cirrus ttm drm_kms_helper i40e drm ppdev
serio_raw i2c_piix4 virtio_net parport_pc ptp virtio_balloon
crct10dif_pclmul pps_core parport pvpanic crc32_pclmul ghash_clmulni_intel
virtio_blk crc32c_intel virtio_pci virtio_ring virtio ata_generic pata_acpi
[ 16.705053] CPU: 1 PID: 0 Comm: swapper/1 Not tainted
3.18.7-200.fc21.x86_64 #1
[ 16.705053] Hardware name: Fedora Project OpenStack Nova, BIOS
1.7.5-20140709_153950- 04/01/2014
[ 16.705053] 0000000000000000 2e5932b294d0c473 ffff88043fc83d48
ffffffff8175e686
[ 16.705053] 0000000000000000 ffff88043fc83da0 ffff88043fc83d88
ffffffff810991d1
[ 16.705053] ffff88042958f5c0 0000000000000001 ffff88042865f000
0000000000000001
[ 16.705053] Call Trace:
[ 16.705053] <IRQ> [<ffffffff8175e686>] dump_stack+0x46/0x58
[ 16.705053] [<ffffffff810991d1>] warn_slowpath_common+0x81/0xa0
[ 16.705053] [<ffffffff81099245>] warn_slowpath_fmt+0x55/0x70
[ 16.705053] [<ffffffff8166e62e>] dev_watchdog+0x23e/0x250
[ 16.705053] [<ffffffff8166e3f0>] ? dev_graft_qdisc+0x80/0x80
[ 16.705053] [<ffffffff810fd52a>] call_timer_fn+0x3a/0x120
[ 16.705053] [<ffffffff8166e3f0>] ? dev_graft_qdisc+0x80/0x80
[ 16.705053] [<ffffffff810ff692>] run_timer_softirq+0x212/0x2f0
[ 16.705053] [<ffffffff8109d7a4>] __do_softirq+0x124/0x2d0
[ 16.705053] [<ffffffff8109db75>] irq_exit+0x125/0x130
[ 16.705053] [<ffffffff817681d8>] smp_apic_timer_interrupt+0x48/0x60
[ 16.705053] [<ffffffff817662bd>] apic_timer_interrupt+0x6d/0x80
[ 16.705053] <EOI> [<ffffffff811005c8>] ? hrtimer_start+0x18/0x20
[ 16.705053] [<ffffffff8105ca96>] ? native_safe_halt+0x6/0x10
[ 16.705053] [<ffffffff810f81d3>] ? rcu_eqs_enter+0xa3/0xb0
[ 16.705053] [<ffffffff8101ec7f>] default_idle+0x1f/0xc0
[ 16.705053] [<ffffffff8101f64f>] arch_cpu_idle+0xf/0x20
[ 16.705053] [<ffffffff810dad35>] cpu_startup_entry+0x3c5/0x410
[ 16.705053] [<ffffffff8104a2af>] start_secondary+0x1af/0x1f0
[ 16.705053] ---[ end trace 7bda53aeda558267 ]---
[ 16.705053] i40e 0000:00:05.0 eth1: tx_timeout recovery level 1
[ 16.705053] i40e 0000:00:05.0: i40e_vsi_control_tx: VSI seid 519 Tx ring
0 disable timeout
[ 16.744198] i40e 0000:00:05.0: i40e_vsi_control_tx: VSI seid 520 Tx ring
64 disable timeout
[ 16.779322] i40e 0000:00:05.0: i40e_ptp_init: added PHC on eth1
[ 16.791819] i40e 0000:00:05.0: PF 40 attempted to control timestamp mode
on port 1, which is owned by PF 1
[ 16.933869] i40e 0000:00:05.0 eth1: NIC Link is Up 40 Gbps Full Duplex,
Flow Control: None
[ 18.853624] SELinux: initialized (dev tmpfs, type tmpfs), uses
transition SIDs
[ 22.720083] i40e 0000:00:05.0 eth1: tx_timeout recovery level 2
[ 22.826993] i40e 0000:00:05.0: i40e_vsi_control_tx: VSI seid 519 Tx ring
0 disable timeout
[ 22.935288] i40e 0000:00:05.0: i40e_vsi_control_tx: VSI seid 520 Tx ring
64 disable timeout
[ 23.669555] i40e 0000:00:05.0: i40e_ptp_init: added PHC on eth1
[ 23.682067] i40e 0000:00:05.0: PF 40 attempted to control timestamp mode
on port 1, which is owned by PF 1
[ 23.722423] i40e 0000:00:05.0 eth1: NIC Link is Up 40 Gbps Full Duplex,
Flow Control: None
[ 23.800206] i40e 0000:00:06.0: i40e_ptp_init: added PHC on eth2
[ 23.813804] i40e 0000:00:06.0: PF 48 attempted to control timestamp mode
on port 0, which is owned by PF 0
[ 23.855275] i40e 0000:00:06.0 eth2: NIC Link is Up 40 Gbps Full Duplex,
Flow Control: None
[ 38.720091] i40e 0000:00:05.0 eth1: tx_timeout recovery level 3
[ 38.725844] random: nonblocking pool is initialized
[ 38.729874] i40e 0000:00:06.0: HMC error interrupt
[ 38.733425] i40e 0000:00:06.0: i40e_vsi_control_tx: VSI seid 518 Tx ring
0 disable timeout
[ 38.738886] i40e 0000:00:06.0: i40e_vsi_control_tx: VSI seid 521 Tx ring
64 disable timeout
[ 39.689569] i40e 0000:00:06.0: i40e_ptp_init: added PHC on eth2
[ 39.704197] i40e 0000:00:06.0: PF 48 attempted to control timestamp mode
on port 0, which is owned by PF 0
[ 39.746879] i40e 0000:00:06.0 eth2: NIC Link is Down
[ 39.838356] i40e 0000:00:05.0: i40e_ptp_init: added PHC on eth1
[ 39.851788] i40e 0000:00:05.0: PF 40 attempted to control timestamp mode
on port 1, which is owned by PF 1
[ 39.892822] i40e 0000:00:05.0 eth1: NIC Link is Down
[ 43.011610] i40e 0000:00:06.0 eth2: NIC Link is Up 40 Gbps Full Duplex,
Flow Control: None
[ 43.059976] i40e 0000:00:05.0 eth1: NIC Link is Up 40 Gbps Full Duplex,
Flow Control: None
Would appreciate any information on how to debug this issue further and if
the "unhandled rdmsr" logs from KVM indicate some issues with the device
passthrough.
Thanks
Jacob
