** Description changed:
Virtual Machines with SR-IOV VFs from an Intel E810-XXV [8086:159b] get
no traffic flow and produce error messages in both the host and guest
during network configuration.
Environment: Ubuntu OpenStack Focal-Ussuri with OVN
Host Kernel: v5.15.0-41-generic 20.04 Focal-HWE
Guest Kernels: v5.4.x Focal, v5.15.0-41-generic Jammy
Host Error Messages:
ice 0000:98:00.1: VF 7 failed opcode 6, retval: -5
Guest Error Messages:
iavf 0000:00:05.0: PF returned error -5 (IAVF_ERR_PARAM) to our request 6
In the context of these errors "6" refers to the value of
VIRTCHNL_OP_CONFIG_VSI_QUEUES
It was found in these cases that the VM is able to successfully transmit
packets but never receives any and the RX packet drop counters for the
VF in "ip link" on the host increase equal to the RX packet count.
There is a prior commit e6ba5273d4ede03d075d7a116b8edad1f6115f4d
claiming to resolve this error in some cases. It is already included in
the test kernel v5.15.0-41 and did not resolve the issue.
The following conditions are required to trigger the bug:
- A port VLAN must be assigned by the host
- The MTU must be set >1500 by the guest
It appears that this ice/iavf driver is quite immature as many
significant SR-IOV related fixes have landed in each of the recent
kernel releases and we may need to consider pro-actively backporting
more fixes.
[ Impact ]
Intel E810 (host: ice) SR-IOV VFs (guest: iavf) fail when a port VLAN is
- assigned by the host and the guest changes the MTU >1500. The guest iavf
driver
- logs the following error and traffic flow stops:
+ assigned by the host and the guest changes the MTU >1500. The guest iavf
+ driver logs the following error and traffic flow stops:
iavf 0000:00:05.0: PF returned error -5 (IAVF_ERR_PARAM) to our request 6
- There is no workaround, Intel E810 SR-IOV VFs with MTU >1500 cannot be used
- without this patch.
-
+ There is no workaround, Intel E810 SR-IOV VFs with MTU >1500 cannot be
+ used without this patch.
[ Fix ]
During queue configuration, VF's max packet size was set to
IAVF_MAX_RXBUFFER but on ice max frame size was smaller by VLAN_HLEN
due to making some space for port VLAN as VF is not aware whether it's
in a port VLAN. This mismatch in sizes caused ice to reject queue
configuration with ERR_PARAM error. Proper max_mtu is sent from ice PF
to VF with GET_VF_RESOURCES msg but VF does not look at this.
In iavf change max_frame from IAVF_MAX_RXBUFFER to max_mtu
received from pf with GET_VF_RESOURCES msg to make vf's
max_frame_size dependent from pf. Add check if received max_mtu is
not in eligible range then set it to IAVF_MAX_RXBUFFER.
-
[ Test Plan ]
1. Create a focal VM and assign an Intel E810 (ice) SR-IOV VF with a port
vlan
Openstack works, as does creating a VM directly with uvtool/libvirt.
The syntax inside <interface> for the port VLAN is <vlan><tag
id=‘998’/></vlan>
2. Set an IP on the interface, set it’s link up and verify operation
sudo ip a a 192.168.1.7/24 dev enp7s0
sudo ip link set up dev enp7s0
ping 192.168.1.1
3. Set the MTU to 9000
sudo ip link set mtu 9000 dev enp7s0
4. Test if the interface still works and whether the error was observed:
iavf 0000:07:00.0: PF returned error -5 (IAVF_ERR_PARAM) to our request 6
-
[ Where problems could occur ]
The iavf driver is unified for multiple host drivers including ice and
i40e.
- A companion patch for i40e was also submitted as it did not set the max_mtu
- although the ice driver already did. We should test the fixed iavf driver
- against both i40e (Intel E7xx) and ice (Intel E8xx) cards with and without
that
- patch.
+ A companion patch for i40e was also submitted as it did not set the
+ max_mtu although the ice driver already did. We should test the fixed iavf
+ driver against both i40e (Intel E7xx) and ice (Intel E8xx) cards with and
+ without that patch on the host kernel.
https://lore.kernel.org/netdev/20220919223428.572091-4-anthony.l.ngu...@intel.com/
-
[ Other Info ]
Patch was developed by intel, has been accepted into v6.0-rc7 and already
released into upstream stable v5.4.215, v5.15.71 and v5.19.12.
- Early SRU is requested as the functionality is required now and there is no
- workaround.
+ Early SRU is requested as the functionality is required now and there is
+ no workaround.
** Changed in: linux (Ubuntu)
Status: Incomplete => Confirmed
** Changed in: linux (Ubuntu)
Importance: Undecided => High
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1983656
Title:
SR-IOV VFs no traffic flow and error on Intel E810 (ice / iavf)
Status in linux package in Ubuntu:
Confirmed
Bug description:
Virtual Machines with SR-IOV VFs from an Intel E810-XXV [8086:159b]
get no traffic flow and produce error messages in both the host and
guest during network configuration.
Environment: Ubuntu OpenStack Focal-Ussuri with OVN
Host Kernel: v5.15.0-41-generic 20.04 Focal-HWE
Guest Kernels: v5.4.x Focal, v5.15.0-41-generic Jammy
Host Error Messages:
ice 0000:98:00.1: VF 7 failed opcode 6, retval: -5
Guest Error Messages:
iavf 0000:00:05.0: PF returned error -5 (IAVF_ERR_PARAM) to our request 6
In the context of these errors "6" refers to the value of
VIRTCHNL_OP_CONFIG_VSI_QUEUES
It was found in these cases that the VM is able to successfully
transmit packets but never receives any and the RX packet drop
counters for the VF in "ip link" on the host increase equal to the RX
packet count.
There is a prior commit e6ba5273d4ede03d075d7a116b8edad1f6115f4d
claiming to resolve this error in some cases. It is already included
in the test kernel v5.15.0-41 and did not resolve the issue.
The following conditions are required to trigger the bug:
- A port VLAN must be assigned by the host
- The MTU must be set >1500 by the guest
It appears that this ice/iavf driver is quite immature as many
significant SR-IOV related fixes have landed in each of the recent
kernel releases and we may need to consider pro-actively backporting
more fixes.
[ Impact ]
Intel E810 (host: ice) SR-IOV VFs (guest: iavf) fail when a port VLAN is
assigned by the host and the guest changes the MTU >1500. The guest iavf
driver logs the following error and traffic flow stops:
iavf 0000:00:05.0: PF returned error -5 (IAVF_ERR_PARAM) to our request 6
There is no workaround, Intel E810 SR-IOV VFs with MTU >1500 cannot be
used without this patch.
[ Fix ]
During queue configuration, VF's max packet size was set to
IAVF_MAX_RXBUFFER but on ice max frame size was smaller by VLAN_HLEN
due to making some space for port VLAN as VF is not aware whether it's
in a port VLAN. This mismatch in sizes caused ice to reject queue
configuration with ERR_PARAM error. Proper max_mtu is sent from ice PF
to VF with GET_VF_RESOURCES msg but VF does not look at this.
In iavf change max_frame from IAVF_MAX_RXBUFFER to max_mtu
received from pf with GET_VF_RESOURCES msg to make vf's
max_frame_size dependent from pf. Add check if received max_mtu is
not in eligible range then set it to IAVF_MAX_RXBUFFER.
[ Test Plan ]
1. Create a focal VM and assign an Intel E810 (ice) SR-IOV VF with a port
vlan
Openstack works, as does creating a VM directly with uvtool/libvirt.
The syntax inside <interface> for the port VLAN is <vlan><tag
id=‘998’/></vlan>
2. Set an IP on the interface, set it’s link up and verify operation
sudo ip a a 192.168.1.7/24 dev enp7s0
sudo ip link set up dev enp7s0
ping 192.168.1.1
3. Set the MTU to 9000
sudo ip link set mtu 9000 dev enp7s0
4. Test if the interface still works and whether the error was observed:
iavf 0000:07:00.0: PF returned error -5 (IAVF_ERR_PARAM) to our request 6
[ Where problems could occur ]
The iavf driver is unified for multiple host drivers including ice and
i40e.
A companion patch for i40e was also submitted as it did not set the
max_mtu although the ice driver already did. We should test the fixed iavf
driver against both i40e (Intel E7xx) and ice (Intel E8xx) cards with and
without that patch on the host kernel.
https://lore.kernel.org/netdev/20220919223428.572091-4-anthony.l.ngu...@intel.com/
[ Other Info ]
Patch was developed by intel, has been accepted into v6.0-rc7 and already
released into upstream stable v5.4.215, v5.15.71 and v5.19.12.
Early SRU is requested as the functionality is required now and there is
no workaround.
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1983656/+subscriptions
--
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help : https://help.launchpad.net/ListHelp
