In addition to the verification provided in comment #26 and #28 we've
taken the neutron-api-plugin-ovn e2e charm gate with an added hairpin
test for a spin for both Focal and Jammy where we found the control to
fail as expected and successful test with the updated kernel.
$ juju run --application nova-compute 'uname -a'
- Stdout: |
Linux civil-ibex 5.4.0-123-generic #139-Ubuntu SMP Mon Jul 11 16:02:31 UTC
2022 x86_64 x86_64 x86_64 GNU/Linux
UnitId: nova-compute/0
- Stdout: |
Linux pumped-pika 5.4.0-123-generic #139-Ubuntu SMP Mon Jul 11 16:02:31 UTC
2022 x86_64 x86_64 x86_64 GNU/Linux
UnitId: nova-compute/1
- Stdout: |
Linux moved-toucan 5.4.0-123-generic #139-Ubuntu SMP Mon Jul 11 16:02:31
UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
UnitId: nova-compute/2
...
2022-07-19 16:25:09 [INFO] XXX HAIRPIN
2022-07-19 16:25:09 [INFO] Attempting to ssh to instance-1(10.78.95.25)
2022-07-19 16:25:09 [INFO] Connected (version 2.0, client OpenSSH_7.6p1)
2022-07-19 16:25:09 [INFO] Authentication (publickey) successful!
2022-07-19 16:25:09 [INFO] Running ping -M do -s 1414 -c 1 10.78.95.25 on
instance-1
2022-07-19 16:25:10 [INFO] Attempting to ssh to instance-1(10.78.95.25)
2022-07-19 16:25:10 [INFO] Connected (version 2.0, client OpenSSH_7.6p1)
2022-07-19 16:25:10 [INFO] Authentication (publickey) successful!
2022-07-19 16:25:10 [INFO] Running ping -M do -s 1414 -c 1 10.78.95.25 on
instance-1
2022-07-19 16:25:10 [INFO] Attempting to ssh to instance-1(10.78.95.25)
2022-07-19 16:25:10 [INFO] Connected (version 2.0, client OpenSSH_7.6p1)
2022-07-19 16:25:10 [INFO] Authentication (publickey) successful!
2022-07-19 16:25:10 [INFO] Running ping -M do -s 1414 -c 1 10.78.95.25 on
instance-1
2022-07-19 16:25:11 [INFO] Attempting to ssh to instance(10.78.95.71)
2022-07-19 16:25:11 [INFO] Connected (version 2.0, client OpenSSH_7.6p1)
2022-07-19 16:25:11 [INFO] Authentication (publickey) successful!
2022-07-19 16:25:11 [INFO] Running ping -c 1 192.168.0.1 on instance
2022-07-19 16:25:11 [INFO] Attempting to ssh to instance(10.78.95.71)
2022-07-19 16:25:11 [INFO] Connected (version 2.0, client OpenSSH_7.6p1)
2022-07-19 16:25:11 [INFO] Authentication (publickey) successful!
2022-07-19 16:25:11 [INFO] Running ping -M do -s 1414 -c 1 192.168.0.1 on
instance
2022-07-19 16:25:12 [INFO] Attempting to ssh to instance(10.78.95.25)
2022-07-19 16:25:12 [INFO] Connected (version 2.0, client OpenSSH_7.6p1)
2022-07-19 16:25:12 [INFO] Authentication (publickey) successful!
2022-07-19 16:25:12 [INFO] Running ping -c 1 192.168.0.1 on instance
2022-07-19 16:25:12 [INFO] Attempting to ssh to instance(10.78.95.25)
2022-07-19 16:25:12 [INFO] Connected (version 2.0, client OpenSSH_7.6p1)
2022-07-19 16:25:12 [INFO] Authentication (publickey) successful!
2022-07-19 16:25:12 [INFO] Running ping -M do -s 1414 -c 1 192.168.0.1 on
instance
2022-07-19 16:25:13 [INFO] Running resource cleanup
2022-07-19 16:25:13 [INFO] Removing resources created by test
(zaza-neutrontests*)
2022-07-19 16:25:21 [INFO] ok
2022-07-19 16:25:21 [INFO]
----------------------------------------------------------------------
2022-07-19 16:25:21 [INFO] Ran 1 test in 48.934s
2022-07-19 16:25:21 [INFO] OK
2022-07-19 16:25:21 [INFO] Events:
Test zaza.openstack.charm_tests.neutron.tests.NeutronNetworkingTest:
Start: 1658240672.1115046
Finish: 1658240721.410828
Elapsed Time: 49.29932355880737
PCT Of Run Time: 100
Metadata: {}
$ juju run --application nova-compute 'uname -a'
- Stdout: |
Linux casual-buck 5.15.0-43-generic #46-Ubuntu SMP Tue Jul 12 10:30:17 UTC
2022 x86_64 x86_64 x86_64 GNU/Linux
UnitId: nova-compute/0
- Stdout: |
Linux neat-serval 5.15.0-43-generic #46-Ubuntu SMP Tue Jul 12 10:30:17 UTC
2022 x86_64 x86_64 x86_64 GNU/Linux
UnitId: nova-compute/1
- Stdout: |
Linux brave-horse 5.15.0-43-generic #46-Ubuntu SMP Tue Jul 12 10:30:17 UTC
2022 x86_64 x86_64 x86_64 GNU/Linux
UnitId: nova-compute/2
...
2022-07-19 16:25:09 [INFO] XXX HAIRPIN
2022-07-19 16:25:09 [INFO] Attempting to ssh to instance-1(10.78.95.92)
2022-07-19 16:25:09 [INFO] Connected (version 2.0, client OpenSSH_7.6p1)
2022-07-19 16:25:09 [INFO] Authentication (publickey) successful!
2022-07-19 16:25:09 [INFO] Running ping -M do -s 1414 -c 1 10.78.95.92 on
instance-1
2022-07-19 16:25:09 [INFO] Attempting to ssh to instance-1(10.78.95.92)
2022-07-19 16:25:09 [INFO] Connected (version 2.0, client OpenSSH_7.6p1)
2022-07-19 16:25:09 [INFO] Authentication (publickey) successful!
2022-07-19 16:25:09 [INFO] Running ping -M do -s 1414 -c 1 10.78.95.92 on
instance-1
2022-07-19 16:25:09 [INFO] Attempting to ssh to instance-1(10.78.95.92)
2022-07-19 16:25:10 [INFO] Connected (version 2.0, client OpenSSH_7.6p1)
2022-07-19 16:25:10 [INFO] Authentication (publickey) successful!
2022-07-19 16:25:10 [INFO] Running ping -M do -s 1414 -c 1 10.78.95.92 on
instance-1
2022-07-19 16:25:10 [INFO] Attempting to ssh to instance(10.78.95.29)
2022-07-19 16:25:10 [INFO] Connected (version 2.0, client OpenSSH_7.6p1)
2022-07-19 16:25:10 [INFO] Authentication (publickey) successful!
2022-07-19 16:25:10 [INFO] Running ping -c 1 192.168.0.1 on instance
2022-07-19 16:25:11 [INFO] Attempting to ssh to instance(10.78.95.29)
2022-07-19 16:25:11 [INFO] Connected (version 2.0, client OpenSSH_7.6p1)
2022-07-19 16:25:11 [INFO] Authentication (publickey) successful!
2022-07-19 16:25:11 [INFO] Running ping -M do -s 1414 -c 1 192.168.0.1 on
instance
2022-07-19 16:25:12 [INFO] Attempting to ssh to instance(10.78.95.92)
2022-07-19 16:25:12 [INFO] Connected (version 2.0, client OpenSSH_7.6p1)
2022-07-19 16:25:12 [INFO] Authentication (publickey) successful!
2022-07-19 16:25:12 [INFO] Running ping -c 1 192.168.0.1 on instance
2022-07-19 16:25:12 [INFO] Attempting to ssh to instance(10.78.95.92)
2022-07-19 16:25:12 [INFO] Connected (version 2.0, client OpenSSH_7.6p1)
2022-07-19 16:25:12 [INFO] Authentication (publickey) successful!
2022-07-19 16:25:12 [INFO] Running ping -M do -s 1414 -c 1 192.168.0.1 on
instance
2022-07-19 16:25:13 [INFO] Running resource cleanup
2022-07-19 16:25:13 [INFO] Removing resources created by test
(zaza-neutrontests*)
2022-07-19 16:25:21 [INFO] ok
2022-07-19 16:25:21 [INFO]
----------------------------------------------------------------------
2022-07-19 16:25:21 [INFO] Ran 1 test in 52.369s
2022-07-19 16:25:21 [INFO] OK
2022-07-19 16:25:21 [INFO] Events:
Test zaza.openstack.charm_tests.neutron.tests.NeutronNetworkingTest:
Start: 1658240668.6095116
Finish: 1658240721.3067656
Elapsed Time: 52.697253942489624
PCT Of Run Time: 100
Metadata: {}
** Tags removed: verification-needed-focal verification-needed-jammy
** Tags added: verification-done-focal verification-done-jammy
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1967856
Title:
Hairpin traffic does not work with centralized NAT gw
Status in linux package in Ubuntu:
Invalid
Status in openvswitch package in Ubuntu:
Invalid
Status in ovn package in Ubuntu:
Invalid
Status in linux source package in Focal:
Fix Committed
Status in openvswitch source package in Focal:
Invalid
Status in ovn source package in Focal:
Invalid
Status in linux source package in Impish:
Won't Fix
Status in openvswitch source package in Impish:
New
Status in ovn source package in Impish:
New
Status in linux source package in Jammy:
Fix Committed
Status in openvswitch source package in Jammy:
Invalid
Status in ovn source package in Jammy:
Invalid
Status in linux source package in Kinetic:
Invalid
Status in openvswitch source package in Kinetic:
Invalid
Status in ovn source package in Kinetic:
Invalid
Bug description:
[Impact]
Users of Open vSwitch on Focal will not be able to upgrade to v2.16.0 or
newer until this long standing kernel bug has been fixed.
Users of Open vSwitch on Jammy will be affected by this bug and
have no user space fix available. This bug currently blocks the
OpenStack Engineering team's charm product gate.
[Test Plan]
Execute the OVN system testsuite utilizing the kernel data path with the test
synthesis patch in comment #7 applied.
In addition to that validating that the OpenStack charm test gate is
unblocked would be valuable.
[Regression Potential]
The regression potential can be considered as low because:
- The calls added in the openvswitch kernel datapath code would
prior to Open vSwitch 2.16.0 have been initiated from the
userspace code and by chance concealed this bug.
- After an optimization done in 2.16.0 the kernel bug was
revealed and these calls now must be made from the kernel
datapath to retain functionality in use in the wild.
[Original Bug Description]
If you have two hvs where hv1 is the gateway chassis and you have an instance
running on hv2.
On instance on hv2 hairpin traffic works for the first session, but
not for the next:
$ ping -c1 10.78.95.89
PING 10.78.95.89 (10.78.95.89) 56(84) bytes of data.
64 bytes from 10.78.95.89: icmp_seq=1 ttl=62 time=1.07 ms
--- 10.78.95.89 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 1.078/1.078/1.078/0.000 ms
$ sudo ovs-appctl -t ovs-vswitchd dpctl/dump-conntrack
icmp,orig=(src=10.78.95.89,dst=192.168.0.211,id=7334,type=8,code=0),reply=(src=192.168.0.211,dst=10.78.95.89,id=7334,type=0,code=0),zone=7
icmp,orig=(src=192.168.0.211,dst=10.78.95.89,id=7334,type=8,code=0),reply=(src=10.78.95.89,dst=192.168.0.211,id=7334,type=0,code=0),zone=7
$ ping -c1 10.78.95.89
PING 10.78.95.89 (10.78.95.89) 56(84) bytes of data.
--- 10.78.95.89 ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms
$ sudo ovs-appctl -t ovs-vswitchd dpctl/dump-conntrack
icmp,orig=(src=10.78.95.89,dst=192.168.0.211,id=7334,type=8,code=0),reply=(src=192.168.0.211,dst=10.78.95.89,id=7334,type=0,code=0),zone=7
icmp,orig=(src=192.168.0.211,dst=10.78.95.89,id=7334,type=8,code=0),reply=(src=10.78.95.89,dst=192.168.0.211,id=7334,type=0,code=0),zone=7
icmp,orig=(src=192.168.0.211,dst=10.78.95.89,id=7335,type=8,code=0),reply=(src=10.78.95.89,dst=192.168.0.211,id=7335,type=0,code=0),zone=7
We made an attempt at using OVN built with [0], but that did
unfortunately not help.
If we however revert [1] it works again:
$ ping -c1 10.78.95.89
PING 10.78.95.89 (10.78.95.89) 56(84) bytes of data.
64 bytes from 10.78.95.89: icmp_seq=1 ttl=62 time=1.31 ms
--- 10.78.95.89 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 1.318/1.318/1.318/0.000 ms
$ sudo ovs-appctl -t ovs-vswitchd dpctl/dump-conntrack
icmp,orig=(src=192.168.0.211,dst=10.78.95.89,id=7336,type=8,code=0),reply=(src=10.78.95.89,dst=192.168.0.211,id=7336,type=0,code=0),zone=7
icmp,orig=(src=10.78.95.89,dst=192.168.0.211,id=7336,type=8,code=0),reply=(src=192.168.0.211,dst=10.78.95.89,id=7336,type=0,code=0),zone=7
icmp,orig=(src=10.78.95.89,dst=192.168.0.211,id=7336,type=8,code=0),reply=(src=192.168.0.211,dst=10.78.95.89,id=7336,type=0,code=0),zone=1
$ ping -c1 10.78.95.89
PING 10.78.95.89 (10.78.95.89) 56(84) bytes of data.
64 bytes from 10.78.95.89: icmp_seq=1 ttl=62 time=0.307 ms
--- 10.78.95.89 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.307/0.307/0.307/0.000 ms
$ sudo ovs-appctl -t ovs-vswitchd dpctl/dump-conntrack
icmp,orig=(src=10.78.95.89,dst=192.168.0.211,id=7337,type=8,code=0),reply=(src=192.168.0.211,dst=10.78.95.89,id=7337,type=0,code=0),zone=7
icmp,orig=(src=10.78.95.89,dst=192.168.0.211,id=7337,type=8,code=0),reply=(src=192.168.0.211,dst=10.78.95.89,id=7337,type=0,code=0),zone=1
icmp,orig=(src=192.168.0.211,dst=10.78.95.89,id=7337,type=8,code=0),reply=(src=10.78.95.89,dst=192.168.0.211,id=7337,type=0,code=0),zone=7
icmp,orig=(src=192.168.0.211,dst=10.78.95.89,id=7336,type=8,code=0),reply=(src=10.78.95.89,dst=192.168.0.211,id=7336,type=0,code=0),zone=7
icmp,orig=(src=10.78.95.89,dst=192.168.0.211,id=7336,type=8,code=0),reply=(src=192.168.0.211,dst=10.78.95.89,id=7336,type=0,code=0),zone=7
icmp,orig=(src=10.78.95.89,dst=192.168.0.211,id=7336,type=8,code=0),reply=(src=192.168.0.211,dst=10.78.95.89,id=7336,type=0,code=0),zone=1
0:
https://patchwork.ozlabs.org/project/ovn/patch/20220401175516.2139179-1-mmich...@redhat.com/
1:
https://github.com/ovn-org/ovn/commit/4deac4509abbedd6ffaecf27eed01ddefccea40a
---
ProblemType: Bug
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 Jun 9 11:35 seq
crw-rw---- 1 root audio 116, 33 Jun 9 11:35 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.20.11-0ubuntu82.1
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq',
'/dev/snd/timer'] failed with exit code 1:
CRDA: N/A
CasperMD5CheckResult: unknown
DistroRelease: Ubuntu 22.04
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
Lsusb:
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Lsusb-t:
/: Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/8p, 5000M
/: Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/8p, 480M
MachineType: QEMU Standard PC (Q35 + ICH9, 2009)
Package: linux (not installed)
PciMultimedia:
ProcEnviron:
TERM=xterm-256color
PATH=(custom, no user)
LANG=C.UTF-8
SHELL=/bin/bash
ProcFB: 0 virtio_gpudrmfb
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-5.15.0-37-generic
root=UUID=63713e6b-8e8d-4f97-ac5a-883317b24711 ro console=tty1 console=ttyS0
ProcVersionSignature: Ubuntu 5.15.0-37.39-generic 5.15.35
RelatedPackageVersions:
linux-restricted-modules-5.15.0-37-generic N/A
linux-backports-modules-5.15.0-37-generic N/A
linux-firmware 20220329.git681281e4-0ubuntu1
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
Tags: jammy uec-images
Uname: Linux 5.15.0-37-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: N/A
_MarkForUpload: True
dmi.bios.date: 02/06/2015
dmi.bios.release: 0.0
dmi.bios.vendor: EFI Development Kit II / OVMF
dmi.bios.version: 0.0.0
dmi.board.name: LXD
dmi.board.vendor: Canonical Ltd.
dmi.board.version: pc-q35-7.0
dmi.chassis.type: 1
dmi.chassis.vendor: QEMU
dmi.chassis.version: pc-q35-7.0
dmi.modalias:
dmi:bvnEFIDevelopmentKitII/OVMF:bvr0.0.0:bd02/06/2015:br0.0:svnQEMU:pnStandardPC(Q35+ICH9,2009):pvrpc-q35-7.0:rvnCanonicalLtd.:rnLXD:rvrpc-q35-7.0:cvnQEMU:ct1:cvrpc-q35-7.0:sku:
dmi.product.name: Standard PC (Q35 + ICH9, 2009)
dmi.product.version: pc-q35-7.0
dmi.sys.vendor: QEMU
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1967856/+subscriptions
--
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help : https://help.launchpad.net/ListHelp
