I've removed ubuntu-fan. I'm working on a fix for this.
What needs to happen is for the container NIC to use the MTU of the VXLAN accompanying the Fan bridge (which appears to be correctly offset from the underlay) rather than the bridge itself. ** No longer affects: ubuntu-fan (Ubuntu) -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to ubuntu-fan in Ubuntu. https://bugs.launchpad.net/bugs/1936842 Title: agent cannot be up on LXD/Fan network on OpenStack OVN/geneve mtu=1442 Status in juju: In Progress Bug description: When one runs OpenStack with MTU=1500 underlying network, Neutron/OVN by default will create a tenant/overlay network with MTU=1442(1500-58). However, when deploying a workload on top, the Fan network with Juju will use MTU=1450 which is actually bigger than that. Then, Juju agent cannot be up inside LXD/Fan on top of OpenStack. OpenStack deployment is based on https://jaas.ai/openstack-base and k8s as a workload on top of OpenStack is: https://jaas.ai/kubernetes-core $ openstack network list +--------------------------------------+----------+--------------------------------------+ | ID | Name | Subnets | +--------------------------------------+----------+--------------------------------------+ | ba9880e3-5f07-4b49-aeda-20dfa8fe66ec | internal | 45efcc4c-fb99-4fe8-98a4-77195ec4aef1 | | e27b1cef-c53b-42e0-b307-8ba2e622c2dd | ext_net | 5315d907-ce90-4605-ad72-b78229965b40 | +--------------------------------------+----------+--------------------------------------+ $ openstack network show internal +---------------------------+--------------------------------------+ | Field | Value | +---------------------------+--------------------------------------+ ... | mtu | 1442 | | name | internal | ... | provider:network_type | geneve | | provider:physical_network | None | | provider:segmentation_id | 1410 | ... +---------------------------+--------------------------------------+ $ juju machines -m k8s-on-openstack Machine State DNS Inst id Series AZ Message 0 started 192.168.151.75 0e96a5b1-3665-44f6-bcb7-4851ab6cd22d focal nova ACTIVE 0/lxd/0 pending juju-b9bd8b-0-lxd-0 focal nova Container started 1 started 192.168.151.66 6bfa5d2e-24e6-42b8-b5ec-1f2a0d0e6b02 focal nova ACTIVE [openstack instance/VM provisioned by Juju - ens3: mtu=1442, fan-252: mtu=1450] juju-b9bd8b-k8s-on-openstack-0:~# ip link 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1442 qdisc fq_codel state UP mode DEFAULT group default qlen 1000 link/ether fa:16:3e:48:85:85 brd ff:ff:ff:ff:ff:ff 3: fan-252: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP mode DEFAULT group default qlen 1000 link/ether 06:28:c7:b4:50:eb brd ff:ff:ff:ff:ff:ff 4: ftun0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1392 qdisc noqueue master fan-252 state UNKNOWN mode DEFAULT group default qlen 1000 link/ether 92:d7:07:6e:b4:db brd ff:ff:ff:ff:ff:ff 5: lxdbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default qlen 1000 link/ether 00:16:3e:03:0e:96 brd ff:ff:ff:ff:ff:ff 7: 0lxd0-0@if6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master fan-252 state UP mode DEFAULT group default qlen 1000 link/ether 06:28:c7:b4:50:eb brd ff:ff:ff:ff:ff:ff link-netnsid 0 [lxc config of LXD container as juju machine 0/lxd/0] juju-b9bd8b-k8s-on-openstack-0:~# lxc config show juju-b9bd8b-0-lxd-0 ... user.user-data: | #cloud-config apt_mirror: "" bootcmd: - install -D -m 644 /dev/null '/etc/netplan/99-juju.yaml' - |- printf '%s\n' 'network: version: 2 ethernets: eth0: match: macaddress: 00:16:3e:82:f9:44 dhcp4: true nameservers: search: [openstack.internal] addresses: [8.8.8.8, 8.8.4.4] mtu: 1450 ... devices: eth0: host_name: 0lxd0-0 hwaddr: 00:16:3e:82:f9:44 mtu: "1450" name: eth0 nictype: bridged parent: fan-252 type: nic [cloud-init-output.log - stuck at the initial apt update and also failing to fetch the agent binary] Cloud-init v. 21.2-3-g899bfaa9-0ubuntu2~20.04.1 running 'modules:config' at Mon, 19 Jul 2021 03:27:32 +0000. Up 53.99 seconds. Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease Err:2 http://security.ubuntu.com/ubuntu focal-security InRelease Connection failed [IP: 192.168.151.1 8000] Err:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease Connection failed [IP: 192.168.151.1 8000] Err:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease Connection failed [IP: 192.168.151.1 8000] Ign:5 http://archive.ubuntu.com/ubuntu focal/universe amd64 Packages Ign:6 http://archive.ubuntu.com/ubuntu focal/universe Translation-en Ign:7 http://archive.ubuntu.com/ubuntu focal/universe amd64 c-n-f Metadata Ign:8 http://archive.ubuntu.com/ubuntu focal/multiverse amd64 Packages Ign:9 http://archive.ubuntu.com/ubuntu focal/multiverse Translation-en Ign:10 http://archive.ubuntu.com/ubuntu focal/multiverse amd64 c-n-f Metadata Get:5 http://archive.ubuntu.com/ubuntu focal/universe amd64 Packages [8628 kB] Get:5 http://archive.ubuntu.com/ubuntu focal/universe amd64 Packages [8628 kB] Get:5 http://archive.ubuntu.com/ubuntu focal/universe amd64 Packages [8628 kB] Get:5 http://archive.ubuntu.com/ubuntu focal/universe amd64 Packages [8628 kB] Get:5 http://archive.ubuntu.com/ubuntu focal/universe amd64 Packages [8628 kB] Get:5 http://archive.ubuntu.com/ubuntu focal/universe amd64 Packages [8628 kB] + printf Attempt 5 to download agent binaries from %s...\n https://192.168.151.101:17070/model/9e5ed9b4-224d-4486-8dac-b56a70b9bd8b/tools/2.9.8-ubuntu-amd64 Attempt 5 to download agent binaries from https://192.168.151.101:17070/model/9e5ed9b4-224d-4486-8dac-b56a70b9bd8b/tools/2.9.8-ubuntu-amd64... + curl -sSfw agent binaries from %{url_effective} downloaded: HTTP %{http_code}; time %{time_total}s; size %{size_download} bytes; speed %{speed_download} bytes/s --connect-timeout 20 --noproxy * --insecure -o /var/lib/juju/tools/2.9.8-ubuntu-amd64/tools.tar.gz https://192.168.151.101:17070/model/9e5ed9b4-224d-4486-8dac-b56a70b9bd8b/tools/2.9.8-ubuntu-amd64 curl: (28) Operation timed out after 20000 milliseconds with 0 out of 0 bytes received agent binaries from https://192.168.151.101:17070/model/9e5ed9b4-224d-4486-8dac-b56a70b9bd8b/tools/2.9.8-ubuntu-amd64 downloaded: HTTP 000; time 20.000946s; size 0 bytes; speed 0.000 bytes/s + echo Download failed, retrying in 15s Download failed, retrying in 15s + sleep 15 [manually run tracepath to get pmtu] juju-b9bd8b-0-lxd-0:~# tracepath -n -m 5 192.168.151.1 1?: [LOCALHOST] pmtu 1450 1: 252.16.0.1 0.346ms 1: 252.16.0.1 0.051ms 2: 252.16.0.1 0.053ms pmtu 1442 2: no reply 3: 192.168.151.1 3.283ms reached Resume: pmtu 1442 hops 3 back 3 To manage notifications about this bug go to: https://bugs.launchpad.net/juju/+bug/1936842/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
