Between initramfs-tools 0.142ubuntu8 and 0.142ubuntu9, we've Replaced
dhclient by dhcpcd (LP: #2024164).
When a DHCP server provides MTU settings to dhcpcd, it configures the
routes with the appropriate mtu value (due to "option interface_mtu" in
/etc/dhcpcd.conf), but it does not configure the MTU setting in the
interface. So, we end up having a mismatch between interface and route
MTU setting, as observed below:
root@(none):/# ip a
1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group
default qlen 1000
link/ether 02:00:17:05:ee:5a brd ff:ff:ff:ff:ff:ff
altname enp0s3
inet 10.0.0.21/24 brd 10.0.0.255 scope global dynamic noprefixroute ens3
valid_lft 86048sec preferred_lft 75248sec
inet6 fe80::17ff:fe05:ee5a/64 scope link
valid_lft forever preferred_lft forever
root@(none):/# ip route show
default via 10.0.0.1 dev ens3 proto dhcp src 10.0.0.21 metric 1002 mtu 9000
10.0.0.0/24 dev ens3 proto dhcp scope link src 10.0.0.21 metric 1002 mtu 9000
169.254.0.0/16 dev ens3 proto dhcp scope link src 10.0.0.21 metric 1002 mtu 9000
If we go back to initramfs-tools 0.142ubuntu8 (and, consequently to
dhclient), the MTU settings will match:
root@(none):/# ip a
1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP group
default qlen 1000
link/ether 02:00:17:05:ee:5a brd ff:ff:ff:ff:ff:ff
altname enp0s3
inet 10.0.0.21/24 brd 10.0.0.255 scope global ens3
valid_lft forever preferred_lft forever
inet6 fe80::17ff:fe05:ee5a/64 scope link
valid_lft forever preferred_lft forever
root@(none):/# ip route show
default via 10.0.0.1 dev ens3
10.0.0.0/24 dev ens3 proto kernel scope link src 10.0.0.21
169.254.0.0/16 dev ens3 scope link
Due to this mismatch when using dhcpcd, certain network activities might
fail. For example, in Oracle instances, a curl to the DataSource will
just hang:
curl --connect-timeout 10 --max-time 15 -H "Authorization: Bearer
Oracle" -L http://169.254.169.254/opc/v2/instance/
If you either reduce the route MTU setting down to 1500 OR increase the
interface MTU setting to 9000, curl works:
option 1 - Reduce route setting:
ip route delete 169.254.0.0/16 dev ens3 proto dhcp scope link src 10.0.0.21
metric 1002 mtu 9000
ip route add 169.254.0.0/16 dev ens3 scope link
option 2 - Increase interface setting:
ip link set mtu 9000 ens3
The correct MTU setting here is 9000. When you fully boot the instance,
the MTU gets configured appropriately (outside of initramfs)
I also checked the Paravirtualized instances (uses virtio_net interfaces
instead of mlx5_core) and the mismatch is also there, but curl still
works, so the Paravirtualized instance is also exposed to the bug but
more resistant to it.
While investigating this, I noticed that in past releases of dhcpcd, we
had a hook 10-mtu, that would automatically configure the mtu setting
based on information it received from the dhcp server. This file was
removed by the following commit:
https://github.com/NetworkConfiguration/dhcpcd/commit/ca6cdf5847cda720b65793ea6827b1b373c62382
There are some discussion around why this was removed here:
https://github.com/NetworkConfiguration/dhcpcd/issues/67
Anyway, this causes such mismatch between interface and route settings,
which cause curl to the IMDS to hang and cloud-init to fail.
** Bug watch added: github.com/NetworkConfiguration/dhcpcd/issues #67
https://github.com/NetworkConfiguration/dhcpcd/issues/67
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2056194
Title:
Networking broken in early boot on Oracle Native instances
To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-images/+bug/2056194/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
