Public bug reported:
This bug represents intent to SRU a fix into Ubuntu Resolute to resolve
long boot timeouts on Oracle ISCSI root boots for images which are
dracut based.
Affects:
Cloud-init version: 26.1-0ubuntu2
Operating System Distribution: Ubuntu 26.04 (Resolute), dracut-based
initramfs
Cloud provider, platform or installer type: Oracle Cloud Infrastructure
(OCI), bare metal shapes (iSCSI root)
Description:
cloud-init is responsible for emitting keep_configuration (rendered as
critical: true in netplan, which sets KeepConfiguration=yes in systemd-
networkd) on the primary NIC of iSCSI-root instances. On Ubuntu 26.04
(Resolute) this is missing from the generated config, so systemd-
networkd may tear down the interface while the iSCSI root filesystem is
still mounted.
Without KeepConfiguration=yes, the address and route configuration on
the iSCSI-root interface is not preserved during shutdown. This appears
to leave the iSCSI target unreachable while the root filesystem is still
mounted, which is consistent with the I/O errors and journal abort seen
on the console. Manually applying critical: true reduced the reboot from
over 100 minutes to a few minutes.
More details are present in the upstream bug
https://github.com/canonical/cloud-init/issues/6915.
[ Impact ]
* On Oracle platform, cloud-init is unable to determine if the images
is booting with an iSCSI root on Dracut-based images which no longer
surfaces the klibc /run/net-* files. This prevented cloud-init from
defining the primary NIC as `critical: true` in /etc/netplan/50-cloud-
init.yaml which results in 100 minute reboot delays on a system as the
iSCSI root device gets torn down too early during reboot.
* This upload fixes the bug, by allowing cloud-init to correctly detect iSCSI
root from the kernel iBFT (/sys/firmware/ibft/target*/
flags) while keeping klibc as a fallback in the event that the ibft/target flag
files do not represent an iSCSI root device.
[ Test Plan ]
* Launch Oracle bare metal iSCSI-based instance shape with old cloud-
init
* Evaluate current failure symptoms, upgrade and clean reboot
# confirm cloud-init health with current release
cloud-init --version
cloud-init status --wait --format=yaml
echo $?
# Confirm baseline orig systemd boot time for comparison across clean
upgrade reboot
systemd-analyze blame
# confirm absent 'critical: true'
grep 'critical: true' /etc/netplan/50-cloud-init.yaml
# preserve orig netplan config
cp /etc/netplan/50-cloud-init.yaml 50-cloud-init.yaml.orig
# upgrade to -proposed cloud-init
echo deb "http://archive.ubuntu.com/ubuntu $(lsb_release -sc)-proposed main"
| sudo tee /etc/apt/sources.list.d/proposed.list
sudo apt update
sudo apt install cloud-init || echo "FAILURE bad exit code from -proposed
deb upgrade"
# Perform clean reboot with cloud-init
sudo cloud-init -clean -c all --logs --reboot
# SSH back into the SUT to confirm cloud-init health with -proposed release
cloud-init --version
cloud-init status --wait --format=yaml
echo $?
# Validate no extended boot time seen across reboot
systemd-analyze blame
# Compare netplan delta orig to -proposed to ensure it is only the added
critical: true
diff -urN 50-cloud-init.yaml.orig /etc/netplan/50-cloud-init.yaml
# Assert no expensive hang on reboot
sudo reboot
[ Where problems could occur ]
* Setting critical: true on a NIC in netplan can affect treatment of
DHCP lease renewals for systemd-networkd based images. This fix though
is a correction for a regression in previous expected behavior for
Oracle iSCSI-root based environment on Questing and earlier images where
`critical: true` was provided on Oracle systems.
* Typically areas of impact for critical: true/critical: false
behavior show up in either timeouts in journalctl logs related to the
NIC in question if it is torn down too early, or extended initial boot
times while waiting for a critical device that is not even present or
ready on the system.
[ Other Info ]
* Upstream bug fix and additional testing on Noble to ensure there
isn't regression potential if this were released to non-dracut images
https://github.com/canonical/cloud-init/pull/6921
** Affects: cloud-init (Ubuntu)
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2161665
Title:
resolute: Oracle 100 minute boot/reboot delay due to ISCSI root not
detected on dracut-based images
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/2161665/+subscriptions
--
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
