Public bug reported: The boot transaction doesn't finish on Ubuntu AWS Pro Xenial instances:
``` # cat /etc/cloud/build.info build_name: pro-server serial: 20241002 # systemctl status | grep State State: starting ``` snapd.seeded.service does never finish and blocks cloud-config.service, ua-auto-attach.service, cloud-final.service and the rest of the boot transaction: ``` # systemctl status snapd.seeded.service ● snapd.seeded.service - Wait until snapd is fully seeded Loaded: loaded (/lib/systemd/system/snapd.seeded.service; enabled; vendor preset: enabled) Active: activating (start) since Tue 2024-10-08 10:32:22 UTC; 15min ago Main PID: 1277 (snap) Tasks: 8 Memory: 12.0M CPU: 649ms CGroup: /system.slice/snapd.seeded.service └─1277 /usr/bin/snap wait system seed.loaded Oct 08 10:32:22 ip-10-0-3-253 systemd[1]: Starting Wait until snapd is fully seeded... root@ip-10-0-3-253:~/aciba# systemctl list-dependencies --reverse snapd.seeded.service snapd.seeded.service ● ├─cloud-final.service ● └─multi-user.target ● └─graphical.target # systemctl status ua-auto-attach.service ● ua-auto-attach.service - Ubuntu Pro auto attach Loaded: loaded (/lib/systemd/system/ua-auto-attach.service; enabled; vendor preset: enabled) Active: activating (start) since Tue 2024-10-08 10:32:19 UTC; 22min ago Main PID: 1094 (python3) Tasks: 9 Memory: 98.3M CPU: 3.170s CGroup: /system.slice/ua-auto-attach.service ├─1094 /usr/bin/python3 /usr/lib/ubuntu-advantage/auto_attach.py └─2185 /usr/bin/snap wait system seed.loaded Oct 08 10:32:19 ip-10-0-3-253 systemd[1]: Starting Ubuntu Pro auto attach... Oct 08 10:33:04 ip-10-0-3-253 python3[1094]: ["2024-10-08T10:33:04.175", "INFO", "ubuntupro.timer.update_messaging", "update_motd_messages", 70, "Updating Ubuntu Pro messages for MOTD.", {}] Oct 08 10:33:04 ip-10-0-3-253 python3[1094]: ["2024-10-08T10:33:04.272", "WARNING", "ubuntupro.system", "subp", 740, "Stderr: error: too early for operation, device not yet seeded or device model not acknowledge Oct 08 10:33:04 ip-10-0-3-253 python3[1094]: ["2024-10-08T10:33:04.788", "WARNING", "ubuntupro.system", "subp", 740, "Stderr: error: too early for operation, device not yet seeded or device model not acknowledge Oct 08 10:33:05 ip-10-0-3-253 python3[1094]: ["2024-10-08T10:33:05.806", "WARNING", "ubuntupro.system", "subp", 740, "Stderr: error: too early for operation, device not yet seeded or device model not acknowledge Oct 08 10:33:10 ip-10-0-3-253 python3[1094]: ["2024-10-08T10:33:10.827", "WARNING", "ubuntupro.system", "subp", 740, "Stderr: error: too early for operation, device not yet seeded or device model not acknowledge Oct 08 10:33:10 ip-10-0-3-253 python3[1094]: ["2024-10-08T10:33:10.827", "WARNING", "ubuntupro.entitlements.livepatch", "_perform_enable", 112, "Failed to install snapd as a snap", {"exc_info": "Traceback (most ...skipping... ● ua-auto-attach.service - Ubuntu Pro auto attach Loaded: loaded (/lib/systemd/system/ua-auto-attach.service; enabled; vendor preset: enabled) Active: activating (start) since Tue 2024-10-08 10:32:19 UTC; 22min ago Main PID: 1094 (python3) Tasks: 9 Memory: 98.3M CPU: 3.170s CGroup: /system.slice/ua-auto-attach.service ├─1094 /usr/bin/python3 /usr/lib/ubuntu-advantage/auto_attach.py └─2185 /usr/bin/snap wait system seed.loaded Oct 08 10:32:19 ip-10-0-3-253 systemd[1]: Starting Ubuntu Pro auto attach... Oct 08 10:33:04 ip-10-0-3-253 python3[1094]: ["2024-10-08T10:33:04.175", "INFO", "ubuntupro.timer.update_messaging", "update_motd_messages", 70, "Updating Ubuntu Pro messages for MOTD.", {}] Oct 08 10:33:04 ip-10-0-3-253 python3[1094]: ["2024-10-08T10:33:04.272", "WARNING", "ubuntupro.system", "subp", 740, "Stderr: error: too early for operation, device not yet seeded or device model not acknowledge Oct 08 10:33:04 ip-10-0-3-253 python3[1094]: ["2024-10-08T10:33:04.788", "WARNING", "ubuntupro.system", "subp", 740, "Stderr: error: too early for operation, device not yet seeded or device model not acknowledge Oct 08 10:33:05 ip-10-0-3-253 python3[1094]: ["2024-10-08T10:33:05.806", "WARNING", "ubuntupro.system", "subp", 740, "Stderr: error: too early for operation, device not yet seeded or device model not acknowledge Oct 08 10:33:10 ip-10-0-3-253 python3[1094]: ["2024-10-08T10:33:10.827", "WARNING", "ubuntupro.system", "subp", 740, "Stderr: error: too early for operation, device not yet seeded or device model not acknowledge Oct 08 10:33:10 ip-10-0-3-253 python3[1094]: ["2024-10-08T10:33:10.827", "WARNING", "ubuntupro.entitlements.livepatch", "_perform_enable", 112, "Failed to install snapd as a snap", {"exc_info": "Traceback (most ``` The issue is that amazon-ssm-agent is preseeded in the image and it requires core22, which requires snapd 2.55, but snapd 2.55 is not available at boot time provoking snaps.preseed.service to never finish: ``` # cat /var/lib/snapd/seed/seed.yaml snaps: - name: core22 channel: stable file: core22_1621.snap - name: snapd channel: stable file: snapd_21759.snap - name: amazon-ssm-agent channel: stable/ubuntu-16.04 classic: true file: amazon-ssm-agent_9565.snap # apt-cache show snapd | grep Version Version: 2.54.3+16.04.0ubuntu0.1~esm6 Version: 2.48.3 Version: 2.0.2 # journalctl -u snapd.seeded -- Logs begin at Mon 2024-10-14 13:03:41 UTC, end at Mon 2024-10-14 13:48:54 UTC. -- Oct 14 13:03:52 ip-10-0-3-166 systemd[1]: Starting Wait until snapd is fully seeded... # journalctl -u snapd* -- Logs begin at Mon 2024-10-14 13:03:41 UTC, end at Mon 2024-10-14 13:48:54 UTC. -- Oct 14 13:03:45 ubuntu systemd[1]: Starting Load AppArmor profiles managed internally by snapd... Oct 14 13:03:45 ubuntu snapd-apparmor[667]: find: '/var/lib/snapd/apparmor/profiles/': No such file or directory Oct 14 13:03:45 ubuntu systemd[1]: Started Load AppArmor profiles managed internally by snapd. Oct 14 13:03:50 ip-10-0-3-166 systemd[1]: Starting Socket activation for snappy daemon. Oct 14 13:03:50 ip-10-0-3-166 systemd[1]: Listening on Socket activation for snappy daemon. Oct 14 13:03:50 ip-10-0-3-166 systemd[1]: Starting Snap Daemon... Oct 14 13:03:51 ip-10-0-3-166 snapd[1097]: AppArmor status: apparmor is enabled and all features are available Oct 14 13:03:51 ip-10-0-3-166 snapd[1097]: daemon.go:246: started snapd/2.54.3+16.04.0ubuntu0.1~esm6 (series 16; classic) ubuntu/16.04 (amd64) linux/4.4.0-1174-aws. Oct 14 13:03:51 ip-10-0-3-166 snapd[1097]: daemon.go:339: adjusting startup timeout by 30s (pessimistic estimate of 30s plus 5s per snap) Oct 14 13:03:52 ip-10-0-3-166 snapd[1097]: helpers.go:137: error trying to compare the snap system key: system-key missing on disk Oct 14 13:03:52 ip-10-0-3-166 systemd[1]: Started Snap Daemon. Oct 14 13:03:52 ip-10-0-3-166 systemd[1]: Starting Wait until snapd is fully seeded... Oct 14 13:04:10 ip-10-0-3-166 snapd[1097]: stateengine.go:149: state ensure error: devicemgr: cannot seed: snap "core22" assumes unsupported features: snapd2.55.5 (try to refresh snapd) Oct 14 13:04:10 ip-10-0-3-166 snapd[1097]: api_snaps.go:307: Installing snap "snapd" revision unset Oct 14 13:04:11 ip-10-0-3-166 snapd[1097]: api_snaps.go:307: Installing snap "snapd" revision unset Oct 14 13:04:13 ip-10-0-3-166 snapd[1097]: api_snaps.go:307: Installing snap "snapd" revision unset Oct 14 13:04:18 ip-10-0-3-166 snapd[1097]: api_snaps.go:307: Installing snap "snapd" revision unset Oct 14 13:08:54 ip-10-0-3-166 snapd[1097]: stateengine.go:149: state ensure error: devicemgr: cannot seed: snap "core22" assumes unsupported features: snapd2.55.5 (try to refresh snapd) Oct 14 13:13:54 ip-10-0-3-166 snapd[1097]: stateengine.go:149: state ensure error: devicemgr: cannot seed: snap "core22" assumes unsupported features: snapd2.55.5 (try to refresh snapd) Oct 14 13:18:54 ip-10-0-3-166 snapd[1097]: stateengine.go:149: state ensure error: devicemgr: cannot seed: snap "core22" assumes unsupported features: snapd2.55.5 (try to refresh snapd) Oct 14 13:23:54 ip-10-0-3-166 snapd[1097]: stateengine.go:149: state ensure error: devicemgr: cannot seed: snap "core22" assumes unsupported features: snapd2.55.5 (try to refresh snapd) Oct 14 13:28:54 ip-10-0-3-166 snapd[1097]: stateengine.go:149: state ensure error: devicemgr: cannot seed: snap "core22" assumes unsupported features: snapd2.55.5 (try to refresh snapd) Oct 14 13:33:54 ip-10-0-3-166 snapd[1097]: stateengine.go:149: state ensure error: devicemgr: cannot seed: snap "core22" assumes unsupported features: snapd2.55.5 (try to refresh snapd) Oct 14 13:38:54 ip-10-0-3-166 snapd[1097]: stateengine.go:149: state ensure error: devicemgr: cannot seed: snap "core22" assumes unsupported features: snapd2.55.5 (try to refresh snapd) Oct 14 13:43:54 ip-10-0-3-166 snapd[1097]: stateengine.go:149: state ensure error: devicemgr: cannot seed: snap "core22" assumes unsupported features: snapd2.55.5 (try to refresh snapd) Oct 14 13:48:54 ip-10-0-3-166 snapd[1097]: stateengine.go:149: state ensure error: devicemgr: cannot seed: snap "core22" assumes unsupported features: snapd2.55.5 (try to refresh snapd) ``` ** Affects: cloud-images Importance: Undecided Status: New ** Affects: snapd (Ubuntu) Importance: Undecided Status: New ** Affects: snapd (Ubuntu Xenial) Importance: Undecided Status: New ** Also affects: snapd (Ubuntu) Importance: Undecided Status: New ** Also affects: snapd (Ubuntu Xenial) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2084667 Title: AWS Pro xenial images do not fully boot due to snapd.seeded.service never finishing To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-images/+bug/2084667/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs