Public bug reported:

The boot transaction doesn't finish on Ubuntu AWS Pro Xenial instances:

```
# cat /etc/cloud/build.info
build_name: pro-server
serial: 20241002

# systemctl status | grep State
    State: starting
```

snapd.seeded.service does never finish and blocks cloud-config.service,
ua-auto-attach.service, cloud-final.service and the rest of the boot
transaction:

```
# systemctl status snapd.seeded.service 
● snapd.seeded.service - Wait until snapd is fully seeded
   Loaded: loaded (/lib/systemd/system/snapd.seeded.service; enabled; vendor 
preset: enabled)
   Active: activating (start) since Tue 2024-10-08 10:32:22 UTC; 15min ago
 Main PID: 1277 (snap)
    Tasks: 8
   Memory: 12.0M
      CPU: 649ms
   CGroup: /system.slice/snapd.seeded.service
           └─1277 /usr/bin/snap wait system seed.loaded

Oct 08 10:32:22 ip-10-0-3-253 systemd[1]: Starting Wait until snapd is fully 
seeded...
root@ip-10-0-3-253:~/aciba# systemctl list-dependencies --reverse 
snapd.seeded.service 
snapd.seeded.service
● ├─cloud-final.service
● └─multi-user.target
●   └─graphical.target

# systemctl status ua-auto-attach.service                                       
                                                                                
                         
● ua-auto-attach.service - Ubuntu Pro auto attach
   Loaded: loaded (/lib/systemd/system/ua-auto-attach.service; enabled; vendor 
preset: enabled)
   Active: activating (start) since Tue 2024-10-08 10:32:19 UTC; 22min ago
 Main PID: 1094 (python3)
    Tasks: 9
   Memory: 98.3M
      CPU: 3.170s
   CGroup: /system.slice/ua-auto-attach.service
           ├─1094 /usr/bin/python3 /usr/lib/ubuntu-advantage/auto_attach.py
           └─2185 /usr/bin/snap wait system seed.loaded

Oct 08 10:32:19 ip-10-0-3-253 systemd[1]: Starting Ubuntu Pro auto attach...
Oct 08 10:33:04 ip-10-0-3-253 python3[1094]: ["2024-10-08T10:33:04.175", 
"INFO", "ubuntupro.timer.update_messaging", "update_motd_messages", 70, 
"Updating Ubuntu Pro messages for MOTD.", {}]
Oct 08 10:33:04 ip-10-0-3-253 python3[1094]: ["2024-10-08T10:33:04.272", 
"WARNING", "ubuntupro.system", "subp", 740, "Stderr: error: too early for 
operation, device not yet seeded or device model not acknowledge
Oct 08 10:33:04 ip-10-0-3-253 python3[1094]: ["2024-10-08T10:33:04.788", 
"WARNING", "ubuntupro.system", "subp", 740, "Stderr: error: too early for 
operation, device not yet seeded or device model not acknowledge
Oct 08 10:33:05 ip-10-0-3-253 python3[1094]: ["2024-10-08T10:33:05.806", 
"WARNING", "ubuntupro.system", "subp", 740, "Stderr: error: too early for 
operation, device not yet seeded or device model not acknowledge
Oct 08 10:33:10 ip-10-0-3-253 python3[1094]: ["2024-10-08T10:33:10.827", 
"WARNING", "ubuntupro.system", "subp", 740, "Stderr: error: too early for 
operation, device not yet seeded or device model not acknowledge
Oct 08 10:33:10 ip-10-0-3-253 python3[1094]: ["2024-10-08T10:33:10.827", 
"WARNING", "ubuntupro.entitlements.livepatch", "_perform_enable", 112, "Failed 
to install snapd as a snap", {"exc_info": "Traceback (most 
...skipping...
● ua-auto-attach.service - Ubuntu Pro auto attach
   Loaded: loaded (/lib/systemd/system/ua-auto-attach.service; enabled; vendor 
preset: enabled)
   Active: activating (start) since Tue 2024-10-08 10:32:19 UTC; 22min ago
 Main PID: 1094 (python3)
    Tasks: 9
   Memory: 98.3M
      CPU: 3.170s
   CGroup: /system.slice/ua-auto-attach.service
           ├─1094 /usr/bin/python3 /usr/lib/ubuntu-advantage/auto_attach.py
           └─2185 /usr/bin/snap wait system seed.loaded

Oct 08 10:32:19 ip-10-0-3-253 systemd[1]: Starting Ubuntu Pro auto attach...
Oct 08 10:33:04 ip-10-0-3-253 python3[1094]: ["2024-10-08T10:33:04.175", 
"INFO", "ubuntupro.timer.update_messaging", "update_motd_messages", 70, 
"Updating Ubuntu Pro messages for MOTD.", {}]
Oct 08 10:33:04 ip-10-0-3-253 python3[1094]: ["2024-10-08T10:33:04.272", 
"WARNING", "ubuntupro.system", "subp", 740, "Stderr: error: too early for 
operation, device not yet seeded or device model not acknowledge
Oct 08 10:33:04 ip-10-0-3-253 python3[1094]: ["2024-10-08T10:33:04.788", 
"WARNING", "ubuntupro.system", "subp", 740, "Stderr: error: too early for 
operation, device not yet seeded or device model not acknowledge
Oct 08 10:33:05 ip-10-0-3-253 python3[1094]: ["2024-10-08T10:33:05.806", 
"WARNING", "ubuntupro.system", "subp", 740, "Stderr: error: too early for 
operation, device not yet seeded or device model not acknowledge
Oct 08 10:33:10 ip-10-0-3-253 python3[1094]: ["2024-10-08T10:33:10.827", 
"WARNING", "ubuntupro.system", "subp", 740, "Stderr: error: too early for 
operation, device not yet seeded or device model not acknowledge
Oct 08 10:33:10 ip-10-0-3-253 python3[1094]: ["2024-10-08T10:33:10.827", 
"WARNING", "ubuntupro.entitlements.livepatch", "_perform_enable", 112, "Failed 
to install snapd as a snap", {"exc_info": "Traceback (most 
```

The issue is that amazon-ssm-agent is preseeded in the image and it
requires core22, which requires snapd 2.55, but snapd 2.55 is not
available at boot time provoking snaps.preseed.service to never finish:

```
# cat /var/lib/snapd/seed/seed.yaml 
snaps:
  -
    name: core22
    channel: stable
    file: core22_1621.snap
  -
    name: snapd
    channel: stable
    file: snapd_21759.snap
  -
    name: amazon-ssm-agent
    channel: stable/ubuntu-16.04
    classic: true
    file: amazon-ssm-agent_9565.snap

# apt-cache show snapd | grep Version
Version: 2.54.3+16.04.0ubuntu0.1~esm6
Version: 2.48.3
Version: 2.0.2

# journalctl -u snapd.seeded
-- Logs begin at Mon 2024-10-14 13:03:41 UTC, end at Mon 2024-10-14 13:48:54 
UTC. --
Oct 14 13:03:52 ip-10-0-3-166 systemd[1]: Starting Wait until snapd is fully 
seeded...

# journalctl -u snapd*
-- Logs begin at Mon 2024-10-14 13:03:41 UTC, end at Mon 2024-10-14 13:48:54 
UTC. --
Oct 14 13:03:45 ubuntu systemd[1]: Starting Load AppArmor profiles managed 
internally by snapd...
Oct 14 13:03:45 ubuntu snapd-apparmor[667]: find: 
'/var/lib/snapd/apparmor/profiles/': No such file or directory
Oct 14 13:03:45 ubuntu systemd[1]: Started Load AppArmor profiles managed 
internally by snapd.
Oct 14 13:03:50 ip-10-0-3-166 systemd[1]: Starting Socket activation for snappy 
daemon.
Oct 14 13:03:50 ip-10-0-3-166 systemd[1]: Listening on Socket activation for 
snappy daemon.
Oct 14 13:03:50 ip-10-0-3-166 systemd[1]: Starting Snap Daemon...
Oct 14 13:03:51 ip-10-0-3-166 snapd[1097]: AppArmor status: apparmor is enabled 
and all features are available
Oct 14 13:03:51 ip-10-0-3-166 snapd[1097]: daemon.go:246: started 
snapd/2.54.3+16.04.0ubuntu0.1~esm6 (series 16; classic) ubuntu/16.04 (amd64) 
linux/4.4.0-1174-aws.
Oct 14 13:03:51 ip-10-0-3-166 snapd[1097]: daemon.go:339: adjusting startup 
timeout by 30s (pessimistic estimate of 30s plus 5s per snap)
Oct 14 13:03:52 ip-10-0-3-166 snapd[1097]: helpers.go:137: error trying to 
compare the snap system key: system-key missing on disk
Oct 14 13:03:52 ip-10-0-3-166 systemd[1]: Started Snap Daemon.
Oct 14 13:03:52 ip-10-0-3-166 systemd[1]: Starting Wait until snapd is fully 
seeded...
Oct 14 13:04:10 ip-10-0-3-166 snapd[1097]: stateengine.go:149: state ensure 
error: devicemgr: cannot seed: snap "core22" assumes unsupported features: 
snapd2.55.5 (try to refresh snapd)
Oct 14 13:04:10 ip-10-0-3-166 snapd[1097]: api_snaps.go:307: Installing snap 
"snapd" revision unset
Oct 14 13:04:11 ip-10-0-3-166 snapd[1097]: api_snaps.go:307: Installing snap 
"snapd" revision unset
Oct 14 13:04:13 ip-10-0-3-166 snapd[1097]: api_snaps.go:307: Installing snap 
"snapd" revision unset
Oct 14 13:04:18 ip-10-0-3-166 snapd[1097]: api_snaps.go:307: Installing snap 
"snapd" revision unset
Oct 14 13:08:54 ip-10-0-3-166 snapd[1097]: stateengine.go:149: state ensure 
error: devicemgr: cannot seed: snap "core22" assumes unsupported features: 
snapd2.55.5 (try to refresh snapd)
Oct 14 13:13:54 ip-10-0-3-166 snapd[1097]: stateengine.go:149: state ensure 
error: devicemgr: cannot seed: snap "core22" assumes unsupported features: 
snapd2.55.5 (try to refresh snapd)
Oct 14 13:18:54 ip-10-0-3-166 snapd[1097]: stateengine.go:149: state ensure 
error: devicemgr: cannot seed: snap "core22" assumes unsupported features: 
snapd2.55.5 (try to refresh snapd)
Oct 14 13:23:54 ip-10-0-3-166 snapd[1097]: stateengine.go:149: state ensure 
error: devicemgr: cannot seed: snap "core22" assumes unsupported features: 
snapd2.55.5 (try to refresh snapd)
Oct 14 13:28:54 ip-10-0-3-166 snapd[1097]: stateengine.go:149: state ensure 
error: devicemgr: cannot seed: snap "core22" assumes unsupported features: 
snapd2.55.5 (try to refresh snapd)
Oct 14 13:33:54 ip-10-0-3-166 snapd[1097]: stateengine.go:149: state ensure 
error: devicemgr: cannot seed: snap "core22" assumes unsupported features: 
snapd2.55.5 (try to refresh snapd)
Oct 14 13:38:54 ip-10-0-3-166 snapd[1097]: stateengine.go:149: state ensure 
error: devicemgr: cannot seed: snap "core22" assumes unsupported features: 
snapd2.55.5 (try to refresh snapd)
Oct 14 13:43:54 ip-10-0-3-166 snapd[1097]: stateengine.go:149: state ensure 
error: devicemgr: cannot seed: snap "core22" assumes unsupported features: 
snapd2.55.5 (try to refresh snapd)
Oct 14 13:48:54 ip-10-0-3-166 snapd[1097]: stateengine.go:149: state ensure 
error: devicemgr: cannot seed: snap "core22" assumes unsupported features: 
snapd2.55.5 (try to refresh snapd)
```

** Affects: cloud-images
     Importance: Undecided
         Status: New

** Affects: snapd (Ubuntu)
     Importance: Undecided
         Status: New

** Affects: snapd (Ubuntu Xenial)
     Importance: Undecided
         Status: New

** Also affects: snapd (Ubuntu)
   Importance: Undecided
       Status: New

** Also affects: snapd (Ubuntu Xenial)
   Importance: Undecided
       Status: New

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2084667

Title:
  AWS Pro xenial images do not fully boot due to snapd.seeded.service
  never finishing

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-images/+bug/2084667/+subscriptions


-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

Reply via email to