Public bug reported: This was observed while testing landscape autopilot to deploy an openstack cluster. This uses juju deployed lxd containers to host various openstack services. Creation of a container failed, causing the entire openstack deployment to fail.
lxd: 2.0.9 (xenial) juju: 1:2.1~rc2-0ubuntu1~16.04.1~juju1 landscape: 17.01~bzr10906+jenkins3476-2 I don't know if this is an issue with lxd, juju, or something else. From what I can see, the container requested by juju failed to start due to err="Container is already running a start operation". Juju then deleted the container and tried again about a minute later. The retry was successful. Landscape errors out on the first failure and isn't aware that things worked after the retry. [from landscape-0-inner-logs/base-machine-2/var/log/lxd/lxd.log] ephemeral=false lvl=info msg="Creating container" name=juju-2c21a7-1-lxd-4 t=2017-02-18T14:04:57+0000 ephemeral=false lvl=info msg="Created container" name=juju-2c21a7-1-lxd-4 t=2017-02-18T14:04:57+0000 action=start creation date=2017-02-18T14:04:57+0000 ephemeral=false lvl=info msg="Starting container" name=juju-2c21a7-1-lxd-4 stateful=false t=2017-02-18T14:05:27+0000 container=juju-2c21a7-1-lxd-4 err="Container is already running a start operation" lvl=eror msg="stop hook failed" t=2017-02-18T14:05:28+0000 action=start creation date=2017-02-18T14:04:57+0000 ephemeral=false lvl=eror msg="Failed starting container" name=juju-2c21a7-1-lxd-4 stateful=false t=2017-02-18T14:05:33+0000 creation date=2017-02-18T14:04:57+0000 ephemeral=false lvl=info msg="Deleting container" name=juju-2c21a7-1-lxd-4 t=2017-02-18T14:05:34+0000 creation date=2017-02-18T14:04:57+0000 ephemeral=false lvl=info msg="Deleted container" name=juju-2c21a7-1-lxd-4 t=2017-02-18T14:05:34+0000 ephemeral=false lvl=info msg="Creating container" name=juju-2c21a7-1-lxd-4 t=2017-02-18T14:05:46+0000 ephemeral=false lvl=info msg="Created container" name=juju-2c21a7-1-lxd-4 t=2017-02-18T14:05:46+0000 action=start creation date=2017-02-18T14:05:46+0000 ephemeral=false lvl=info msg="Starting container" name=juju-2c21a7-1-lxd-4 stateful=false t=2017-02-18T14:06:11+0000 action=start creation date=2017-02-18T14:05:46+0000 ephemeral=false lvl=info msg="Started container" name=juju-2c21a7-1-lxd-4 stateful=false t=2017-02-18T14:06:11+0000 The failure and retry is visible in the juju logsink.log: [from landscape-0-inner-logs/bootstrap/var/log/juju/logsink.log] failed to start instance (Error calling 'lxd forkstart juju-2c21a7-1-lxd-4 /var/lib/lxd/containers /var/log/lxd/juju-2c21a7-1-lxd-4/lxc.conf': err='exit status 1' lxc 20160218140528.391 ERROR lxc_apparmor - lsm/apparmor.c:apparmor_process_label_set:234 - No such file or directory - failed to change apparmor profile to lxd-juju-2c21a7-1-lxd-4_</var/lib/lxd>//&:lxd-juju-2c21a7-1-lxd-4_<var-lib-lxd>: lxc 20160218140528.391 ERROR lxc_sync - sync.c:__sync_wait:57 - An error occurred in another process (expected sequence number 5) lxc 20160218140528.391 ERROR lxc_start - start.c:__lxc_start:1346 - Failed to spawn container "juju-2c21a7-1-lxd-4". lxc 20160218140528.986 ERROR lxc_conf - conf.c:run_buffer:405 - Script exited with status 1. lxc 20160218140528.986 ERROR lxc_start - start.c:lxc_fini:546 - Failed to run lxc.hook.post-stop for container "juju-2c21a7-1-lxd-4". ), retrying in 10s (3 more attempts) ... e331b8e0-d8ca-4007-8573-d9dd932c21a7: machine-1-lxd-4 2017-02-18 14:07:02 INFO juju.cmd supercommand.go:63 running jujud [2.1-rc2 gc go1.6] e331b8e0-d8ca-4007-8573-d9dd932c21a7: machine-1-lxd-4 2017-02-18 14:07:02 DEBUG juju.cmd supercommand.go:64 args: []string{"/var/lib/juju/tools/machine-1-lxd-4/jujud", "machine", "--data-dir", "/var/lib/juju", "--machine-id", "1/lxd/4", "--debug"} This was found by an automated test [1]. We run 8-12 or so of these per day and this is the first time I've seen this particular error. [1] - https://ci.lscape.net/job/landscape-system-tests/5330/ ** Affects: lxd (Ubuntu) Importance: Undecided Status: New ** Attachment added: "all-logs.tar.gz" https://bugs.launchpad.net/bugs/1667444/+attachment/4825387/+files/all-logs.tar.gz -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1667444 Title: Failed to start lxd container because it is already running a start operation To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/lxd/+bug/1667444/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs