[Bug 1675760] Re: rootfs issues - 2.8 LXD release

2017-03-27 Thread Stéphane Graber
Ok, marking invalid then, since the problem boils down to the directory backend not being able to do instant copies of the image and so there being a time where the container is still unpacking its filesystem. To reflect our IRC discussion, the directory backend is really meant as a fallback optio

[Bug 1675760] Re: rootfs issues - 2.8 LXD release

2017-03-27 Thread Lorenzo Cavassa
Running the containers 'init' operation in the concurrent way and looking with a 'top', you can see the related 'unsquashfs' operations running: for X in `seq 1 4`; do lxc init ubuntu: test$X & done top Waiting until the unsquashfs terminates and then starting the containers, all is good and the

[Bug 1675760] Re: rootfs issues - 2.8 LXD release

2017-03-27 Thread Lorenzo Cavassa
Yes, I unmounted it. getfacl: Removing leading '/' from absolute path names # file: var/lib/lxd/storage-pools/default/containers/test1 # owner: 10 # group: 10 user::rwx user:10:r-x group::r-x mask::r-x other::r-x -- You received this bug notification because you are a member of Ubunt

[Bug 1675760] Re: rootfs issues - 2.8 LXD release

2017-03-24 Thread Stéphane Graber
Did you try the unmount I mentioned earlier? Filesystem permissions look reasonable too, unless the posix ACL is messing with things somehow. Can you post the output of: - getfacl /var/lib/lxd/storage-pools/default/containers/test1 -- You received this bug notification because you are a membe

[Bug 1675760] Re: rootfs issues - 2.8 LXD release

2017-03-24 Thread Lorenzo Cavassa
VM upgraded to: Linux ubuntu 4.4.0-66-generic #87 Same behaviour. Here attached the output of the 'ls' commands. ** Attachment added: "lxd.listing" https://bugs.launchpad.net/ubuntu/+source/lxd/+bug/1675760/+attachment/4844752/+files/lxd.listing -- You received this bug notification becau

[Bug 1675760] Re: rootfs issues - 2.8 LXD release

2017-03-24 Thread Stéphane Graber
Hmm, not seeing anything obviously wrong with /proc here though... Can you try: - umount /proc/sys/fs/binfmt_misc And then attempt to start a container again? If that still fails, please post the output of: - ls -lh /var/lib/lxd - ls -lh /var/lib/lxd/containers - ls -lh /var/lib/lxd/storage-

[Bug 1675760] Re: rootfs issues - 2.8 LXD release

2017-03-24 Thread Stéphane Graber
Also note that both kernels you mentioned in this report so far are severely outdated. You initially listed 4.8.0-36 when current is 4.8.0-42 and on your LXD 2.12 system, you listed 4.4.0-31 when current is 4.4.0-67. Since the error hints at a kernel mount protection issue, you really ought to mak

[Bug 1675760] Re: rootfs issues - 2.8 LXD release

2017-03-24 Thread Lorenzo Cavassa
Here attached there is the log of a single container (test1), as requested by Christian. lxc info --show-log test1 ** Attachment added: "'test1' container log" https://bugs.launchpad.net/ubuntu/+source/lxd/+bug/1675760/+attachment/4844639/+files/test1.log -- You received this bug notificati

Re: [Bug 1675760] Re: rootfs issues - 2.8 LXD release

2017-03-24 Thread Christian Brauner
@stgraber, that sounds like one of those empty directories similar to the issues we had with the empty xen directory. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1675760 Title: rootfs issues - 2.

[Bug 1675760] Re: rootfs issues - 2.8 LXD release

2017-03-24 Thread Lorenzo Cavassa
cat /proc/self/mounts http://pastebin.ubuntu.com/24241515/ -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1675760 Title: rootfs issues - 2.8 LXD release To manage notifications about this bug go t

[Bug 1675760] Re: rootfs issues - 2.8 LXD release

2017-03-24 Thread Stéphane Graber
lxc 20170324150557.624 ERRORlxc_conf - conf.c:lxc_mount_auto_mounts:801 - Permission denied - error mounting proc on /usr/lib/x86_64-linux-gnu/lxc/proc flags 14 That's the kernel overmounting protection kicking in for some reason. Can you paste "cat /proc/self/mounts" on the host? -- You re

Re: [Bug 1675760] Re: rootfs issues - 2.8 LXD release

2017-03-24 Thread Christian Brauner
Can you please run LXD in debug mode lxd --debug --group lxd and then append the full log for one of the containers that fails to start? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1675760 Title:

[Bug 1675760] Re: rootfs issues - 2.8 LXD release

2017-03-24 Thread Lorenzo Cavassa
'lxc info' output: http://pastebin.ubuntu.com/24241376/ 'lxc config show' output: http://pastebin.ubuntu.com/24241379/ I ran: for X in `seq 1 4`; do lxc init ubuntu: test$X & done and then, after a while: for X in `seq 1 4`; do lxc start test$X & done This is what I got in the /var/log/lxd/

[Bug 1675760] Re: rootfs issues - 2.8 LXD release

2017-03-24 Thread Lorenzo Cavassa
A freshly created one (KVM VM). LXD 2.12 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1675760 Title: rootfs issues - 2.8 LXD release To manage notifications about this bug go to: https://bugs.lau

Re: [Bug 1675760] Re: rootfs issues - 2.8 LXD release

2017-03-24 Thread Christian Brauner
Was this on a freshly created LXD instance or on an upgraded LXD instance? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1675760 Title: rootfs issues - 2.8 LXD release To manage notifications abou

[Bug 1675760] Re: rootfs issues - 2.8 LXD release

2017-03-24 Thread Stéphane Graber
Please post "lxc info" and "lxc config show" for the LXD 2.12 based setup. ** Changed in: lxd (Ubuntu) Status: New => Incomplete -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1675760 Title:

[Bug 1675760] Re: rootfs issues - 2.8 LXD release

2017-03-24 Thread Stéphane Graber
Also, since this is a public bug, please refrain from using private paste services. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1675760 Title: rootfs issues - 2.8 LXD release To manage notificat

[Bug 1675760] Re: rootfs issues - 2.8 LXD release

2017-03-24 Thread Lorenzo Cavassa
We ran new tests: some issue shows up even using LXD 2.12 If I run: for X in `seq 1 4`; do lxc init ubuntu: test$X & done LXD creates the containers but it can't start them. This is the output of 'lxc info --show-log test1': https://pastebin.canonical.com/183666/ If I run: for X in `seq 1 4