Le mardi 26 janvier 2021 à 00:59:52+0100, Andras Korn a écrit :
> Hi,
>
> I hit the same issue.
>
> I upgraded from 1:4.0.4-6 to 1:4.0.5-2, and from kernel 5.9.0-4-amd64 to
> 5.10.0-2-amd64, and some of my containers that used to work before don't work
> anyomre. The ones that still work don't drop sys_admin.
>
> stracing lxc-start I see this:
>
> openat2(33</usr/lib/x86_64-linux-gnu/lxc/rootfs>, "/sys/fs/cgroup",
> {flags=O_RDONLY|O_CLOEXEC|O_PATH,
> resolve=RESOLVE_NO_XDEV|RESOLVE_NO_MAGICLINKS|RESOLVE_NO_SYMLINKS|RESOLVE_BENEATH},
> 24) = -1 EXDEV (Invalid cross-device link)
>
> The corresponding message from lxc-start with loglevel debug is:
>
> lxc-start unifiadmin 20210125231743.129 ERROR conf -
> conf.c:lxc_mount_auto_mounts:727 - Invalid cross-device link - Failed to
> mount "/sys/fs/cgroup"
>
> Some context from lxc-start log output:
>
> lxc-start unifiadmin 20210125231742.854 INFO start - start.c:lxc_init:837
> - Container "unifiadmin" is initialized
> lxc-start unifiadmin 20210125231742.876 WARN cgfsng -
> cgroups/cgfsng.c:mkdir_eexist_on_last:1152 - File exists - Failed to create
> directory "/sys/fs/cgroup/cpuset//lxc.monitor.unifiadmin"
> lxc-start unifiadmin 20210125231742.886 INFO cgfsng -
> cgroups/cgfsng.c:cgfsng_monitor_create:1368 - The monitor process uses
> "lxc.monitor.unifiadmin" as cgroup
> lxc-start unifiadmin 20210125231742.904 WARN cgfsng -
> cgroups/cgfsng.c:mkdir_eexist_on_last:1152 - File exists - Failed to create
> directory "/sys/fs/cgroup/cpuset//lxc.payload.unifiadmin"
> lxc-start unifiadmin 20210125231742.916 INFO cgfsng -
> cgroups/cgfsng.c:cgfsng_payload_create:1471 - The container process uses
> "lxc.payload.unifiadmin" as cgroup
> lxc-start unifiadmin 20210125231742.944 INFO start -
> start.c:lxc_spawn:1700 - Cloned CLONE_NEWNS
> lxc-start unifiadmin 20210125231742.944 INFO start -
> start.c:lxc_spawn:1700 - Cloned CLONE_NEWPID
> lxc-start unifiadmin 20210125231742.945 INFO start -
> start.c:lxc_spawn:1700 - Cloned CLONE_NEWUTS
> lxc-start unifiadmin 20210125231742.945 INFO start -
> start.c:lxc_spawn:1700 - Cloned CLONE_NEWIPC
> lxc-start unifiadmin 20210125231742.945 INFO start -
> start.c:lxc_spawn:1700 - Cloned CLONE_NEWNET
> lxc-start unifiadmin 20210125231742.945 DEBUG start -
> start.c:lxc_try_preserve_namespaces:166 - Preserved mnt namespace via fd 31
> lxc-start unifiadmin 20210125231742.945 DEBUG start -
> start.c:lxc_try_preserve_namespaces:166 - Preserved pid namespace via fd 32
> lxc-start unifiadmin 20210125231742.946 DEBUG start -
> start.c:lxc_try_preserve_namespaces:166 - Preserved uts namespace via fd 33
> lxc-start unifiadmin 20210125231742.946 DEBUG start -
> start.c:lxc_try_preserve_namespaces:166 - Preserved ipc namespace via fd 34
> lxc-start unifiadmin 20210125231742.946 DEBUG start -
> start.c:lxc_try_preserve_namespaces:166 - Preserved net namespace via fd 35
> lxc-start unifiadmin 20210125231742.949 INFO cgfsng -
> cgroups/cgfsng.c:cgfsng_setup_limits_legacy:2881 - Limits for the legacy
> cgroup hierarchies have been setup
> lxc-start unifiadmin 20210125231742.955 WARN cgfsng -
> cgroups/cgfsng.c:cgfsng_setup_limits:2942 - Invalid argument - Ignoring
> cgroup2 limits on legacy cgroup system
> lxc-start unifiadmin 20210125231743.315 INFO network -
> network.c:instantiate_veth:285 - Retrieved mtu 1500 from intra
> lxc-start unifiadmin 20210125231743.666 INFO network -
> network.c:instantiate_veth:333 - Attached "veth-unifi" to bridge "intra"
> lxc-start unifiadmin 20210125231743.687 DEBUG network -
> network.c:instantiate_veth:449 - Instantiated veth tunnel "veth-unifi <-->
> vethv7jzuF"
> lxc-start unifiadmin 20210125231743.699 WARN start -
> start.c:do_start:1166 - Using /dev/null from the host for container init's
> standard file descriptors. Migration will not work
> lxc-start unifiadmin 20210125231743.704 INFO start -
> start.c:do_start:1198 - Unshared CLONE_NEWCGROUP
> lxc-start unifiadmin 20210125231743.731 DEBUG storage -
> storage/storage.c:get_storage_by_name:211 - Detected rootfs type "dir"
> lxc-start unifiadmin 20210125231743.734 DEBUG conf -
> conf.c:lxc_mount_rootfs:1259 - Mounted rootfs
> "/var/lib/lxc/unifiadmin/rootfs" onto "/usr/lib/x86_64-linux-gnu/lxc/rootfs"
> with options "(null)"
> lxc-start unifiadmin 20210125231743.738 INFO conf -
> conf.c:setup_utsname:751 - Set hostname to "unifiadmin"
> lxc-start unifiadmin 20210125231743.740 DEBUG network -
> network.c:lxc_network_setup_in_child_namespaces_common:3510 - Network device
> "" has been setup
> lxc-start unifiadmin 20210125231743.977 DEBUG network -
> network.c:setup_hw_addr:3360 - Mac address "00:16:3e:11:22:33" on "eth0" has
> been setup
> lxc-start unifiadmin 20210125231743.103 DEBUG network -
> network.c:lxc_network_setup_in_child_namespaces_common:3510 - Network device
> "eth0" has been setup
> lxc-start unifiadmin 20210125231743.103 INFO network -
> network.c:lxc_setup_network_in_child_namespaces:3532 - Network has been setup
> lxc-start unifiadmin 20210125231743.116 DEBUG conf -
> conf.c:mount_entry:1943 - Remounting "/shared/cache/apt/lists" on
> "/usr/lib/x86_64-linux-gnu/lxc/rootfs/var/lib/apt/lists" to respect bind or
> remount options
> lxc-start unifiadmin 20210125231743.116 DEBUG conf -
> conf.c:mount_entry:1962 - Flags for "/shared/cache/apt/lists" were 1038,
> required extra flags are 14
> lxc-start unifiadmin 20210125231743.117 DEBUG conf -
> conf.c:mount_entry:1971 - Mountflags already were 5134, skipping remount
> lxc-start unifiadmin 20210125231743.117 DEBUG conf -
> conf.c:mount_entry:2006 - Mounted "/shared/cache/apt/lists" on
> "/usr/lib/x86_64-linux-gnu/lxc/rootfs/var/lib/apt/lists" with filesystem type
> "none"
> lxc-start unifiadmin 20210125231743.118 DEBUG conf -
> conf.c:mount_entry:1943 - Remounting "/shared/cache/apt/archives" on
> "/usr/lib/x86_64-linux-gnu/lxc/rootfs/var/cache/apt/archives" to respect bind
> or remount options
> lxc-start unifiadmin 20210125231743.118 DEBUG conf -
> conf.c:mount_entry:1962 - Flags for "/shared/cache/apt/archives" were 1038,
> required extra flags are 14
> lxc-start unifiadmin 20210125231743.118 DEBUG conf -
> conf.c:mount_entry:1971 - Mountflags already were 5134, skipping remount
> lxc-start unifiadmin 20210125231743.118 DEBUG conf -
> conf.c:mount_entry:2006 - Mounted "/shared/cache/apt/archives" on
> "/usr/lib/x86_64-linux-gnu/lxc/rootfs/var/cache/apt/archives" with filesystem
> type "none"
> lxc-start unifiadmin 20210125231743.119 DEBUG conf -
> conf.c:mount_entry:1943 - Remounting
> "/usr/lib/x86_64-linux-gnu/lxc/rootfs/dev/null" on
> "/usr/lib/x86_64-linux-gnu/lxc/rootfs/proc/kcore" to respect bind or remount
> options
> lxc-start unifiadmin 20210125231743.119 DEBUG conf -
> conf.c:mount_entry:1962 - Flags for
> "/usr/lib/x86_64-linux-gnu/lxc/rootfs/dev/null" were 1024, required extra
> flags are 0
> lxc-start unifiadmin 20210125231743.120 DEBUG conf -
> conf.c:mount_entry:1971 - Mountflags already were 4096, skipping remount
> lxc-start unifiadmin 20210125231743.120 DEBUG conf -
> conf.c:mount_entry:2006 - Mounted
> "/usr/lib/x86_64-linux-gnu/lxc/rootfs/dev/null" on
> "/usr/lib/x86_64-linux-gnu/lxc/rootfs/proc/kcore" with filesystem type "none"
> lxc-start unifiadmin 20210125231743.123 DEBUG conf -
> conf.c:mount_entry:1943 - Remounting "/sys/fs/fuse/connections" on
> "/usr/lib/x86_64-linux-gnu/lxc/rootfs/sys/fs/fuse/connections" to respect
> bind or remount options
> lxc-start unifiadmin 20210125231743.123 DEBUG conf -
> conf.c:mount_entry:1962 - Flags for "/sys/fs/fuse/connections" were 4110,
> required extra flags are 14
> lxc-start unifiadmin 20210125231743.123 DEBUG conf -
> conf.c:mount_entry:2006 - Mounted "/sys/fs/fuse/connections" on
> "/usr/lib/x86_64-linux-gnu/lxc/rootfs/sys/fs/fuse/connections" with
> filesystem type "none"
> lxc-start unifiadmin 20210125231743.125 DEBUG conf -
> conf.c:mount_entry:1943 - Remounting "/sys/fs/fuse/connections" on
> "/usr/lib/x86_64-linux-gnu/lxc/rootfs/sys/fs/fuse/connections" to respect
> bind or remount options
> lxc-start unifiadmin 20210125231743.125 DEBUG conf -
> conf.c:mount_entry:1962 - Flags for "/sys/fs/fuse/connections" were 4110,
> required extra flags are 14
> lxc-start unifiadmin 20210125231743.125 DEBUG conf -
> conf.c:mount_entry:2006 - Mounted "/sys/fs/fuse/connections" on
> "/usr/lib/x86_64-linux-gnu/lxc/rootfs/sys/fs/fuse/connections" with
> filesystem type "none"
> lxc-start unifiadmin 20210125231743.127 DEBUG conf -
> conf.c:mount_entry:2006 - Mounted "run" on
> "/usr/lib/x86_64-linux-gnu/lxc/rootfs/run" with filesystem type "tmpfs"
> lxc-start unifiadmin 20210125231743.128 DEBUG conf -
> conf.c:mount_entry:2006 - Mounted "none" on
> "/usr/lib/x86_64-linux-gnu/lxc/rootfs/dev/shm" with filesystem type "tmpfs"
> lxc-start unifiadmin 20210125231743.129 ERROR conf -
> conf.c:lxc_mount_auto_mounts:727 - Invalid cross-device link - Failed to
> mount "/sys/fs/cgroup"
> lxc-start unifiadmin 20210125231743.130 ERROR conf - conf.c:lxc_setup:3365
> - Failed to setup remaining automatic mounts
> lxc-start unifiadmin 20210125231743.130 ERROR start -
> start.c:do_start:1218 - Failed to setup container "unifiadmin"
> lxc-start unifiadmin 20210125231743.131 ERROR sync - sync.c:__sync_wait:36
> - An error occurred in another process (expected sequence number 5)
> lxc-start unifiadmin 20210125231743.132 DEBUG network -
> network.c:lxc_delete_network:3665 - Deleted network devices
> lxc-start unifiadmin 20210125231743.133 ERROR lxccontainer -
> lxccontainer.c:wait_on_daemonized_start:859 - Received container state
> "ABORTING" instead of "RUNNING"
> lxc-start unifiadmin 20210125231743.134 ERROR lxc_start -
> tools/lxc_start.c:main:308 - The container failed to start
> lxc-start unifiadmin 20210125231743.135 ERROR lxc_start -
> tools/lxc_start.c:main:311 - To get more details, run the container in
> foreground mode
> lxc-start unifiadmin 20210125231743.135 ERROR start -
> start.c:__lxc_start:1999 - Failed to spawn container "unifiadmin"
> lxc-start unifiadmin 20210125231743.136 ERROR lxc_start -
> tools/lxc_start.c:main:313 - Additional information can be obtained by
> setting the --logfile and --logpriority options
> lxc-start unifiadmin 20210125231743.136 WARN start -
> start.c:lxc_abort:1012 - No such process - Failed to send SIGKILL via pidfd
> 30 for process 15227
> lxc-start unifiadmin 20210125231743.748 INFO conf -
> conf.c:run_script_argv:342 - Executing script
> "/usr/share/lxcfs/lxc.reboot.hook" for container "unifiadmin"
> lxc-start unifiadmin 20210125231744.288 INFO conf -
> conf.c:run_script_argv:342 - Executing script
> "/usr/share/lxcfs/lxc.reboot.hook" for container "unifiadmin"
>
> If I don't drop the sys_admin capability, it works again.
>
> Before the upgrade, it also worked if I dropped sys_admin.
>
> The configfile for this guest is:
>
> ----- 8< -----
> lxc.include = /usr/share/lxc/config/common.conf
>
> lxc.apparmor.profile = generated
> lxc.apparmor.allow_nesting = 0
> lxc.hook.version = 1
>
> lxc.mount.entry = run run tmpfs
> rw,nodev,relatime,mode=755,size=20m,create=dir 0 0
> lxc.mount.entry = none dev/shm tmpfs
> rw,nosuid,nodev,mode=1777,size=100m,create=dir 0 0
> lxc.cap.drop = sys_resource audit_write block_suspend linux_immutable
> mac_admin mac_override sys_admin sys_module sys_pacct sys_rawio sys_resource
> sys_time sys_tty_config syslog
> lxc.start.auto = 1
> lxc.cgroup.devices.deny = a
> lxc.cgroup.devices.allow = c 1:3 rwm
> lxc.cgroup.devices.allow = c 1:5 rwm
> lxc.cgroup.devices.allow = c 1:7 rwm
> lxc.cgroup.devices.allow = c 1:8 rwm
> lxc.cgroup.devices.allow = c 1:9 rwm
> lxc.cgroup.devices.allow = c 5:0 rwm
> lxc.cgroup.devices.allow = c 5:1 rwm
> lxc.cgroup.devices.allow = c 5:2 rwm
> lxc.cgroup.devices.allow = c 136:* rwm
> lxc.cgroup.devices.allow = c 10:229 rwm
> lxc.cgroup.devices.allow = c 254:0 rm
> lxc.cgroup.devices.allow = c 10:200 rwm
> lxc.cgroup.devices.allow = c 10:228 rwm
> lxc.cgroup.devices.allow = c 10:232 rwm
>
> lxc.autodev = 0
> lxc.tty.dir =
> lxc.tty.max = 0
>
> # Container specific configuration
> lxc.rootfs.path = dir:/var/lib/lxc/unifiadmin/rootfs
> lxc.uts.name = unifiadmin
> lxc.arch = amd64
>
> # Network configuration
> lxc.net.0.type = empty
> lxc.net.1.type = veth
> lxc.net.1.link = intra
> lxc.net.1.flags = up
> lxc.net.1.name = eth0
> lxc.net.1.veth.pair = veth-unifi
> lxc.net.1.hwaddr = 00:16:3e:11:22:33
> lxc.mount.fstab = /var/lib/lxc/unifiadmin/fstab
> ----- >8 -----
>
> The fstab doesn't reference cgroup or /sys.
>
> I googled around and found this post from 2012:
> https://lists.linuxfoundation.org/pipermail/containers/2012-November/030827.html
> -- based on this, maybe the problem is that cap_sys_admin is dropped too
> early now?
>
> Also, https://github.com/lxc/lxc/issues/1737 looks related.
> https://blog.iwakd.de/lxc-cap_sys_admin-jessie also suggests that running
> containers without cap_sys_admin used to be possible.
>
> Or maybe I should be using cgroup2?
>
> FWIW, both host and guest use runit, so systemd is not involved; runit
> doesn't interfere with cgroups or capabilities on its own in any way.It's not possible if the init in the container is systemd, as far as I know. I can't see how this can be dealt with without systemd's devs trying to make it a bit less capabilities-demanding. -- Pierre-Elliott Bécue GPG: 9AE0 4D98 6400 E3B6 7528 F493 0D44 2664 1949 74E2 It's far easier to fight for principles than to live up to them.
signature.asc
Description: PGP signature
