I am arriving here as I would also like to run containers (docker) over a zpool as the storage backend. In my case, I am running Ubuntu Server (Focal) on a Pi 3B, I would ideally have root on zfs but with it being a Pi, I have not done this yet. I have an externally attached USB pool for real data and have left the OS as it is on the SD card, however, as the above says, I would like to move more of my systems towards ZFS with time and use Ubuntu-based distros and containers on a lot of them, where these can't be run under illumos as VMs.
I have installed docker-rootless on this system, however due to kernel limitations, it is impossible to mount datasets as non-root on linux (long-standing issue, I am working on a bug report with details of options for a workaround and will attach), and using a noverlay over the top of zfs would be fine for me, as I can snapshot and work with the higher-level system and back up easily, even if not as fast as native zfs performance, however, the daemon will not start, and I have fallen back to the "vfs" driver on my data-root filesystem. Some googling hasn't indicated what feature(s) would be required in zfs for overlay to be used on top, docker's documentation doesn't seem to clarify anything, and searching for the "fstype=1" requirement in xfs, I find from the manual page for `mkfs.xfs`: ``` ftype=value This feature allows the inode type to be stored in the directory structure so that the readdir(3) and getdents(2) do not need to look up the inode to determine the inode type. ``` which perhaps is not the issue at hand. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to zfs-linux in Ubuntu. https://bugs.launchpad.net/bugs/1718761 Title: It's not possible to use OverlayFS (mount -t overlay) to stack directories on a ZFS volume Status in zfs-linux package in Ubuntu: Confirmed Bug description: ---------- Configuration # echo -e $(grep VERSION= /etc/os-release)\\nSIGNATURE=\"$(cat /proc/version_signature)\" VERSION="16.04.3 LTS (Xenial Xerus)" SIGNATURE="Ubuntu 4.10.0-35.39~16.04.1-generic 4.10.17" # dpkg --list | grep zfs ii libzfs2linux 0.6.5.6-0ubuntu18 ii zfs-doc 0.6.5.6-0ubuntu18 ii zfs-initramfs 0.6.5.6-0ubuntu18 ii zfs-zed 0.6.5.6-0ubuntu18 ii zfsutils-linux 0.6.5.6-0ubuntu18 ---------- Fault: Creating an overlay of multiple directories on a ZFS volume does not work # df /var/tmp Filesystem Type 1K-blocks Used Available Use% Mounted on tank07/var/tmp zfs 129916288 128 129916160 1% /var/tmp # mkdir /var/tmp/{lower,middle,upper,workdir,overlay} # mount -t overlay overlay -olowerdir=/var/tmp/middle:/var/tmp/lower,upperdir=/var/tmp/upper,workdir=/var/tmp/workdir /var/tmp/overlay mount: wrong fs type, bad option, bad superblock on overlay, missing codepage or helper program, or other error In some cases useful info is found in syslog - try dmesg | tail or so. # dmesg|tail -1 [276328.438284] overlayfs: filesystem on '/var/tmp/upper' not supported as upperdir ---------- Control test 1: Creating an overlay of multiple directories on another filesystem works # df /tmp Filesystem Type 1K-blocks Used Available Use% Mounted on tmpfs tmpfs 1048576 133492 915084 13% /tmp # mkdir /tmp/{lower,middle,upper,workdir,overlay} # mount -t overlay overlay -olowerdir=/tmp/middle:/tmp/lower,upperdir=/tmp/upper,workdir=/tmp/workdir /tmp/overlay # mount | grep overlay overlay on /tmp/overlay type overlay (rw,relatime,lowerdir=/tmp/middle:/tmp/lower,upperdir=/tmp/upper,workdir=/tmp/workdir) ---------- Control test 2: Creating an overlay using AuFS works on ZFS volume and elsewhere # mount -t aufs -obr=/tmp/lower:/tmp/middle:/tmp/upper:/tmp/workdir none /tmp/overlay # mount -t aufs -obr=/var/tmp/lower:/var/tmp/middle:/var/tmp/upper:/var/tmp/workdir none /var/tmp/overlay # mount | grep aufs none on /var/tmp/overlay type aufs (rw,relatime,si=9ead1ecae778b250) none on /tmp/overlay type aufs (rw,relatime,si=9ead1ec9257d1250) ---------- Remark While the use of AuFS can be used as a workaround in the above scenario, AuFS in turn will not work with [fuse.]glusterfs mounts (this has been documented elsewhere). Given that OverlayFS is part of the (upstream) kernel and Ubuntu now officially supports ZFS, the above should be fixed. --- AlsaDevices: total 0 crw-rw---- 1 root audio 116, 1 Sep 18 16:12 seq crw-rw---- 1 root audio 116, 33 Sep 18 16:12 timer AplayDevices: Error: [Errno 2] No such file or directory ApportVersion: 2.20.1-0ubuntu2.10 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: DistroRelease: Ubuntu 16.04 IwConfig: Error: [Errno 2] No such file or directory Lspci: Error: [Errno 2] No such file or directory Lsusb: Error: [Errno 2] No such file or directory MachineType: Red Hat KVM NonfreeKernelModules: zfs zunicode zavl zcommon znvpair Package: linux (not installed) PciMultimedia: ProcFB: 0 cirrusdrmfb ProcKernelCmdLine: BOOT_IMAGE=/ROOT/ubuntu1604@/boot/vmlinuz-4.10.0-35-generic root=ZFS=tank07/ROOT/ubuntu1604 ro ProcVersionSignature: Ubuntu 4.10.0-35.39~16.04.1-generic 4.10.17 RelatedPackageVersions: linux-restricted-modules-4.10.0-35-generic N/A linux-backports-modules-4.10.0-35-generic N/A linux-firmware 1.157.12 RfKill: Error: [Errno 2] No such file or directory Tags: xenial Uname: Linux 4.10.0-35-generic x86_64 UpgradeStatus: No upgrade log present (probably fresh install) UserGroups: _MarkForUpload: True dmi.bios.date: 04/01/2014 dmi.bios.vendor: SeaBIOS dmi.bios.version: 1.9.1-5.el7_3.3 dmi.chassis.type: 1 dmi.chassis.vendor: Red Hat dmi.chassis.version: RHEL 7.3.0 PC (i440FX + PIIX, 1996) dmi.modalias: dmi:bvnSeaBIOS:bvr1.9.1-5.el7_3.3:bd04/01/2014:svnRedHat:pnKVM:pvrRHEL7.3.0PC(i440FX+PIIX,1996):cvnRedHat:ct1:cvrRHEL7.3.0PC(i440FX+PIIX,1996): dmi.product.name: KVM dmi.product.version: RHEL 7.3.0 PC (i440FX + PIIX, 1996) dmi.sys.vendor: Red Hat --- AlsaDevices: total 0 crw-rw---- 1 root audio 116, 1 Sep 18 16:12 seq crw-rw---- 1 root audio 116, 33 Sep 18 16:12 timer AplayDevices: Error: [Errno 2] No such file or directory ApportVersion: 2.20.1-0ubuntu2.10 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: DistroRelease: Ubuntu 16.04 IwConfig: Error: [Errno 2] No such file or directory Lspci: Error: [Errno 2] No such file or directory Lsusb: Error: [Errno 2] No such file or directory MachineType: Red Hat KVM NonfreeKernelModules: zfs zunicode zavl zcommon znvpair Package: linux (not installed) PciMultimedia: ProcFB: 0 cirrusdrmfb ProcKernelCmdLine: BOOT_IMAGE=/ROOT/ubuntu1604@/boot/vmlinuz-4.10.0-35-generic root=ZFS=tank07/ROOT/ubuntu1604 ro ProcVersionSignature: Ubuntu 4.10.0-35.39~16.04.1-generic 4.10.17 RelatedPackageVersions: linux-restricted-modules-4.10.0-35-generic N/A linux-backports-modules-4.10.0-35-generic N/A linux-firmware 1.157.12 RfKill: Error: [Errno 2] No such file or directory Tags: xenial Uname: Linux 4.10.0-35-generic x86_64 UpgradeStatus: No upgrade log present (probably fresh install) UserGroups: _MarkForUpload: True dmi.bios.date: 04/01/2014 dmi.bios.vendor: SeaBIOS dmi.bios.version: 1.9.1-5.el7_3.3 dmi.chassis.type: 1 dmi.chassis.vendor: Red Hat dmi.chassis.version: RHEL 7.3.0 PC (i440FX + PIIX, 1996) dmi.modalias: dmi:bvnSeaBIOS:bvr1.9.1-5.el7_3.3:bd04/01/2014:svnRedHat:pnKVM:pvrRHEL7.3.0PC(i440FX+PIIX,1996):cvnRedHat:ct1:cvrRHEL7.3.0PC(i440FX+PIIX,1996): dmi.product.name: KVM dmi.product.version: RHEL 7.3.0 PC (i440FX + PIIX, 1996) dmi.sys.vendor: Red Hat To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/zfs-linux/+bug/1718761/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp