** Description changed: [Impact] - * Considering a setup of a encrypted rootfs on top of md RAID1 device, Ubuntu is currently unable to decrypt the rootfs if the array gets degraded, like for example if one of the array members gets removed. + * Considering a setup of a encrypted rootfs on top of md RAID1 device, Ubuntu is currently unable to decrypt the rootfs if the array gets degraded, like for example if one of the array's members gets removed. * The problem has 2 main aspects: first, cryptsetup initramfs script attempts to decrypt the array only in the local-top boot stage, and in case it fails, it gives-up and show user a shell (boot is aborted). * Second, mdadm initramfs script that assembles degraded arrays executes later on boot, in the local-block stage. So, in a stacked setup of encrypted root on top of RAID, if the RAID is degraded, cryptsetup fails early in the boot, preventing mdadm to assemble the degraded array. - [Proposed solution] - * The solution hereby proposed has 3 components: first, cryptsetup script is modified to allow a gentle failure on local-top stage, then it retries for a while in a later stage (local-block). This gives time to other initramfs scripts to run, like mdadm in local-block stage. + * The hereby proposed solution has 2 components: first, cryptsetup + script is modified to allow a gentle failure on local-top stage, then it + retries for a while (according to a heuristic based on ROOTDELAY with + minimum of 30 executions) in a later stage (local-block). This gives + time to other initramfs scripts to run, like mdadm in local-block stage. + And this is meant to work this way according to initramfs-tools + documentation (although Ubuntu changed it a bit with wait-for-root, + hence we stopped looping on local-block, see next bullet). - * Second, mdadm script was adjusted - currently, it retries for a while - to assemble the arrays in a non-degraded mode, by waiting for udev to - discover the missing devices. After some time, it gives-up and assembles - all arrays as degraded. The adjust we made was only to reduce this - number of attempts and fallback a bit faster to degraded array assembly. - - * Third, there's a difference in Ubuntu initramfs-tools compared to - Debian's : in Ubuntu, we rely on wait-for-root, a binary meant to speed- - up the boot process. Problem is that currently this approach prevents - local-block scripts to run more than once (thus retrying mechanisms will - fail). Our proposed solution changes initramfs-tools to allow some - looping in local-block stage (as Debian), but still relying on wait-for- - root. Also, we increased a bit the default rootdelay from 30 seconds to - 60 seconds. - + * Second, initramfs-tools was adjusted - currently, it runs for a while + the mdadm local-block script, in order to assemble the arrays in a non- + degraded mode. We extended this approach to also execute cryptsetup, in + a way that after mdadm ends its execution, we execute at least once more + time cryptsetup. In an ideal world we should loop on local-block as + Debian's initramfs (in a way to remove hardcoded mdadm/cryptsetup + mentions from initramfs-tools code), but this would be really a big + change, non-SRUable probably. I plan to work that for future Ubuntu + releases. [Test case] * Install Ubuntu in a Virtual Machine with 2 disks. Use the installer to create a RAID1 volume and an encrypted root on top of it. * Boot the VM, and use "sgdisk"/"wipefs" to erase the partition table from one of the RAID members. Reboot and it will fail to mount rootfs and continue boot process. + * If using the initramfs-toos/cryptsetup patches hereby proposed, the + rootfs can be mounted normally. [Regression potential] - * There are potential for regressions, since this is a change in 3 boot + * There are potential for regressions, since this is a change in 2 boot components. The patches were designed in a way to keep the regular case working, it changes the failure case which is not currently working anyway. - * A potential issue in the initramfs-tools change is a bad script in - local-block, lacking a good "halt" mechanism, to induce a boot loop - condition. This problem would be exposed by the hereby proposed - modification. + * A modification in the behavior of cryptsetup was introduced: right + now, if we fail the password 3 times (the default maximum attempts), the + script doesn't "panic" and drop to a shell immediately; instead it runs + once more (or twice, if mdadm is installed) before failing. This is a + minor change given the benefit of the being able to mount rootfs in a + degraded RAID1 scenario. + + * Other potential regressions could show-up as boot problems, but the + change in initramfs-tools specifically is not invasive, it just may + delay boot time a bit, given we now run cryptsetup multiple times on + local-block, with 1 sec delays between executions.
-- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1879980 Title: Fail to boot with LUKS on top of RAID1 if the array is broken/degraded To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cryptsetup/+bug/1879980/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
