Control: severity -1 important Control: clone -1 -2 Control: reassign -2 systemd 230-7 Control: affects -1 cryptsetup Control: affects -2 open-iscsi cryptsetup Control: retitle -1 open-iscsi: No support for disabling LUKS volumes on shutdown Control: retitle -2 systemd: cryptsetup-generator does not support LUKS on network devices Control: tags -1 + confirmed stretch sid Control: tags -2 + upstream
First of all, sorry for the late reply, but I was ill for a while and then I was quite busy with catching up with other stuff. The immediate bug you're experiencing is a problem with systemd, so I am cloning the bug report and reassigning it to the systemd package. However, in reviewing your setup I've also noticed that there is a separate problem for your kind of setup in open-iscsi. I'm including the main part of your bug report for the systemd maintainers' benefit. > This is how I setup my volume: > > # The iSCSI part works fine. > iscsiadm … --login > > # Device shows up as /dev/sdb; I create a /dev/sdb1 partition using > # fdisk, of type 8e. > > # Create encrypted LUKS volume on /dev/sdb1, open and map as > # /dev/mapper/sdb1_crypt. > cryptsetup luksFormat /dev/sdb1 > cryptsetup luksOpen /dev/sdb1 sdb1_crypt \ > --key-file /root/blackbird-ullu > > # Set up LVM PV, VG, and LV mapped to /dev/mapper/blackbird-ullu, > # with an ext4 filesystem on top. > pvcreate /dev/mapper/sdb1_crypt > vgcreate blackbird /dev/mapper/sdb1_crypt > lvcreate -n ullu -l 100%VG blackbird > mkfs.ext4 /dev/mapper/blackbird-ullu > > mount /dev/mapper/blackbird-ullu /media/nas > > I have an entry in /etc/crypttab like this: > > sdb1_crypt UUID=ae6b9263-d63c-4515-b7ce-51e5cc4caa9f /root/blackbird-ullu > luks > > And an entry in /etc/fstab like this (I've tried various variants here, > see below): > > /dev/mapper/blackbird-ullu /media/nas ext4 defaults,nofail,_netdev 0 6 > > There are three devices involved: > > - /dev/disk/by-uuid/<x>: is the iSCSI target (/dev/sdb) > - /dev/mapper/sdb1_crypt: result of cryptsetup luksOpen /dev/sdb1 > - /dev/mapper/blackbird-ullu: LV built on top of sdb1_crypt > > Now I suffer from the 90s wait on startup (before network-online), where > systemd waits for the dev-mapper-blackbird\x2dullu.device to become > available, along with dev-disk-by\x2duuid-<xxx>.device and > dev-mapper-sdb1_crypt.device. The problem here is that the cryptsetup-generator in systemd only supports local block devices, and the dependencies it generates assume that everything orders indirectly before the local filesystems. This causes systemd to wait for the block device underlying the LUKS encryption _before_ open-iscsi is even started. This can clearly also be seen by the ordering cycle that's generated if you manually force the dependencies: > open-iscsi.service: Found ordering cycle on open-iscsi.service/start > open-iscsi.service: Found dependency on network-online.target/start > open-iscsi.service: Found dependency on networking.service/start > open-iscsi.service: Found dependency on local-fs.target/start > open-iscsi.service: Found dependency on lvm2-activation.service/start > open-iscsi.service: Found dependency on cryptsetup.target/start > open-iscsi.service: Found dependency on > systemd-cryptsetup@sdb1_crypt.service/start > open-iscsi.service: Found dependency on > dev-disk-by\x2duuid-ae6b9263\x2dd63c\x2d4515\x2db7ce\x2d51e5cc4caa9f.device/start > open-iscsi.service: Found dependency on open-iscsi.service/start Here you see that systemd-cryptsetup@sdb1_crypt.service depends on open-iscsi.service (because of your manual dependency), but that cryptsetup.target depends on systemd-cryptsetup@sdb1_crypt.service, local-fs.target depends on cryptsetup.target and open-iscsi.service (correctly) depends on local-fs.target. Your trick in lowering the device timeout to 1s instead of the explicit ordering makes this "work" because it causes systemd to immediately fail systemd-cryptsetup@sdb1_crypt.service immediately at early boot (before open-iscsi.service is started), and systemd will then start it again once the device actually appears after open-iscsi logs in to the target. Ideally one would want to have an option like _netdev also for /etc/crypttab, so that it's clear that this specific block device will only appear later. Unfortunately that doesn't exist yet. [1] So your workaround in explicitly lowering the device timeout (you can just do that for dev-disk-by\x2duuid-ae6b9263\x2dd63c\x2d4515\x2db7ce\x2d51e5cc4caa9f.device btw., you don't need to set it for other devices) is the only thing that will make this "work" currently though. As for the open-iscsi bug I also noticed: on shutdown open-iscsi tries to tear down all mounts on iSCSI, including layers such as LVM or multipath, but it doesn't detect LUKS yet, so data may not be written to disk properly once the iSCSI logout happens. I plan on doing an upload of open-iscsi today (to get some other stuff into Stretch), which will include a fix for the shutdown part (I just need to test it first). However it won't be possible to fix the shutdown issue in Jessie, because the code would be too invasive (the shutdown code was reworked for Stretch and is much better now), so you'll have to use the backport of open-iscsi for this (once that hits jessie-backports, which will take a while, because of the time it takes for packages to migrate to Stretch at the moment). I'll also try to work on a patch for the systemd issue in the next couple of days. Regards, Christian [1] You can see that the string _netdev clearly appears in http://sources.debian.net/src/systemd/232-14/src/fstab-generator/fstab-generator.c/ which parses /etc/fstab for systemd, but doesn't appear in http://sources.debian.net/src/systemd/232-14/src/cryptsetup/cryptsetup-generator.c/ which parses /etc/crypttab for systemd.
