Bugzilla – Bug 1172894
Delay to unlock fully encrypted partition
Last modified: 2023-10-02 10:11:33 UTC
On my system with fully encrypted partitions the LUKS encrypted data partition is mounted via crypttab using an encryption key. This has been very fast, needing very little time to unlock during boot. Since a day or two it is suddenly very slow, taking several seconds. Testing with cryptsetup it gives a longer delay than normal. When I boot another install with Leap that uses the same data partition, unlock takes still place much much faster. What has changed in cryptsetup or other component that makes it so slow?
I am noticing the same thing. But I am not sure that it is real. Yes, this started at the same time as an update to cryptsetup. Previously, after giving the LUKS key, I would see messages about cryptography setup finished. Instead, I now see a period of time with no apparent activity. However, if I watch disk activity instead of watching the screen, then I still see activity. I have not tried timing it to compare. But most of the change that I am seeing is with screen activity rather than disk activity. Something is being less verbose than before. I no longer see the message that cryptography setup has finished. I think that message came from a systemd unit, rather than from "cryptsetup". But the update to "cryptsetup" might have also updated the associated systemd unit.
I am sure it is much slower, before it would take less than a second to mount, after the update it took up till 8 seconds. I was able to bring it down by moving the slot with the keyfile to slot 0, but still it takes 4 to 5 seconds. I tested on my other laptop with Tumbleweed which still hadn't received it's update to the later version. It booted fast. Run the update to the newer build, rebooted, and it hung for several seconds at de crypto stuff mounting the partition. I always have full terminal output when booting so I can always see what happens. Besides that, running systemd-analyze blame gives me the real time needed to mount the partition, it is on top needing the most time.
I am still disagreeing. I have a KVM virtual machine, with Tumbleweed and Leap 15.2 installed side by side. Both systems use the same encrypted LVM, but different logical volumes within that LVM. I booted Tumbleweed. The command line does not include "splash=silent", so I normally see activity during boot. I booted. When I got the prompt for the LUKS key, I typed it in. I closed my eyes before hitting enter on that LUKS key, so that I would not be distracted by the output on screen. To check the time, I monitored my pulse. It took around 1.5 heart beats before I could hear disk activity. I then did the same with booting the same VM to Leap 15.2. In that case, I normally do have "splash=silent". So this time, I edited the grub command line to remove "splash=silent" so that it would be equivalent. Again, I typed in the key, then closed my eyes before hitting enter. Monitoring my pulse, it took arouhd 1.5 heart beats before I could hear disk activity. After that first boot (of Tumbleweed), I checked the output of "systemd-analyze blame". It showed 18.32 seconds for the cryptography. I then rebooted, and repeated the same. But this time, I was deliberately slow in typing in the LUKS key and hitting enter. And this time, "systemd-analyze blame" gave me 40 seconds for the crypto. So the time given by "systemd-analyze blame" includes human typing time, so it is not a useful measure of what is reported in this bug. I should have mentioned that the VM disk image is on a somewhat noisy disk, so that listening for disk activity noise works pretty well. On my physical computers, I have to look at the disk activity light, and that's harder to do. I do actually think there's a bug. There used to be output indicating completion of the crypto. And now there isn't. I don't like that. After entering the LUKS key, when I don't see immediate output, I wonder whether something is broken or I forgot to hit ENTER. I would like to get that line of output back. But the actual cryptography speed doesn't seem to be any slower than it was.
And I disagree with what you write. First of all, I think, but may be mistaken, that aes_ni hardware encryption support doesn't work in a virtual machine so is hard to compare. Second, I don't have to type any password since the unlock key is stored on the already unlocked root partition, so there is no delay. I suspect the cryptsetup binary was compiled without aes_ni support. I think that because of the following: I have a fully encrypted system, that includes /boot, so only after grub has decrypted it is there access to the kernel. Decryption in grub is slow because it somehow doesn't have hardware support, i.e. aes_ni. The second stage of the boot is that the system will have to mount the root partition, again. This is done by a key stored in initrd, which was stored on the encrypted /boot. Then during the boot process my fstab devices are mounted, among them this encrypted data partition, which is also done by a key. I know that even the decryption of the root partition is slower than normal but didn't mention that because I can not really measure that because it always decrypts faster than the data partition, maybe due to it's smaller size, so that decryption is now done in around a second or less. I know there is a huge difference because the first time it happened I thought the system had locked up during the decryption fase and then after a few seconds it continued. When I started changing the order of the key slots with cryptsetup it also took me much longer than before so I think the culprit is cryptsetup, which also happened to be updated just before this.
Same issue appears in Leap 15.3. Workaround: create new key slot with lower iter-time and remove old key slot; increase passphrase complexity as needed to compensate for lower expected time for brute force attack to succeed
Same problem on Laptop Thinkpad T440p i7-4710MQ. Grub LUKS unlocking takes about 26 seconds due to to many key iterations. Originaly it was ~1.9M I changed it to 80K with --iter-time 100. sudo cryptsetup luksAddKey --iter-time 100 --key-slot 0 /dev/sdaX
@Marcus, could this be also related to the merge of the hmac files for openssl in bsc#1185116?
Note that this bug is from 2020 and 2021, so it can not be related. The standard delay is 2 seconds. So cryptsetup measures PBKDF2 passphrase during creation of the encrypted volume, so that it takes around 2 seconds to unlock the passphrase.
(In reply to Marcus Meissner from comment #8) > Note that this bug is from 2020 and 2021, so it can not be related. Right, I overlooked that minor detail ;) Thanks! > The standard delay is 2 seconds. > > So cryptsetup measures PBKDF2 passphrase during creation of the encrypted > volume, so that it takes around 2 seconds to unlock the passphrase. @anyone, is this still an issue with the latest versions?
I'll have an opportunity to test this with Leap 15.5 on a real machine in September that had the issue before with Leap 15.3. (For 15.4, I booted the rescue system and created all my luks containers manually with iter-time 20ms, after discarding all blocks) There is no point in testing this on a machine that is not known to have had this issue before, e.g. if the machine has no AES acceleration of course the iteration time will be the same at installation and when booting. Same applies to VMs. If somebody has time to test it in a VM, first find the necessary VM CPU feature settings to reproduce the problem with Leap 15.3. Only then test with 15.5 and/or current Tumbleweed using the same settings.
Confirmed with Leap 15.5 on a 2021 PC with AMD Ryzen 9 CPU. 22 seconds from Grub's "Attempting to decrypt master key..." to the openSUSE menu. Adding a key slot with iter-time 4000 and pbkdf2 and removing the old key slot doubles the delay to 44 seconds, confirming the issue is with the time to open the key slot. Repeating this with iter-time 60 reduces the delay to below one second.