Description of problem: System does not provide means to enable discards on LUKS encrypted root volume. Version-Release number of selected component (if applicable): 024-15.git20121218.fc18 How reproducible: Always Steps to Reproduce: 1. Install Fedora 18 with encryption & LVM on root partition 2. Add allow-discards option to first entry in /etc/crypttab 3. Reboot Actual results: 'dmsetup table' shows that discards are disabled Expected results: discards should be enabled Additional info: rd.luks.allow-discards has no effect
More info: I chose a minimal installation of Fedora 18. I have another encrypted volume, set up as a second entry in /etc/crypttab, with allow-discards too. This one is properly opened on boot-up with discards enabled. Only the first entry, for the partition containing the system's root, is experiencing the bug. I don't know if this is related, but I can also observe bug #879431.
Turns out that removing the rd.luks.uuid= argument from the linux command fixes this. After investigation, it seems to come from the logic within systemd-cryptsetup-generator. It first decrypts volumes specified in the linux command (with no knowledge of rd.luks.allow-discards), and only afterwards reads /etc/crypttab to decrypt volumes not already decrypted. Removing the rd.luks.uuid entry from the linux command makes it go straight to processing /etc/crypttab, forwarding the specified options correctly. By the way, and a bit off-topic, this also solves another issue: the timeout=0 option was also completely ignored, which resulted in a timeout at boot-up. This was reported as bug #868421 Perhaps to avoid confusion, it would make sense to have systemd-cryptsetup-generator apply any relevant options from /etc/crypttab when encountering an rd.luks.uuid entry from the linux command.
(In reply to comment #2) > By the way, and a bit off-topic, this also solves another issue: the > timeout=0 option was also completely ignored, which resulted in a timeout at > boot-up. > This was reported as bug #868421 Sorry, spoke too soon, timeout still occurs. And apologies for being off-topic.
I am pretty sure we should continue with giving kernel settings preference over configuration in /etc if kernel settings are specified. Merging configuration from different sources is always problematic, as the general expectation should be that the kernel cmdline overrides the same setting in /etc/crypttab. Or to turn this around, I think making the case for adding luks.allow-discards=1 as option on the kernel cmdline would make more sense... Are you asking for that?
I'm all for giving priority to kernel settings, but luks.allow-discards=1 is not even honored by systemd-cryptsetup-generator, which therefore mounts the encrypted volume without the allow-discards option. That's essentially what the problem comes down to.
Yeah, that was my question, if you'd like us to add support for luks.allow-discards=1. I guess the answer to that is yes, as I understand you...
I would find that useful, yes.
Ran into this today, having luks.allow-discards=1 supported would be very handy. Have simply removed the rd.luks.uuid argument as a workaround.
Lennart, systemd guys, can you please fix this soon? I have a new SSD drive, I need to use LUKS, and I simply can't get TRIM working. The workaround of removing rd.luks.uuid doesn't work for me. It might be related to the fact that I have two drives, two VGs, two LUKS partitions. Anyway, I have enabled discard everywhere and I still get "Operation not supported" from fstrim. dmsetup table doesn't show discards enabled. As a consequence I have a SSD drive that I cannot trim. My view is that if dracut manpage says "rd.luks.allow-discards" enables TRIM everywhere, it should, no matter what. If it is not specified, then crypttab configuration should take place. Please fix that in systemd. Technical detail: man dracut.kernel speaks of rd.luks.allow-discards=<luks uuid> or rd.luks.allow-discards. So the syntax of rd.luks.allow-discards=1 is not really correct.
(In reply to comment #9) > Lennart, systemd guys, can you please fix this soon? > > I have a new SSD drive, I need to use LUKS, and I simply can't get TRIM > working. The workaround of removing rd.luks.uuid doesn't work for me. It > might be related to the fact that I have two drives, two VGs, two LUKS > partitions. Anyway, I have enabled discard everywhere and I still get > "Operation not supported" from fstrim. dmsetup table doesn't show discards > enabled. As a consequence I have a SSD drive that I cannot trim. > > My view is that if dracut manpage says "rd.luks.allow-discards" enables TRIM > everywhere, it should, no matter what. If it is not specified, then crypttab > configuration should take place. Please fix that in systemd. > > Technical detail: man dracut.kernel speaks of rd.luks.allow-discards=<luks > uuid> or rd.luks.allow-discards. So the syntax of rd.luks.allow-discards=1 > is not really correct. why not configure that in your /etc/crypttab IIRC, /etc/crypttab is included in the initramfs. # lsinitrd /boot/initramfs-$(uname -r).img etc/crypttab
(In reply to comment #10) > why not configure that in your /etc/crypttab I have it configured in my crypttab (allow_discards). I have updated initramfs. This bug documents that crypttab configuration is not applied because of a problem in systemd-cryptsetup-generator.
(In reply to comment #10) > # lsinitrd /boot/initramfs-$(uname -r).img etc/crypttab Interesting, /etc/crypttab is *not* included in the initramfs. Shouldn't it be? Would it influence this bug somehow? Because it is not stored in initramfs, I can't influence any luks options regarding the root partition (because we need to decrypt it first to access the crypttab in the first place), without using kernel command line. If crypttab were in initramfs, systemd could know in advance what luks options to use and using kernel options wouldn't be necessary. Unfortunately, neither of the approaches work at the moment.
It is not included for me as well, but removing rd.luks.uuid= from the linux command line as explained in #2 fixed it for me. The /etc/crypttab is respected and I can trim my root fs. % sudo fstrim -v / /: 2126782464 bytes were trimmed I made the mistake of doing # grub2-mkconfig -o /boot/grub2/grub.cfg where it should have been # grub2-mkconfig -o /boot/efi/EFI/fedora/grub.cfg in my case because I'm using UEFI. For reference: [root@localhost ~]# cat /etc/crypttab luks-12345 UUID=12345 none allow-discards [root@localhost ~]# cat /etc/default/grub | grep CMDLINE GRUB_CMDLINE_LINUX="rd.lvm.lv=fedora/swap rd.md=0 rd.dm=0 $([ -x /usr/sbin/rhcrashkernel-param ] && /usr/sbin/rhcrashkernel-param || :) vconsole.keymap=us rd.lvm.lv=fedora/root rhgb quiet" [root@localhost ~]# cat /etc/lvm/lvm.conf | grep discard issue_discards = 1 (I'm not sure lvm.conf is related though). I didn't change my fstab because I don't want to use online discard - I cron ionice fstrim instead.
(In reply to comment #11) > (In reply to comment #10) > > why not configure that in your /etc/crypttab > > I have it configured in my crypttab (allow_discards). Kamil, I think it should be allow-disacards (- for _) according to man crypttab. Maybe that's your problem?
(In reply to comment #12) > (In reply to comment #10) > > # lsinitrd /boot/initramfs-$(uname -r).img etc/crypttab > > Interesting, /etc/crypttab is *not* included in the initramfs. Shouldn't it > be? Would it influence this bug somehow? Are you sure it's not in the initramfs? Did you install dracut-nohostonly? # lsinitrd | grep crypttab # lsinitrd /boot/initramfs-$(uname -r).img etc/crypttab Note the "etc/crypttab" with a missing "/" at the beginning!
(In reply to comment #15) > (In reply to comment #12) > > (In reply to comment #10) > > > # lsinitrd /boot/initramfs-$(uname -r).img etc/crypttab > > > > Interesting, /etc/crypttab is *not* included in the initramfs. Shouldn't it > > be? Would it influence this bug somehow? > > Are you sure it's not in the initramfs? Did you install dracut-nohostonly? > > # lsinitrd | grep crypttab > # lsinitrd /boot/initramfs-$(uname -r).img etc/crypttab > > Note the "etc/crypttab" with a missing "/" at the beginning! I can confirm that for, crypttab is not in initrd. Strangely, luks with TRIM works. [pschyska@localhost:~] % sudo lsinitrd /boot/initramfs-$(uname -r).img etc/crypttab [pschyska@localhost:~] % sudo lsinitrd /boot/initramfs-$(uname -r).img /etc/crypttab [pschyska@localhost:~] % sudo lsinitrd /boot/initramfs-$(uname -r).img | grep crypttab [pschyska@localhost:~] % Maybe the file got another name in initrd?
(In reply to comment #14) > Kamil, I think it should be allow-disacards (- for _) according to man > crypttab. Maybe that's your problem? Great call, I changed it to allow-discards, run dracut -f, removed all rd.luks.uuid, but it didn't help. > $ dmesg | grep discard > [ 0.000000] Command line: BOOT_IMAGE=/vmlinuz-3.8.6-203.fc18.x86_64 rd.md=0 rd.dm=0 rd.luks.allow-discards rd.lvm.lv=vg_ssd/lv_root rd.lvm.lv=vg/lv_swap root=/dev/mapper/vg_ssd-lv_root ro LANG=en_US.UTF-8 rhgb quiet > [ 0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-3.8.6-203.fc18.x86_64 rd.md=0 rd.dm=0 rd.luks.allow-discards rd.lvm.lv=vg_ssd/lv_root rd.lvm.lv=vg/lv_swap root=/dev/mapper/vg_ssd-lv_root ro LANG=en_US.UTF-8 rhgb quiet > [ 1.909351] systemd-cryptsetup-generator[97]: Unknown kernel switch rd.luks.allow-discards. Ignoring. > [ 2.527424] systemd-cryptsetup-generator[253]: Unknown kernel switch rd.luks.allow-discards. Ignoring. > [ 10.162015] systemd-cryptsetup-generator[363]: Unknown kernel switch rd.luks.allow-discards. Ignoring. > [ 12.826054] EXT4-fs (dm-1): mounting with "discard" option, but the device does not support discard > [ 12.826064] EXT4-fs (dm-1): mounted filesystem with ordered data mode. Opts: discard > [ 23.293054] EXT4-fs (dm-1): re-mounted. Opts: discard > $ sudo cat /etc/crypttab > luks-73c97d09-6d02-460c-9c66-94cbd86476d6 UUID=73c97d09-6d02-460c-9c66-94cbd86476d6 none allow-discards > luks-a3249b9e-3b9e-45b4-82ee-1cac21523253 UUID=a3249b9e-3b9e-45b4-82ee-1cac21523253 none allow-discards (In reply to comment #15) > Are you sure it's not in the initramfs? Yes, it's not present. > Did you install dracut-nohostonly? No, it's not installed.
(In reply to comment #17) > Are you sure it's not in the initramfs? Yes, it's not present. > > Did you install dracut-nohostonly? > > No, it's not installed. May I ask you to attach the debug.log of: # dracut -f test.img --debug &> debug.log ; rm -f test.img
Created attachment 737295 [details] dracut debug log Debug log attached. I have found out that if I get into the dracut shell (I've intentionally provided invalid password three times during boot), there is /etc/crypttab and it looks like this: dracut:/# cat /etc/crypttab luks-a3249b9e-3b9e-45b4-82ee-1cac21523253 /dev/sdb3 - timeout=0, luks-73c97d09-6d02-460c-9c66-94cbd86476d6 /dev/sda3 - timeout=0, But I'm quite sure that's dynamically generated, because if I disconnect one of the drives, only a single line is present. Also, since I haven't provided the password, it couldn't have accessed my root partition. Which confirms my belief that the proper crypttab is not copied into initramfs, and a new file is generated instead during boot with default options. Interesting is that I have a colleague with a very similar setup, also two drives - sda a standard disk and sdb an SSD disk, also luks encrypted, and the workaround mentioned above (removing rd.luks.uuid) works for him. The only difference between my setup and his is that I use LVM and he doesn't. But since dmsetup table doesn't report allow-discards in its output for me, my LVM can't make any difference, because even the underlying luks device is not mounted with the proper options.
I have found out that if I run: $ sudo dracut -f -I /etc/crypttab then all my problems are solved, and I don't even have to remove rd.luks.uuid from the command line. TRIM finally works. So the basic problem here seems to be in dracut, it's not including crypttab by default. Harald, we have hijacked this bug report a bit, the original report is about rd.luks.allow-discards being ignored by systemd, should I create a separate bug for dracut?
(In reply to comment #20) > I have found out that if I run: > > $ sudo dracut -f -I /etc/crypttab > > then all my problems are solved, and I don't even have to remove > rd.luks.uuid from the command line. TRIM finally works. > > So the basic problem here seems to be in dracut, it's not including crypttab > by default. > > Harald, we have hijacked this bug report a bit, the original report is about > rd.luks.allow-discards being ignored by systemd, should I create a separate > bug for dracut? ah, oh, this is F18.. yeah this should work: "rd.luks.allow-discards=a3249b9e-3b9e-45b4-82ee-1cac21523253 rd.luks.allow-discards=73c97d09-6d02-460c-9c66-94cbd86476d6 rd.luks.uuid=a3249b9e-3b9e-45b4-82ee-1cac21523253 rd.luks.uuid=73c97d09-6d02-460c-9c66-94cbd86476d6"
Thanks Kamil for pointing me at the real cause for this! Just wanted to add that if you like to be flexible (and tend to forget things like I do ;), you can also use "rd.luks.allow-discards=1" _without_ any rd.luks.uuid entry. I had to remove the uuid hint to make it work this way, but now it works automatically for all my luks disks. btw: Is this an exected result? I prefer it this way, cause having entries in the crypttab and on cmdline seems unnecessarily redundant to me...
The host-only mode in F19 will do exactly what you want (including the relevant /etc/crypttab entries). F18 is not ready to do so.
(In reply to comment #21) > this should work: > > "rd.luks.allow-discards=a3249b9e-3b9e-45b4-82ee-1cac21523253 > rd.luks.allow-discards=73c97d09-6d02-460c-9c66-94cbd86476d6 > rd.luks.uuid=a3249b9e-3b9e-45b4-82ee-1cac21523253 > rd.luks.uuid=73c97d09-6d02-460c-9c66-94cbd86476d6" No it doesn't. I have tried all possible combinations, only this works for me: 1. include a proper /etc/crypttab into initramfs. kernel boot options then doesn't matter -or- 2. remove all rd.luks.uuid and add rd.luks.allow-discards=UUID. Please note that removing all rd.luks.uuid adds 15 more seconds to boot during partition discovery. Also I have to use rd.luks.allow-discards=UUID, neither rd.luks.allow-discards nor rd.luks.allow-discards=1 work for me. Because of the boot time increase in 2), I'm going to stick with 1) now, until this is fixed. I have no idea whether this is several bugs, or all of it stems from the lack of support of rd.luks.allow-discards in systemd.
For me it only worked when I did sudo dracut -f -I /etc/crypttab AND removed the rd.luks.uuid=... from the kernel command line please fix this :-)
Should dracut.conf perhaps have an option for include crypttab like it does for lvm.conf and mdadm.conf? -c
Wakey Wakey on this bug guys. Either 1) rd.luks.allow-discards=1 needs to be added as a boot parameter 2) systemd needs to respect /etc/crypttab (on host) 3) systemd needs to respect /etc/crypttab AND crypttab needs to be added to the initramfs (or optionally include it via dracut.conf option) But this needs a solid fix with the documentation updated to reflect said fix. If you need a tester I've got an SSD here running F20 Alpha that I can test with.
Fedora 20 Alpha install. All updates applied. I had to do everything below in order to get fstrim to function and not error out. Added rd.luks.allow-discards=(UUID) [egriffith@eric-laptop ~]$ cat /etc/default/grub GRUB_TIMEOUT=5 GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)" GRUB_DEFAULT=saved GRUB_DISABLE_SUBMENU=true GRUB_TERMINAL_OUTPUT="console" GRUB_CMDLINE_LINUX="rd.luks.allow-discards=luks-cc54c314-7363-440f-b92f-499426d2539c rd.luks.allow-discards=luks-b11379f8-3415-4c29-8bb6-9fcdef19afc6 rd.luks.uuid=luks-cc54c314-7363-440f-b92f-499426d2539c rd.luks.uuid=luks-b11379f8-3415-4c29-8bb6-9fcdef19afc6 vconsole.font=latarcyrheb-sun16 vconsole.keymap=us $([ -x /usr/sbin/rhcrashkernel-param ] && /usr/sbin/rhcrashkernel-param || :) rhgb quiet" GRUB_DISABLE_RECOVERY="true" [egriffith@eric-laptop ~]$ Added allow-discards to crypttab [egriffith@eric-laptop ~]$ sudo cat /etc/crypttab luks-b11379f8-3415-4c29-8bb6-9fcdef19afc6 UUID=b11379f8-3415-4c29-8bb6-9fcdef19afc6 none allow-discards luks-cc54c314-7363-440f-b92f-499426d2539c UUID=cc54c314-7363-440f-b92f-499426d2539c none allow-discards [egriffith@eric-laptop ~]$ AND had to add it to the initrd [egriffith@eric-laptop ~]$ sudo lsinitrd | grep crypttab -rw-r--r-- 1 root root 234 Sep 12 15:36 etc/crypttab All before I could run: [egriffith@eric-laptop ~]$ sudo fstrim -v / /: 5 GiB (5332529152 bytes) trimmed [egriffith@eric-laptop ~]$ Without it erroring out. Not okay. WAAAAAAAAAY too many hoops given that upstream's documentation says that all you have to do is add the options for /etc/crypttab.
I'm using host-only initrd on F19, and it works OK here. I had to edit /etc/crypttab and regenerate initrd. No kernel cmdline adjustments necessary. Maybe it changed again in F20? Eric, are you using host-only or no-host-only mode in dracut? Are you sure that you can't remove rd.luks.allow-discards= ?
The problem is dracut by default doesn't include /etc/crypttab (maybe if you have host-only it does. But host-only isn't the default, and therefore the default doesn't include it) The initrd i've got right now is only valid until a kernel update when dracut will be re-executed but it won't include "-I /etc/crypttab" so I'll lose it and will have to re-run it by hand. That's not okay for end users. Now, if host-only mode DOES include /etc/crypttab, then great. But then host-only mode needs to be the new default setting.
Okay. So a default (no modifications) dracut does NOT include /etc/crypttab. A host-only dracut DOES include /etc/crypttab. So either default dracut needs to include /etc/crypttab. Or host-only mode needs to be the default, and then Fedora will actually be respecting LUKS' upstream documentation in regards to getting TRIM working on encrypted SSD's.
Eric, I just did a default installation of F20 Alpha RC3 Live x86_64 [1] with encrypted disks (so that /etc/crypttab is not empty). Dracut is in host-only mode by default and initrd contains crypttab by default. Can you try for yourself and then find the difference to your previous approach? [1] http://dl.fedoraproject.org/pub/alt/stage/
Behaviour may have changed, I installed with RC1 the night it came out. Wasn't there a dracut update in the last couple days? If so that may have finally enabled host-only mode by default because... All updates applied [egriffith@eric-laptop ~]$ sudo dnf update Setting up Upgrade Process Resolving dependencies --> Starting dependency resolution --> Finished dependency resolution Nothing to do. [egriffith@eric-laptop ~]$ Default dracut. [egriffith@eric-laptop ~]$ sudo dracut -f [egriffith@eric-laptop ~]$ sudo lsinitrd | grep crypttab -rw-r--r-- 1 root root 117 Sep 17 12:05 etc/crypttab [egriffith@eric-laptop ~]$ Dracut forced host-only [egriffith@eric-laptop ~]$ sudo dracut -f -H [egriffith@eric-laptop ~]$ sudo lsinitrd | grep crypttab -rw-r--r-- 1 root root 117 Sep 17 12:06 etc/crypttab [egriffith@eric-laptop ~]$ Dracut forced no-host-only [egriffith@eric-laptop ~]$ sudo dracut -f -N [egriffith@eric-laptop ~]$ sudo lsinitrd | grep crypttab [egriffith@eric-laptop ~]$ I left my dracut.conf unedited during these runs. (I had at one point uncommented hostonly="yes" I re-commented it for these runs) [egriffith@eric-laptop ~]$ cat /etc/dracut.conf | grep host #hostonly="yes" [egriffith@eric-laptop ~]$ If hostonly="yes" is the new default, shouldn't hostonly="" default to "no" in the config file to reflect the opposite decision of defaults?
$ fgrep -r hostonly /etc/dracut.conf* /usr/lib/dracut/dracut.conf.d/ /etc/dracut.conf:#hostonly="yes" /usr/lib/dracut/dracut.conf.d/01-dist.conf:hostonly="yes" Do you have the dracut-config-generic rpm installed?
[egriffith@eric-laptop ~]$ fgrep -r hostonly /etc/dracut.conf* /usr/lib/dracut/dracut.conf.d/ /etc/dracut.conf:#hostonly="yes" /usr/lib/dracut/dracut.conf.d/01-dist.conf:hostonly="yes" [egriffith@eric-laptop ~]$ Apparently not [egriffith@eric-laptop ~]$ sudo rpm -qa | grep dracut dracut-033-3.git20130913.fc20.x86_64 dracut-config-rescue-033-3.git20130913.fc20.x86_64 dracut-network-033-3.git20130913.fc20.x86_64 [egriffith@eric-laptop ~]$ Like I said, the behaviour NOW seems to be hostonly mode. Which is good, But it wasn't a week ago when I started posting to this bug. Hence my thinking that the dracut update in the last couple days may have been the one to finally enable it by default.
Host-only is the default since F19, and you can easily distinguish it by the initrd size. But I find very interesting that if you use -N option in dracut, the crypttab is not included. That might be a bug. Harald?
This message is a reminder that Fedora 18 is nearing its end of life. Approximately 4 (four) weeks from now Fedora will stop maintaining and issuing updates for Fedora 18. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as WONTFIX if it remains open with a Fedora 'version' of '18'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version prior to Fedora 18's end of life. Thank you for reporting this issue and we are sorry that we may not be able to fix it before Fedora 18 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior to Fedora 18's end of life. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete.
With F-20 as of today and trying to "fstrim /" (root filesystem). Using LUKS only, no LVM, no mdadm. With rd.luks.allow-discards=... suggested above I get: systemd-cryptsetup-generator[368]: Unknown kernel switch rd.luks.allow-discards=1. Ignoring. I had to use in /etc/grub2.cfg: rd.luks.options=discard and I also had to keep in place: rd.luks.uuid=luks-5b30fc63-6b52-4746-b883-394fbe23c3dd (otherwise it does not boot) /etc/crypttab is present in initramfs by default but its options for root device are ignored. Not updating Bug Version to 20 as this is yum-upgraded OS, not a fresh installation. Maybe new Anaconda installation would have it right, who knows.
On a fresh fedora 20 install, with LUKS+BTRFS no LVM. (My only unusual is to have to separate BTRFS partitions, unencrypted root, and encrypted /home). I needed to add rd.luks.allow-discards=luks-3365373e-6849-4a38-8d6a-2032b3575d34 to the GRUB_CMDLINE_LINUX in /etc/default/grub and run sudo grub2-mkconfig -o /boot/grub2/grub.cfg and add allow-discards to the end of the lines in /etc/crypttab and run sudo dracut -f then I can run fstrim.
I don't think at this point it makes sense to add support for rd.luks.allow-discards. We already have rd.luks.options=discard which should be good enough and more sytematic as you can specify any crypttab option you like there...
(In reply to Lennart Poettering from comment #40) > I don't think at this point it makes sense to add support for > rd.luks.allow-discards. We already have rd.luks.options=discard which should > be good enough and more sytematic as you can specify any crypttab option you > like there... This is true, if systemd is included in the initrd, especially the systemd-cryptsetup-generator.
(In reply to Lennart Poettering from comment #40) > I don't think at this point it makes sense to add support for > rd.luks.allow-discards. We already have rd.luks.options=discard which should > be good enough and more sytematic as you can specify any crypttab option you > like there... What is the status here as this does not match the current https://www.kernel.org/pub/linux/utils/boot/dracut/dracut.html ?