Description of problem: When booting rawhide with 2.6.25 it is unable to decrypt my encrypted root partition that contains my LVM volumes. Version-Release number of selected component (if applicable): 2.6.25-0.35.rc1.fc9.i686 How reproducible: Always Steps to Reproduce: 1. Boot into 2.6.25-0.35.rc1.fc9.i686 2. When prompted, enter password to decrypt partition. 3. Actual results: Can't decrypt partition and boot fails. Expected results: Partition is decrypted and system continues to boot up. Works fine in 2.6.24.1-26.fc9.i686. Additional info: Screen shots of the failed boot can be found here: http://jcollie.fedorapeople.org/2625-luks-1.jpg http://jcollie.fedorapeople.org/2625-luks-2.jpg
# mkinitrd -f /boot/initrd-2.6.25-0.40.rc1.git2.fc9.img 2.6.25-0.40.rc1.git2.fc9 no path for essiv # dmsetup table luks-sda2: 0 77753568 crypt aes-cbc-essiv:sha256 00000000000000000000000000000000 0 8:2 1032 pc21225-swap: 0 8388608 linear 253:0 69271936 pc21225-root: 0 69271552 linear 253:0 384
Created attachment 294957 [details] Verbose log from mkinitrd
See bug 433078, which is the root cause of this failure.
http://jcollie.fedorapeople.org/2625-luks-3.jpg Definitely appears that #433078 is what's wrong... going to close as a duplicate. *** This bug has been marked as a duplicate of 433078 ***
Blarrrg! I was wrong in bug 433078 - it's not modprobe that's the problem.
*** Bug 433078 has been marked as a duplicate of this bug. ***
So, yes, something is wrong in kernel-land. The key messages are: device-mapper: table: 253:0: crypt: Error allocating crypto tfm device-mapper: ioctl: error adding target to table device-mapper: ioctl: device doesn't appear to be in the dev hash table. Failed to setup dm-crypt key mapping. Check kernel for support for the aes-cbc-essiv:sha256 cipher spec and verify that /dev/sda2 contains at least 133 sectors. I guess that means either: 1) we're not loading the AES modules 2) they're not providing needed bits (aes-cbc-essiv:sha256 cipher spec), or 3) cryptsetup is failing to check for those bits properly. It's probably *not* modprobe's fault, as determined in bug 433078. So that leaves 2) or 3).
*** Bug 433463 has been marked as a duplicate of this bug. ***
I also tried to load the disk in rescue mode to see if I could see anything. I find it very interesting that there is no apparent even attempt to load the encrypted volumes or to request a password for them. This is very "not good". Should I file a RFE for this one, or is that in progress or something.
Rescue mode pieces are in progress
If it helps any this system was working fine, and then I ran the automatic update and broke it. This is also an experimental system. I am pretty sure I could wipe and reload and get a working system, that I could then try separately updating anything you want me to and see if that is what breaks it.
Encryption support for rescue mode is bug 430786.
My laptop is running rawhide and with an encrypted root now, so I'll be able to dig a little more once I get home and have the USB keys I foolishly left sitting on the coffee table :)
For both this and the rescue mode peices, I have taken the fc8 "why don't you guys test the alphas" comments to heart. I have 4-8 hours a week which they have set aside for me to test things like this. So let me know what you need.
You want a guess, I did notice that the update claimed to be updating some kernel files. Could this be the problem?
Yes. It's something to do with kernel 2.6.25. We're working on it.
*** Bug 433022 has been marked as a duplicate of this bug. ***
There is change in 2.6.25 in dm-crypt to use async crypto (if available). Which crypt modules are loaded in ramdisk? Is there crypto_blkcipher module in the ramdisk?
Created attachment 295417 [details] Listing of the files in /boot/initrd-2.6.25-0.40.rc1.git2.fc9.img I've attached a list of the files in /boot/initrd-2.6.25-0.40.rc1.git2.fc9.img
I have the machine this has failed on "frozen". From what I see here it sounds like even once it is fixed that the solution to implement the fix may be to simply reload. So are you going to need victims to try your fix on, or should I just reload and continue testing without doing to update which breaks things?
You should be able to boot the previously working kernel (probably 2.6.24.1-31.fc9) from the GRUB menu, since yum is set up to save the previous kernel by default. So you can get the system working again just by booting the older kernel. When we have a fixed kernel package, you'll just need to update the kernel and everything should be OK. Unless you did something strange like removing the older kernel, there should be no need to reinstall.
OK this works. On to trying to break other things. ;-) Basically I am going through all my old bug reports and seeing if I can have problems doing the same things. So you probably will see more reports, but probably not to many we hope.
Seems that there is missing crypto module "chainiv.ko" in ramdisk for 2.6.25 kernel. Please could anyone add it to ramdisk and test it again?
See http://lkml.org/lkml/2008/2/20/403
(In reply to comment #23) > Seems that there is missing crypto module "chainiv.ko" in ramdisk for 2.6.25 kernel. > > Please could anyone add it to ramdisk and test it again? Woohoo! rebuilding my initrd with --with=chainiv and 2.6.25 is able to decrypt my partition now...
Works for me too. I rebuilt my initrd like so: mkinitrd -f --with=chainiv 2.6.25-0.40.rc1.git2.fc9 /boot/initrd-2.6.25-0.40.rc1.git2.fc9.img and now the system boots properly. So - how long until this is fixed in the upstream kernel? Should we make crypto_blkcipher depend on chainiv until then?
No idea about upstream, but for Fedora there should be some workaround asap. Note that fresh network installation of rawhide (with disk encryption) now produces completely unbootable system (there is no old kernel in repository, just 2.6.25).
*** Bug 433834 has been marked as a duplicate of this bug. ***
From Bug 433834 this appears it may be fixed: Comment #8 From Bob Agel (cragel.com) on 2008-02-21 14:34 EST [reply] Today's update to 2.6.25-0.54.rc2.fc9 fixes the problem - booting normally now.
Interesting booting this kernel has not fixed the problem here.
There's basically no way that 2.6.25-0.54.rc2.fc9 could have fixed the problem, since it made no changes to the crypt code. On the other hand, kernel-2.6.25-0.64.rc2.git5.fc9 has: * Thu Feb 21 2008 Kyle McMartin <kmcmartin> - crypto_blkcipher: big hack caused module dep loop, try another fix I can confirm that it fixes the problem for me. It should appear with the next rawhide update - or fetch it from koji in the meantime: http://koji.fedoraproject.org/koji/buildinfo?buildID=39187
2.6.25-0.64.rc2.git5.fc9 fixes it for me too.
*** Bug 434636 has been marked as a duplicate of this bug. ***
Confirming this is fixed in the latest release. But that did break so things so let me get on to creating some new bug reports.