Bug 432813

# mkinitrd -f /boot/initrd-2.6.25-0.40.rc1.git2.fc9.img 2.6.25-0.40.rc1.git2.fc9
no path for essiv

# dmsetup table
luks-sda2: 0 77753568 crypt aes-cbc-essiv:sha256
00000000000000000000000000000000 0 8:2 1032
pc21225-swap: 0 8388608 linear 253:0 69271936
pc21225-root: 0 69271552 linear 253:0 384

Created attachment 294957 [details]
Verbose log from mkinitrd

See bug 433078, which is the root cause of this failure.

http://jcollie.fedorapeople.org/2625-luks-3.jpg

Definitely appears that #433078 is what's wrong... going to close as a duplicate.

*** This bug has been marked as a duplicate of 433078 ***

Blarrrg! I was wrong in bug 433078 - it's not modprobe that's the problem. 

*** Bug 433078 has been marked as a duplicate of this bug. ***

So, yes, something is wrong in kernel-land. The key messages are:

device-mapper: table: 253:0: crypt: Error allocating crypto tfm
device-mapper: ioctl: error adding target to table
device-mapper: ioctl: device doesn't appear to be in the dev hash table.
Failed to setup dm-crypt key mapping. 
Check kernel for support for the aes-cbc-essiv:sha256 cipher spec and verify
that /dev/sda2 contains at least 133 sectors.

I guess that means either:

1) we're not loading the AES modules
2) they're not providing needed bits (aes-cbc-essiv:sha256 cipher spec), or 
3) cryptsetup is failing to check for those bits properly.

It's probably *not* modprobe's fault, as determined in bug 433078. So that
leaves 2) or 3).

*** Bug 433463 has been marked as a duplicate of this bug. ***

I also tried to load the disk in rescue mode to see if I could see anything.   I
find it very interesting that there is no apparent even attempt to load the
encrypted volumes or to request a password for them.   This is very "not good".

Should I file a RFE for this one, or is that in progress or something.

Rescue mode pieces are in progress

If it helps any this system was working fine, and then I ran the automatic
update and broke it.  This is also an experimental system.  I am pretty sure I
could wipe and reload and get a working system, that I could then try separately
updating anything you want me to and see if that is what breaks it.

Encryption support for rescue mode is bug 430786.

My laptop is running rawhide and with an encrypted root now, so I'll be able to
dig a little more once I get home and have the USB keys I foolishly left sitting
on the coffee table :)

For both this and the rescue mode peices, I have taken the  fc8 "why don't you
guys test the alphas" comments to heart.  I have 4-8 hours a week which they
have set aside for me to test things like this.

So let me know what you need.

You want a guess, I did notice that the update claimed to be updating some
kernel files.   Could this be the problem?

Yes. It's something to do with kernel 2.6.25. We're working on it.

*** Bug 433022 has been marked as a duplicate of this bug. ***

There is change in 2.6.25 in dm-crypt to use async crypto (if available).

Which crypt modules are loaded in ramdisk?
Is there crypto_blkcipher module in the ramdisk?

Created attachment 295417 [details]
Listing of the files in /boot/initrd-2.6.25-0.40.rc1.git2.fc9.img

I've attached a list of the files in /boot/initrd-2.6.25-0.40.rc1.git2.fc9.img

I have the machine this has failed on "frozen".   From what I see here it sounds
like even once it is fixed that the solution to implement the fix may be to
simply reload.   So are you going to need victims to try your fix on, or should
I just reload and continue testing without doing to update which breaks things?

You should be able to boot the previously working kernel (probably
2.6.24.1-31.fc9) from the GRUB menu, since yum is set up to save the previous
kernel by default.

So you can get the system working again just by booting the older kernel. When
we have a fixed kernel package, you'll just need to update the kernel and
everything should be OK.

Unless you did something strange like removing the older kernel, there should be
no need to reinstall.

OK this works.   On to trying to break other things.  ;-)   Basically I am going
through all my old bug reports and seeing if I can have problems doing the same
things.   So you probably will see more reports, but probably not to many we hope.

Seems that there is missing crypto module "chainiv.ko" in ramdisk for 2.6.25 kernel.

Please could anyone add it to ramdisk and test it again?

See http://lkml.org/lkml/2008/2/20/403

(In reply to comment #23)
> Seems that there is missing crypto module "chainiv.ko" in ramdisk for 2.6.25
kernel.
> 
> Please could anyone add it to ramdisk and test it again?

Woohoo! rebuilding my initrd with --with=chainiv and 2.6.25 is able to decrypt
my partition now...

Works for me too. I rebuilt my initrd like so:

mkinitrd -f --with=chainiv 2.6.25-0.40.rc1.git2.fc9
/boot/initrd-2.6.25-0.40.rc1.git2.fc9.img

and now the system boots properly.

So - how long until this is fixed in the upstream kernel? Should we make
crypto_blkcipher depend on chainiv until then?

No idea about upstream, but for Fedora there should be some workaround asap.

Note that fresh network installation of rawhide (with disk encryption) now
produces completely unbootable system (there is no old kernel in repository,
just 2.6.25).

*** Bug 433834 has been marked as a duplicate of this bug. ***

From Bug 433834 this appears it may be fixed:

Comment #8 From Bob Agel (cragel.com) 	on 2008-02-21 14:34 EST 	[reply] 	 

Today's update to 2.6.25-0.54.rc2.fc9 fixes the problem - booting normally now.

Interesting booting this kernel has not fixed the problem here.

There's basically no way that 2.6.25-0.54.rc2.fc9 could have fixed the problem,
since it made no changes to the crypt code. 

On the other hand, kernel-2.6.25-0.64.rc2.git5.fc9 has:

* Thu Feb 21 2008 Kyle McMartin <kmcmartin>
- crypto_blkcipher: big hack caused module dep loop, try another fix

I can confirm that it fixes the problem for me. It should appear with the next
rawhide update - or fetch it from koji in the meantime:

http://koji.fedoraproject.org/koji/buildinfo?buildID=39187

2.6.25-0.64.rc2.git5.fc9 fixes it for me too.

*** Bug 434636 has been marked as a duplicate of this bug. ***

Confirming this is fixed in the latest release.  But that did break so things so
let me get on to creating some new bug reports.