Description of problem: I have not been able to boot on one of my machines since the 3.0 kernel release. The kernel-PAE-3.0-0.rc7.git10.1.fc16.i686 works, but so far every later kernel (currently through kernel-PAE-3.1.0-0.rc0.git11.2.fc17.i686) fails to boot because no raid devices are detected and it is unable to mount the root file system. I have another machine where this doesn't happen. Both machines have an encrypted root device on top of software raid 1. The machine that works has version 0.90 arrays and the one that doesn't has version 1.2 arrays, except for /boot which has a version 1.0 array. While booting the problem system do md arrays are noted prior to trying to using the file system specified on the root= parameter and the boot fails when trying to mount that file system. No password for the luks device is asked for, but given that the array for the root device wasn't detected, this isn't surprising. I haven't tried rerunning dracut on the older kernel entry, as if it breaks things I am fairly hosed. But it is possible that an update to dracut, mdadm or some other tool triggered the problem, rather than the kernel update. Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info:
Created attachment 515982 [details] dmesg output from successful boot
I should be able to test dracut by copying over the 3.0-0.rc7.git10.1.fc16.i686 files in /boot and making a new grub entry. After confirming I can boot off the copies, I'll run the kernel update script that rebuilds initramfs and see if that makes things break. I'm in the middle of processing today's rawhide update, so it will be a bit before I test this.
I ran /sbin/new-kernel-pkg --package kernel-PAE --mkinitrd --dracut --depmod --update 3.0-0.rc7.git10.1.fc16.i686.PAE and the 3.0-0.rc7.git10.1.fc16.i686.PAE kernel still booted. So it's looking more like something that changed between 3.0-0.rc7.git10.1 and 3.0.0-1 that triggers the problem.
I am still seeing this with kernel-PAE-3.1.0-0.rc0.git12.1.fc17.i686. After tonight I won't have physical access to the machine for a week and won't be able to test rebooting it during that time.
The only non-merge changes in the upstream kernel between 3.0-rc7-git10 and 3.0 are 33d8881af5584fb7994f6b3d17fc11dcaf07b3b2 and 2cebaa58b7de775386732bbd6cd11c3f5b73faf0 neither of which have anything to do with the md area of the kernel, so that is odd. In the kernel package itself, we simply switched the source to the release tarball. About the only thing I can see that changed that might be relevant between -rc7-git10 and 3.0.0-1 is that uname went from 3.0-0.rc7 to 3.0.0, so a two digit to three digit change. I would have expected a problem the reverse way though.
Do you have a log of the boot failing?
It doesn't get far enough to log. When I get back from my trip I can take a picture of the screen.
Created attachment 517478 [details] Output on screen when boot failed I tested this again with kernel-PAE-3.1.0-0.rc1.git1.1.fc17.i686 and snapped a picture when it failed.
Out of curiosity, does the grub entry for the failing kernel(s) have an initrd line and do you see something that looks like this during boot: [ 0.770386] Unpacking initramfs... [ 2.545097] Freeing initrd memory: 15340k freed and then later: [ 3.571223] Freeing unused kernel memory: 1908k freed
I am seeing a possibly related problem with 2.6.40 kernels on F15. I filed bug 729743 for this, and because the symptoms were somewhat different was able to record dmesg output. No there isn't an initrd line for the broken kernels. The first entry looks like this: title Fedora (3.1.0-0.rc1.git1.1.fc17.i686.PAE) root (hd0,0) kernel /vmlinuz-3.1.0-0.rc1.git1.1.fc17.i686.PAE ro root=/dev/mapper/luks-9a976b 86-8aaa-40d9-8039-89d710eac5c9 SYSFONT=latarcyrheb-sun16 LANG=en_US.UTF-8 KEYTAB LE=us radeon.agpmode=-1 I can add one and do a test reboot when I get home from work.
(In reply to comment #10) > No there isn't an initrd line for the broken kernels. > > The first entry looks like this: > title Fedora (3.1.0-0.rc1.git1.1.fc17.i686.PAE) > root (hd0,0) > kernel /vmlinuz-3.1.0-0.rc1.git1.1.fc17.i686.PAE ro > root=/dev/mapper/luks-9a976b > 86-8aaa-40d9-8039-89d710eac5c9 SYSFONT=latarcyrheb-sun16 LANG=en_US.UTF-8 > KEYTAB > LE=us radeon.agpmode=-1 > > I can add one and do a test reboot when I get home from work. Yeah. Without the initrd line, the initramfs doesn't get loaded. Then the kernel decides it's going to try and be helpful and look for RAID arrays to assemble, doesn't get it right, and then gives up. The problem here is there is no initramfs being loaded, not really anything with the kernel.
Thanks. Adding the initrd line fixed things. I am not sure how I managed to lose the one that was there.