Bug 561510 - F12 upgrade, lvm root -> No root device found
Summary: F12 upgrade, lvm root -> No root device found
Keywords:
Status: CLOSED CANTFIX
Alias: None
Product: Fedora
Classification: Fedora
Component: dracut
Version: 12
Hardware: i686
OS: Linux
low
medium
Target Milestone: ---
Assignee: Harald Hoyer
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2010-02-03 20:05 UTC by Jeff Hardy
Modified: 2010-08-25 04:17 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2010-04-15 14:15:50 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
Exactly the same bug on my upgrade to F12 (83.70 KB, text/plain)
2010-03-22 20:27 UTC, Dave Brennan
no flags Details

Description Jeff Hardy 2010-02-03 20:05:38 UTC
Description of problem:
I upgraded machine to F12, and upon reboot: "No root device found" "Boot has failed, sleeping forever."  Two drive layout as follows:
/dev/sda1: /boot
/dev/sda2: Volume00 (/, /var, swap)
/dev/sdb1: Volume00 (/, /var, swap)

Remaining F11 kernels still worked.  After successfully booting with F11 kernel, I manually installed F12 kernels hoping new dracut environment would sort itself out, but none worked.  From rescue mode, I manually ran dracut to create initramfs images, but that also did not work.

Version-Release number of selected component (if applicable):
dracut-002-13.4.git8f397a9b.fc12.noarch

How reproducible:
Always.

Steps to Reproduce:
1. Install F11 with disk layout above.
2. Upgrade to F12.
3. Reboot.

Actual results:
No root device found
Boot has failed, sleeping forever.

Expected results:
Boot into F12.

Additional info:
I ended up working around the problem by vgreducing the VolumeGroup off of sda2, copying my root back to plain ext3 on sda2, and manually running dracut.  On reboot, it was able to find root.

Also, this machine has been upgraded from FC1 all the way through F12 sequentially.  As such, there are probably a number of details that affected this.  Possibly that it was still using LVM1 metadata?  I vgconverted to do a pvmove.  Full details of the entire process to restore functional desktop at http://blog.fritzhardy.com/?p=5

Comment 1 Jeff Hardy 2010-02-03 20:12:54 UTC
.
.
.
[drm] nouveau 0000:05:01.0: PFIFO_INTR 0x00010000 - Ch 0
[drm] nouveau 0000:05:01.0: PFIFO still angry after 101 spins, halt
[drm] nouveau 0000:05:01.0: Initial CRTC_OWNER is 0
[drm] nouveau 0000:05:01.0: Enabled TV-Out with tv module option
[drm] nouveau 0000:05:01.0: Detected a DVI-I connector
[drm] nouveau 0000:05:01.0: Saving VGA fonts
[TTM] Failed moving buffer. Proposed placement 0x00070004
[TTM] Out of aperture space or DRM memory quota.
[drm] nouveau 0000:05:01.0: failed to allocate framebuffer
[drm] Initialized nouveau 0.0.15 20090420 for 0000:05:01.0 on minor 2
dracut: Starting plymouth daemon
dracut: Starting plymouth daemon
Fusion MPT base drver 3.04.10
Copyright (c) 1999-2008 LSI Corporation
Fusion MPT SPI Host driver 3.04.10
mptspi 0000:03:05.0: PCI INT A -> GSI 26 (level, low) -> IRQ 26
mptbase: ioc0: Initiating bringup
ioc0: LSI53C1030 B2: Capabilitiies={Initiator}
scsi2 : ioc0: LSI53C1030 B2, FwRev=01030a00h, Ports=1 MaxQ=255, IRQ=27
scsi: waiting for bus probes to complete ...


No root device found


No root device found

Boot has failed, sleeping forever.

Comment 2 Harald Hoyer 2010-02-04 09:42:53 UTC
can you also update dracut and rebuild the initramfs with:

# dracut -f /boot/initramfs-<kernel version>.img <kernel version>

Comment 3 Harald Hoyer 2010-02-04 09:46:01 UTC
(In reply to comment #0)
> Possibly that it was still using LVM1 metadata?  I vgconverted to do a
> pvmove.  Full details of the entire process to restore functional desktop at
> http://blog.fritzhardy.com/?p=5    

LVM1 metadata might have been the cause. dracut looks for 

ID_FS_TYPE="LVM2_member"

which is taken from the blkid output:

# /sbin/blkid -o udev -p <device node>

Comment 4 Harald Hoyer 2010-02-04 09:47:51 UTC
to get more info from dracut add to the kernel command line:

"rdinfo rdshell" and remove "rhgb quiet"

for even more debug info:

"rdinitdebug rdshell" and remove "rhgb quiet"

with messages in dmesg and /init.log

Comment 5 Jeff Hardy 2010-02-04 13:33:29 UTC
I can tell you that I did run the dracut line you noted with the version implicated in this ticket, on several different F12 kernels, but nothing worked.  Also, I should have mentioned that the output I provided was with "rhgb quiet" removed.

Unfortunately since I have already worked around the issue, in my case by moving root to a plain ext3 partition from LVM (definitely LVM1), I am afraid most of the above tests are now moot.

Comment 6 Dave Brennan 2010-03-22 20:27:32 UTC
Created attachment 401868 [details]
Exactly the same bug on my upgrade to F12

I have exactly the same problem and I have included the debug output from dracut.

I have a rootfs LV as part of inca VG. inca/rootfs is spread over 2 PVs.

The only way I can boot the system is to include rdshell on the boot line and the on failure issue

ln -s /dev/inca/rootfs /dev/root

The system will then boot.

I have rebuilt the the initramfs with dracut with no change.

The debug output shows that init is waiting for a script to complete which runs out of loop count.

Comment 7 Harald Hoyer 2010-03-23 09:17:42 UTC
(In reply to comment #6)
> Created an attachment (id=401868) [details]
> Exactly the same bug on my upgrade to F12
> 
> I have exactly the same problem and I have included the debug output from
> dracut.
> 
> I have a rootfs LV as part of inca VG. inca/rootfs is spread over 2 PVs.
> 
> The only way I can boot the system is to include rdshell on the boot line and
> the on failure issue
> 
> ln -s /dev/inca/rootfs /dev/root
> 
> The system will then boot.
> 
> I have rebuilt the the initramfs with dracut with no change.
> 
> The debug output shows that init is waiting for a script to complete which runs
> out of loop count.    

you specified on the kernel command line:
 root=/dev/inca/rootfs/

try to remove the trailing "/"!!!

Comment 8 Dave Brennan 2010-03-23 18:25:02 UTC
(In reply to comment #7)
> (In reply to comment #6)
> > Created an attachment (id=401868) [details] [details]
> > Exactly the same bug on my upgrade to F12
> > 
> > I have exactly the same problem and I have included the debug output from
> > dracut.
> > 
> > I have a rootfs LV as part of inca VG. inca/rootfs is spread over 2 PVs.
> > 
> > The only way I can boot the system is to include rdshell on the boot line and
> > the on failure issue
> > 
> > ln -s /dev/inca/rootfs /dev/root
> > 
> > The system will then boot.
> > 
> > I have rebuilt the the initramfs with dracut with no change.
> > 
> > The debug output shows that init is waiting for a script to complete which runs
> > out of loop count.    
> you specified on the kernel command line:
>  root=/dev/inca/rootfs/

Oh! Joy.....that worked system now boots
> try to remove the trailing "/"!!!

Comment 9 Dave Brennan 2010-03-23 20:33:58 UTC
Just one point from the original debug output I noticed that the lvm_scan produced:-

dracut: Scanning devices sda2 sdb sdb1  for LVM volume groups 
dracut: Reading all physical volumes. This may take a while...
dracut: Found volume group "inca" using metadata type lvm2
dracut: 2 logical volume(s) in volume group "inca" now active

This suggest that the device list is faulty as I would expect only sda2 and sdb1 not sdb to be listed for scanning

Comment 10 Harald Hoyer 2010-03-24 09:57:35 UTC
(In reply to comment #9)
> Just one point from the original debug output I noticed that the lvm_scan
> produced:-
> 
> dracut: Scanning devices sda2 sdb sdb1  for LVM volume groups 
> dracut: Reading all physical volumes. This may take a while...
> dracut: Found volume group "inca" using metadata type lvm2
> dracut: 2 logical volume(s) in volume group "inca" now active
> 
> This suggest that the device list is faulty as I would expect only sda2 and
> sdb1 not sdb to be listed for scanning    

right

can you please provide the output of:

# blkid -o udev -p /dev/sdb

Comment 11 Dave Brennan 2010-03-24 18:37:06 UTC
Well that was a bit unexpected output was:-

[root@inca ~]# blkid -o udev -p /dev/sdb
ID_FS_UUID=lK7zr0-i6BQ-OLi0-H6jP-nuXd-x6Uf-eYnAgP
ID_FS_UUID_ENC=lK7zr0-i6BQ-OLi0-H6jP-nuXd-x6Uf-eYnAgP
ID_FS_VERSION=LVM2\x20001
ID_FS_TYPE=LVM2_member
ID_FS_USAGE=raid
[root@inca ~]#

for completeness

[root@inca ~]# blkid -o udev -p /dev/sda2
ID_FS_UUID=lXl0ab-VLIY-ilhE-chCy-Rrk1-E7Wz-CB8UFD
ID_FS_UUID_ENC=lXl0ab-VLIY-ilhE-chCy-Rrk1-E7Wz-CB8UFD
ID_FS_VERSION=LVM2\x20001
ID_FS_TYPE=LVM2_member
ID_FS_USAGE=raid
[root@inca ~]#

[root@inca ~]# blkid -o udev -p /dev/sdb1
ID_FS_UUID=7EiSHf-nf65-2JTo-ReOE-85cF-3Tww-JU8rhC
ID_FS_UUID_ENC=7EiSHf-nf65-2JTo-ReOE-85cF-3Tww-JU8rhC
ID_FS_VERSION=LVM2\x20001
ID_FS_TYPE=LVM2_member
ID_FS_USAGE=raid
[root@inca ~]#

Confused as to why sdb is an LVM member

[root@inca ~]# blkid
/dev/sda1: LABEL="/boot" UUID="a86d6961-d455-43f1-913c-c98bebf6651c" SEC_TYPE="ext2" TYPE="ext3"
/dev/sda2: UUID="lXl0ab-VLIY-ilhE-chCy-Rrk1-E7Wz-CB8UFD" TYPE="LVM2_member"
/dev/sdb1: UUID="7EiSHf-nf65-2JTo-ReOE-85cF-3Tww-JU8rhC" TYPE="LVM2_member"
/dev/mapper/inca-rootfs: UUID="db1c341b-240d-465a-903d-4de94a11a70c" TYPE="ext3"
/dev/mapper/inca-swap: TYPE="swap"

[root@inca ~]# pvscan
  PV /dev/sdb1   VG inca   lvm2 [1.36 TB / 0    free]
  PV /dev/sda2   VG inca   lvm2 [931.00 GB / 21.47 GB free]
  Total: 2 [2.27 TB] / in use: 2 [2.27 TB] / in no VG: 0 [0   ]
[root@inca ~]#

Comment 12 Harald Hoyer 2010-03-24 20:32:27 UTC
can you file a bug against blkid please?

Comment 13 Dave Brennan 2010-03-24 20:40:24 UTC
Will do

Comment 14 Kent Reynolds 2010-08-25 04:17:55 UTC
As a work around, you can install FC12 on a second hard drive (make sure to match either Sata or IDE disks with your main disk image or you´ll run into other disasters). Then copy the content of /boot onto your main disk image via a thumb drive, directly onto your main disk, a CD or whatever you can get away with. Once you´ve got the donor kernel on your main disk, just make sure that /boot/grub.grub.conf has an entry for your new kernel.

My system has been running a donor kernel since I first upgraded to FC12. Just watch out for yum kernel updates. Make sure to comment out any changes in your grub.conf

I´ve used this technique a few times. I needed to use it when migrating from an IDE to Sata disk as well as upgrading (to get around controller driver issues). This systems gone sequentially up from FC9.

In saying this, my packages where a complete mess... Needed to add symlinks for libraries to get yum to work and other packages took some wiggling to get ´em right. Which brings me to this thread. I would really like to solve this issue because I can´t even begin thinking about FC13 until I fix it.


Note You need to log in before you can comment on or make changes to this bug.