Bug 431778

Summary: mount: could not find filesystem '/dev/root', system hangs
Product: [Fedora] Fedora Reporter: Exile In Paradise <redhat>
Component: anacondaAssignee: David Lehman <dlehman>
Status: CLOSED CURRENTRELEASE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: low    
Version: 9CC: agk, bmarzins, dwysocha, israelagnouhyattara, jf_nz, kuixing112306, mbroz, pkern, prockai, stephen-rhel, steve
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2008-12-08 19:47:48 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Exile In Paradise 2008-02-06 22:00:15 UTC
Description of problem:
Installed Fedora 9 Alpha, taking pretty much all defaults, options including
encrypted filesystem, remove all partitions and let Fedora 9 choose, default
package set, etc.
Install completes.
On reboot, the following error:
mount: could not find filesystem '/dev/root'
setuproot: moving /dev/failed: no such file or directory
setuproot: error mounting /proc: No such file or directory.
setuproot: error mounting /sys: No such file or directory.
switchroot: mount failed: No such file or directory.
input: ImPS/2 Generic Wheel Mouse as /devices/platform/i8042/serio1/input/input2
(may have been cutoff here?)
usb 1-9: configuration #1 chosen from 1 choice
<system stops at this point, no panic, no hang. CTRL-ALT-DEL still works).

Version-Release number of selected component (if applicable):
Fedora 9 Alpha

How reproducible:
100% - every boot is the same.
Unknown if another reinstall ends up the same.

Steps to Reproduce:
1. Install Fedora 9 Alpha with default encryption, partitioning, and packages.
2. Let system boot through default grub settings.
3. Watch as system stops in the same place every time.
  
Actual results:
System stops booting after usb 1-9 configuration chosen message

Expected results:
System continues booting.

Additional info:
Before the error, I see
device-mapper: uevent: version 1.0.3
device mapper: ioctl: 4.12.0-ioctl (2007-10-02) initialized
I do not see dm_mod, dm_crypt, dm_mirror, or other usual dm suspects loading
from initrd?

A test reboot with root=/dev/mapper/dm-0 or dm-1 fails, slightly different,
complaining about missing fstab.sys and mounting internal defaults.

I booted from rescue
cryptsetup luksOpen /dev/sda2 root
lvm vgchange -a -y VolGroup00
mounted it under /mnt/root

crypttab contained:
luks-sda2 /dev/sda2 none

fstab contained:
/dev/VolGroup00/LogVol00 / ext3 defaults 1 1
/dev/VolGroup00/LogVol01 swap swap defaults 0 0
and the usual suspects for /boot /sys, /proc, etc.

An attempt to remake the initrd with dm_* and crypto modules succeeded, but the
reboot still fails with the /dev/root error.

Comment 1 Exile In Paradise 2008-02-06 22:24:46 UTC
I repeated the rescue boot above, and added the following:
once the disk is mounted under /mnt/root, mknod /mnt/root/dev/console c 5 1
mount -o loop /dev /mnt/root/dev
mount -o loop /sys /mnt/root/sys
mount -o loop /proc /mnt/root/proc
mount -o loop /selinux /mnt/root/proc
chroot /mnt/root

First, I made a /dev/root (chrooted, right) with mknod /dev/root b 253,0 to
match the major and minor number of /dev/dm-0.

Then, I remade the initrd, this time with --with=dm_mod --with=dm_crypt
--with=aes-generic --with=blkcipher --with=sha256-generic

Finally, I changed /etc/crypttab from luks-sda2 to root to match how I decrypted
the volume. I suppose a better idea would be to change the final "root" in my
original cryptsetup line to be luks-sda2 to match crypttab, and skip this change.

In any case, the system now boots and I am into the "New Firstboot" screens.

I may blow this up and see if I can repeat this craziness, but take better notes.

Comment 2 Dag 2008-03-07 12:15:09 UTC
confirmed: same error, F9-Alpha hangs on reboot after a default install (all
defaults, encrypted filesystem ON, on a new empty HDD), Ctrl+Alt+Suppr still works.
this message apears:

device-mapper: uevent: version 1.0.3
device-mapper: ioctl: 4.12.0-ioctl (2007-10-02) initialised: dm-devel
 Reading all physical volumes. This may take while...
 No volume group found
 Volume group "VolGroup00" not found
Unable to access resume device (/dev/VolGroup00/LogVol01)
mount: could not find filesystem '/dev/root'
setuproot: moving /dev/failed: no such file or directory
setuproot: error mounting /proc: No such file or directory.
setuproot: error mounting /sys: No such file or directory.
switchroot: mount failed: No such file or directory.

and then nothing ever happens, it hangs. Ctrl+Alt+Del restarts the system.
I used F9-Alpha i386 rescueCD and a net install over an http server.
So the default install does not boot for me.

Comment 3 Steve Rawlinson 2008-03-28 18:04:37 UTC
I'm seeing identical behaviour with F9 beta and no encryption (but otherwise
default options). I'm using a HP proliant DL380 with a smart-array controller.

Comment 4 Jay F 2008-05-06 08:37:26 UTC
Very similar to my experience with both Fedora 8 Live CD and 9 Preview Live CD
(see bug 388901, comment #1). Ubuntu 8.04 Live CD install works fine.

Comment 5 Bug Zapper 2008-05-14 05:02:41 UTC
Changing version to '9' as part of upcoming Fedora 9 GA.
More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Comment 6 c.h. 2008-05-21 12:55:53 UTC
I'm having problems that may be similar:
https://bugzilla.redhat.com/show_bug.cgi?id=447724


Comment 7 Milan Broz 2008-06-05 10:40:55 UTC
Seems that anaconda or mkinitrd created some unfunctional init ramdisk.
LVM/cryptsetup just doesn't see underlying device in the right place, so cannot
initialize properly.

Comment 8 frollic nilsson 2008-06-09 13:20:59 UTC
I had the same issue after a upgrade from FC6 to F9, and managed to solve it by
creating a new initrd image file.

I booted the install CD in rescue mode, after chrooting to my boot disk, I
created a new initrd image using mkinitrd (no params besides version and fil
name were used). 


Comment 9 Andy Lindeberg 2008-06-18 18:15:29 UTC
I did not encounter this with a fresh install of F9 Final. Can you reproduce
this problem with the final release?

Comment 10 Andy Lindeberg 2008-06-18 18:22:34 UTC
@ Comment 8: We don't support upgrading directly from 6 to 9. We highly
recommend that you upgrade from 6 to 7, 7 to 8, and then 8 to 9.

Comment 11 Stephen Gardner 2008-07-03 22:33:20 UTC
My experiences sound similar to several others. I'm not sure whether my issue 
is specifically encryption related or more to do with the usage of an HP 
Proliant with a SmartArray controller. Here's quick rundown of my install. Also 
I tried the same with Rawhide from 2nd July and had that had same result 
however I didn't /dev/zero out the entire disk with the rawhide test which I 
did for this Fedora 9 (Final) test.

Hardware:
HP DL380-G4 (5GB RAM)
(on board) SmartArray 6i with a mirrored 72GB logical drive (OS 
device: /dev/cciss/c0d0)

Installation:
Clean Fedora 9 (Final) x86_64 booting kernel and initrd via PXE with then 
providing an NFS installation path to the ISO
Graphical install (English, UK)
Remove all partitions selected, default LVM / partition layout used with 
Encrypt system checked
password set to:   1234567890ABCDEF     [for testing obviously]

Untick "Office and Productivity" package group, no further customizations (893 
packages installed)
Reboot to the default kernel which fails (as others have found) with

No volume groups found
Volume group "VolGroup00" not found
..
switchroot: mount failed: No such file or directory

Bringing the system up in rescue mode detected the volume group, prompt me for 
the luks password and everything mounted correctly.
On checking the initrd it lacked any of the encryption modules (algorithms or 
dm-crypt), does not include the cryptsetup binary or any of its library 
dependencies (if any are needed) and more fundamentally didn't include the 
cciss driver either. The system root disk did not include 
an  /etc/modprobe.conf   file and none of the files in  /etc/modprobe.conf.d  
included the cciss module. Recreating the initrd (with a straightforward 
mkinitrd + filename + version) as expected did not produce a workable initrd.

The system does include an /etc/crypttab file which contained

luks-c0d0p2             /dev/cciss/c0d0p2 none


Comment 12 Andy Lindeberg 2008-09-30 15:51:13 UTC
Please try this with F10 Beta once it's released and let us know if the problem still exists.

Comment 13 Stephen Gardner 2008-10-01 02:38:50 UTC
I have just tried the Fedora 10 Beta on the same server but unfortunately it no longer correctly identifies the SmartArray 6i controller's logical drive as a valid mass storage device at least from the perspective of the GUI installer. From the installers bash command-line  /dev/cciss/c0d0  is valid, readable and writable but from the GUI and the text mode installer I get

"
An error occurred - no valid devices were found on which to create new file systems. Please check your hardware for the cause of the problem.
"

We'll have to get beyond this more fundamental new issue before we can take up the encrypted root filesystem creation that was originally the subject of this bug. For reference I only tried the x86_64 installer. I also went back and check the Fedora 9 installer and that is fine on this hardware.

System is the same HP Proliant DL380-G4 as mentioned in Comment #11 except it now has 6GB memory
BIOS date: 07/19/2007
SmartArray 6i firmware revision: 2.84 (with 192MB BBWC, and 1 logical 146GB drive)

Comment 14 Eric Paris 2008-10-11 16:35:38 UTC
@Stephen I think you want to look at BZ 442811 as that is probably a small part of your issues...

Comment 15 David Lehman 2008-10-27 19:07:32 UTC
Git commit 3e221c34238dc5cf658da2c11fc32afbcefcc81b, included in mkinitrd-6.0.68, might help with this.

Comment 16 Stephen Gardner 2008-11-04 22:18:07 UTC
Snap-3 had mkinitrd-6.0.67 so that didn't work but I've just had a chance to try the Preview release which includes mkinitrd-6.0.69. Preview installed without any problems, prompted for the password on system startup and the machine booted with its encrypted volumes correctly mounted and working.

I submitted (what turned out to be a duplicate) BZ report for RHEL5-U3 (beta) for the same issue and it sounds like RHEL5-U3 final will also have encrypted volume support working for those of us with SmartArray controllers in ProLiant servers.

Thanks for the fix, I'm glad it'll make F10 final, much appreciated.

Comment 17 David Lehman 2008-12-08 19:47:48 UTC
Sounds like this is resolved. Please reopen if not.

Comment 18 Philipp Kern 2009-02-23 18:02:01 UTC
I have the same problems on Fedora 11 Alpha x86_64 with a new installation through anaconda on a encrypted default partition scheme.

Comment 19 Philipp Kern 2009-02-23 18:18:12 UTC
Sorry, it looks like I did not search carefully enough.  This is #484508.

Comment 20 Israel Ag Nouh Yattara 2009-04-07 04:17:39 UTC
I am also on Fedora 11 Alpha and running into the same bug. Checked all permissions and device assignments in the boot loader configuration.

Defaults were chosen with the Anaconda installer (with a complete disk wipe), but the install is replying with: 

Reading all physical volumes. This may take a while ...
Found volume group "VolGroup" using metadata type lvm2
2 logical volumes(s) in volume group "VolGroup" now active
mount: could not find filesystem 'dev/root'

Any thoughts on this?