Bug 177287 - kernel-2.6.14-1.1653: panic on boot, can't find label ROOT
kernel-2.6.14-1.1653: panic on boot, can't find label ROOT
Status: CLOSED RAWHIDE
Product: Fedora
Classification: Fedora
Component: mkinitrd (Show other bugs)
4
i386 Linux
medium Severity high
: ---
: ---
Assigned To: Peter Jones
David Lawrence
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2006-01-08 21:02 EST by Jeremy Fitzhardinge
Modified: 2007-11-30 17:11 EST (History)
2 users (show)

See Also:
Fixed In Version: 5.0.28-1
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2006-02-27 11:24:01 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
Screenshot of 2.6.14-1.1656_FC4 failing to boot (72.51 KB, image/jpeg)
2006-01-23 01:18 EST, Jeremy Fitzhardinge
no flags Details

  None (edit)
Description Jeremy Fitzhardinge 2006-01-08 21:02:57 EST
Description of problem:
When booting, the system panics because it can't find the ROOT filesystem. 
My system has a pair of 250Gbyte disks which are mirrored with md, and then the
md0 space is managed with lvm.  

Booting verbosely, it seems the problem is because lvm2 has not run, so it
doesn't find the LVM volumes.  When booting previous kernels
(kernel-2.6.14-1.1644_FC4), lvm reports findings its volumes before the system
tries to mount root.

Version-Release number of selected component (if applicable):
kernel-2.6.14-1.1653
kernel-2.6.14-1.1656

How reproducible:
All the time

Steps to Reproduce:
1.Put root filesystem on lvm over md
2.Mount root by label (root=LABEL=ROOT)
3.Boot system
  
Actual results:
Panic failing to mount root filesystem LABEL=ROOT

Expected results:
Booting system

Additional info:
/proc/mdstat:
Personalities : [raid1]
md0 : active raid1 sdc1[2] sdb1[1]
      244195840 blocks [3/2] [_UU]

pvs:
  PV         VG       Fmt  Attr PSize   PFree
  /dev/md0   primary0 lvm2 a-   232.88G 56.16G
  /dev/sda1  scratch  lvm2 a-   232.88G     0
  /dev/sdd5  scratch  lvm2 a-    29.51G 12.39G

/etc/fstab:
LABEL=ROOT              /                       ext3    defaults        1 1
LABEL=HOME              /home                   ext3    defaults        1 2
LABEL=VAR               /var                    ext3    defaults        1 2
LABEL=SPOOL             /var/spool              ext3    defaults        1 2
/dev/primary0/perforce  /var/perforce           ext3    defaults        1 2

LABEL=/boot             /boot                   ext3    defaults        1 2
LABEL=SCRATCH           /scratch                ext3    defaults        1 2
LABEL=SWAP              /swap                   ext3    defaults        1 2
LABEL=BACKUPS           /backups                ext3    defaults        1 2

none                    /dev/pts                devpts  gid=5,mode=620  0 0
none                    /dev/shm                tmpfs   defaults        0 0
none                    /proc                   proc    defaults        0 0
none                    /sys                    sysfs   defaults        0 0

/swap/swap              swap                    swap    defaults        0 0

/dev/fd0                /media/floppy           auto   
pamconsole,exec,noauto,managed 0 0
/dev/hda                /media/cdrecorder       auto   
pamconsole,exec,noauto,managed 0 0


grub.conf:
# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this file
# NOTICE:  You have a /boot partition.  This means that
#          all kernel and initrd paths are relative to /boot/, eg.
#          root (hd0,1)
#          kernel /vmlinuz-version ro root=/dev/primary0/root
#          initrd /initrd-version.img
#boot=/dev/sdc
default=2
timeout=10
splashimage=(hd0,1)/grub/splash.xpm.gz
title Fedora Core (2.6.14-1.1656_FC4)
        root (hd0,1)
        kernel /vmlinuz-2.6.14-1.1656_FC4 ro root=LABEL=ROOT quiet
        initrd /initrd-2.6.14-1.1656_FC4.img
title Fedora Core (2.6.14-1.1653_FC4)
        root (hd0,1)
        kernel /vmlinuz-2.6.14-1.1653_FC4 ro root=LABEL=ROOT quiet
        initrd /initrd-2.6.14-1.1653_FC4.img
title Fedora Core (2.6.14-1.1644_FC4)
        root (hd0,1)
        kernel /vmlinuz-2.6.14-1.1644_FC4 ro root=/dev/primary0/root quiet
        initrd /initrd-2.6.14-1.1644_FC4.img
title Fedora Core (2.6.14-1.1637_FC4)
        root (hd0,1)
        kernel /vmlinuz-2.6.14-1.1637_FC4 ro root=/dev/primary0/root quiet
        initrd /initrd-2.6.14-1.1637_FC4.img
title Memtest86+ (1.55.1)
        root (hd0,1)
        kernel /memtest86+-1.55.1 ro root=/dev/primary0/root quiet
title Dell Utilities
        rootnoverify (hd0,0)
        chainloader +1
Comment 1 Jeremy Fitzhardinge 2006-01-08 21:05:51 EST
Hm, I notice now that the grub.conf entries for the kernels which don't work are
different from the ones which do: root=LABEL=ROOT vs root=/dev/primary0/root. 
In both cases the grub entries are generated during the RPM install, which
suggests the newer RPMs have some problem in determining the correct name for
the root filesystem.
Comment 2 Dave Jones 2006-01-12 19:48:33 EST
sounds like an mkinitrd bug. (or grubby at least).
Comment 3 Jeremy Fitzhardinge 2006-01-23 01:12:33 EST
No, I think there's more to it than that.  It still fails to boot when I use an
explicit root=/dev/primary0/root, even though that works for earlier kernels. 
I'll attach a shot of a boot failure.
Comment 4 Jeremy Fitzhardinge 2006-01-23 01:18:22 EST
Created attachment 123563 [details]
Screenshot of 2.6.14-1.1656_FC4 failing to boot

If I try a verbose boot, everything scrolls away too quickly, but I could try
to capture something if necessary.
Comment 5 Jeremy Fitzhardinge 2006-01-23 01:29:39 EST
OK, on closer inspection, it does look like an initrd problem.  But mkinitrd
itself hasn't changed in a long time: mkinitrd-4.2.15-1 has a built date of 17
May 2005, and I've certainly installed kernels since then.

initrd-2.6.14-1.1656_FC4.img contains no LVM stuff:
$ zcat initrd-2.6.14-1.1656_FC4.img | cpio -itv
drwx------  10 root     root            0 Jan  6 21:32 .
drwxr-xr-x   2 root     root            0 Jan  6 21:32 proc
drwxr-xr-x   2 root     root            0 Jan  6 21:32 lib
-rw-r--r--   1 root     root        24876 Jan  6 21:32 lib/raid1.ko
-rw-r--r--   1 root     root        14984 Jan  6 21:32 lib/sata_promise.ko
-rw-r--r--   1 root     root        22760 Jan  6 21:32 lib/sd_mod.ko
-rw-r--r--   1 root     root       152924 Jan  6 21:32 lib/ext3.ko
-rw-r--r--   1 root     root        72868 Jan  6 21:32 lib/jbd.ko
-rw-r--r--   1 root     root        68084 Jan  6 21:32 lib/dm-mod.ko
-rw-r--r--   1 root     root       170296 Jan  6 21:32 lib/scsi_mod.ko
-rw-r--r--   1 root     root        13148 Jan  6 21:32 lib/ata_piix.ko
-rw-r--r--   1 root     root        56096 Jan  6 21:32 lib/libata.ko
drwxr-xr-x   2 root     root            0 Jan  6 21:32 sys
drwxr-xr-x   2 root     root            0 Jan  6 21:32 sysroot
-rwxr-xr-x   1 root     root         1088 Jan  6 21:32 init
lrwxrwxrwx   1 root     root            3 Jan  6 21:32 sbin -> bin
drwxr-xr-x   2 root     root            0 Jan  6 21:32 dev
crw-r--r--   1 root     root       4,   0 Jan  6 21:32 dev/systty
brw-r--r--   1 root     root       1,   1 Jan  6 21:32 dev/ram
crw-r--r--   1 root     root       4,   3 Jan  6 21:32 dev/tty3
crw-r--r--   1 root     root       4,   1 Jan  6 21:32 dev/tty1
crw-r--r--   1 root     root       4,   2 Jan  6 21:32 dev/tty2
crw-r--r--   1 root     root       5,   1 Jan  6 21:32 dev/console
crw-r--r--   1 root     root       4,   4 Jan  6 21:32 dev/tty4
brw-r-----   1 root     disk       9,   0 Dec 18 10:53 dev/md0
crw-r--r--   1 root     root       1,   3 Jan  6 21:32 dev/null
drwxr-xr-x   2 root     root            0 Jan  6 21:32 loopfs
drwxr-xr-x   3 root     root            0 Jan  6 21:32 etc
drwxr-xr-x   2 root     root            0 Jan  6 21:32 etc/udev
-rw-r--r--   1 root     root          384 Jan  6 21:32 etc/udev/udev.conf
drwxr-xr-x   2 root     root            0 Jan  6 21:32 bin
-rwxr-xr-x   1 root     root       432260 Jan  6 21:32 bin/insmod
lrwxrwxrwx   1 root     root           10 Jan  6 21:32 bin/hotplug -> /sbin/nash
-rwxr-xr-x   1 root     root       464852 Jan  6 21:32 bin/nash
-rwxr-xr-x   1 root     root       594388 Jan  6 21:32 bin/udevstart
lrwxrwxrwx   1 root     root           10 Jan  6 21:32 bin/modprobe -> /sbin/nash
-rwxr-xr-x   1 root     root       594388 Jan  6 21:32 bin/udev

Compared to initrd-2.6.14-1.1644_FC4.img, which works:
$ zcat initrd-2.6.14-1.1644_FC4.img | cpio -itv
drwx------  10 root     root            0 Nov 30 01:39 .
drwxr-xr-x   2 root     root            0 Nov 30 01:39 proc
drwxr-xr-x   2 root     root            0 Nov 30 01:39 lib
-rw-r--r--   1 root     root        27780 Nov 30 01:39 lib/dm-mirror.ko
-rw-r--r--   1 root     root        24876 Nov 30 01:39 lib/raid1.ko
-rw-r--r--   1 root     root        14604 Nov 30 01:39 lib/sata_promise.ko
-rw-r--r--   1 root     root         3900 Nov 30 01:39 lib/dm-zero.ko
-rw-r--r--   1 root     root        22520 Nov 30 01:39 lib/dm-snapshot.ko
-rw-r--r--   1 root     root        22876 Nov 30 01:39 lib/sd_mod.ko
-rw-r--r--   1 root     root       153604 Nov 30 01:39 lib/ext3.ko
-rw-r--r--   1 root     root        73124 Nov 30 01:39 lib/jbd.ko
-rw-r--r--   1 root     root        68608 Nov 30 01:39 lib/dm-mod.ko
-rw-r--r--   1 root     root       170276 Nov 30 01:39 lib/scsi_mod.ko
-rw-r--r--   1 root     root        13180 Nov 30 01:39 lib/ata_piix.ko
-rw-r--r--   1 root     root        55952 Nov 30 01:39 lib/libata.ko
drwxr-xr-x   2 root     root            0 Nov 30 01:39 sys
drwxr-xr-x   2 root     root            0 Nov 30 01:39 sysroot
-rwxr-xr-x   1 root     root         1463 Nov 30 01:39 init
lrwxrwxrwx   1 root     root            3 Nov 30 01:39 sbin -> bin
drwxr-xr-x   2 root     root            0 Nov 30 01:39 dev
crw-r--r--   1 root     root       4,   0 Nov 30 01:39 dev/systty
brw-r--r--   1 root     root       1,   1 Nov 30 01:39 dev/ram
crw-r--r--   1 root     root       4,   3 Nov 30 01:39 dev/tty3
crw-r--r--   1 root     root       4,   1 Nov 30 01:39 dev/tty1
crw-r--r--   1 root     root       4,   2 Nov 30 01:39 dev/tty2
crw-r--r--   1 root     root       5,   1 Nov 30 01:39 dev/console
crw-r--r--   1 root     root       4,   4 Nov 30 01:39 dev/tty4
brw-r-----   1 root     disk       9,   0 Nov 12 10:18 dev/md0
crw-r--r--   1 root     root       1,   3 Nov 30 01:39 dev/null
drwxr-xr-x   2 root     root            0 Nov 30 01:39 loopfs
drwxr-xr-x   4 root     root            0 Nov 30 01:39 etc
drwxr-xr-x   2 root     root            0 Nov 30 01:39 etc/lvm
-rw-r--r--   1 root     root        10394 Nov 30 01:39 etc/lvm/lvm.conf
drwxr-xr-x   2 root     root            0 Nov 30 01:39 etc/udev
-rw-r--r--   1 root     root          384 Nov 30 01:39 etc/udev/udev.conf
drwxr-xr-x   2 root     root            0 Nov 30 01:39 bin
-rwxr-xr-x   1 root     root       432260 Nov 30 01:39 bin/insmod
lrwxrwxrwx   1 root     root           10 Nov 30 01:39 bin/hotplug -> /sbin/nash
-rwxr-xr-x   1 root     root       464852 Nov 30 01:39 bin/nash
-rwxr-xr-x   1 root     root       594388 Nov 30 01:39 bin/udevstart
lrwxrwxrwx   1 root     root           10 Nov 30 01:39 bin/modprobe -> /sbin/nash
-r-xr-xr-x   1 root     root      1207372 Nov 30 01:39 bin/lvm
-rwxr-xr-x   1 root     root       594388 Nov 30 01:39 bin/udev
Comment 6 Jeremy Fitzhardinge 2006-02-05 18:30:32 EST
OK, so the problem is that mkinitrd doesn't include any lvm stuff unless it
determines that root is an LVM device.  If the root device is a label reference,
it thinks it isn't LVM.  The workaround for me is to change the line in fstab from

LABEL=ROOT              /                       ext3    defaults        1 1

to

/dev/primary0/root      /                       ext3    defaults        1 1

With this in place, mkinitrd generates a correct initrd.
Comment 7 Peter Jones 2006-02-27 11:24:01 EST
Should work in current rawhide.

Note You need to log in before you can comment on or make changes to this bug.