Bug 539325 - SATA RAID0 - Unbootable after F11->F12 upgrade with F12 kernel, F11 kernel OK
Summary: SATA RAID0 - Unbootable after F11->F12 upgrade with F12 kernel, F11 kernel OK
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: util-linux-ng
Version: 12
Hardware: x86_64
OS: Linux
low
urgent
Target Milestone: ---
Assignee: Karel Zak
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
: 539273 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2009-11-19 21:17 UTC by giles
Modified: 2009-12-04 22:45 UTC (History)
7 users (show)

Fixed In Version: 2.16-10.3.fc12
Doc Type: Bug Fix
Doc Text:
Clone Of: 539273
Environment:
Last Closed: 2009-12-01 04:42:15 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
No block devices found (42.93 KB, image/jpeg)
2009-11-20 16:42 UTC, giles
no flags Details
lsmod (3.77 KB, text/plain)
2009-11-23 10:15 UTC, giles
no flags Details
modprobe (33.32 KB, image/jpeg)
2009-11-23 17:21 UTC, giles
no flags Details
modules (38.77 KB, image/jpeg)
2009-11-23 17:36 UTC, giles
no flags Details
after boot /lib/modules (107.74 KB, image/png)
2009-11-23 17:45 UTC, giles
no flags Details
lsmod from FC11 x86_64 (2.42 KB, text/plain)
2009-11-23 22:31 UTC, enrico.gottardo
no flags Details
dmsetup from FC11 x86_64 (392 bytes, text/plain)
2009-11-23 22:33 UTC, enrico.gottardo
no flags Details
dracut then boot into debug shell (129.94 KB, image/jpeg)
2009-11-24 17:53 UTC, giles
no flags Details
dmraid -l dmraid -s (63.74 KB, image/jpeg)
2009-11-25 17:27 UTC, giles
no flags Details
blkid /dev/sd (147.55 KB, image/jpeg)
2009-11-26 17:03 UTC, giles
no flags Details
via.img (90 bytes, application/x-gzip)
2009-11-27 20:03 UTC, giles
no flags Details

Description giles 2009-11-19 21:17:41 UTC
+++ This bug was initially created as a clone of Bug #539273 +++

Description of problem:
I have a VIA VT8237 SATA RAID controller, RAID-0 on two 250Gb disks. Has been working fine on F11. I upgraded from the commandline F11->F12, after boot I get 'no root device, sleeping forever'.

Rebooting and selecting an older F11 kernel boots OK.

Version-Release number of selected component (if applicable):
old kernel boots OK: 
2.6.30.9-90.fc11.x86_64
new kernel boot fails:
2.6.31.5-127.fc12.x86_64

Also, from the F12 install CD Anaconda reports that the disks must be initialized so no RAID appears not to be detected.

How reproducible:

Steps to Reproduce:
1. upgraded F11->F12 from commandline
2. reboot
  
Actual results:
boot halts 'no root device, sleeping forever'

Expected results:
boots

Additional info:
I read a similar bug id=522076, maybe related?

Here's some commands:

[root@katana giles]# parted /dev/mapper/via_djbccigdhe
GNU Parted 1.9.0
Using /dev/mapper/via_djbccigdhe
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) p                                                                
Model: Linux device-mapper (striped) (dm)
Disk /dev/mapper/via_djbccigdhe: 500GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number  Start   End    Size   Type      File system  Flags
 1      32.3kB  210MB  210MB  primary   ext3         boot
 2      214MB   105GB  105GB  extended
 5      214MB   105GB  105GB  logical                lvm
 3      105GB   315GB  210GB  primary   ext4
 4      315GB   500GB  185GB  primary   ntfs

[giles@katana ~]$ sudo cat /boot/grub/menu.lst
# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this file
# NOTICE:  You have a /boot partition.  This means that
#          all kernel and initrd paths are relative to /boot/, eg.
#          root (hd0,0)
#          kernel /vmlinuz-version ro root=/dev/mapper/VolGroup-lv_root
#          initrd /initrd-version.img
#boot=/dev/via_djbccigdhe
default=2
timeout=15
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
title Fedora (2.6.31.5-127.fc12.x86_64)
	root (hd0,0)
	kernel /vmlinuz-2.6.31.5-127.fc12.x86_64 ro root=/dev/mapping/VolGroup-lv_root
	initrd /initramfs-2.6.31.5-127.fc12.x86_64.img
title Fedora (2.6.31.5-122.fc12.x86_64)
	root (hd0,0)
	kernel /vmlinuz-2.6.31.5-122.fc12.x86_64 ro root=/dev/mapper/VolGroup-lv_root SYSFONT=latarcyrheb-sun16 LANG=en_US.UTF-8 KEYTABLE=uk
	initrd /initramfs-2.6.31.5-122.fc12.x86_64.img
title Fedora-base (2.6.30.9-90.fc11.x86_64)
	root (hd0,0)
	kernel /vmlinuz-2.6.30.9-90.fc11.x86_64 ro root=/dev/mapper/VolGroup-lv_root
	initrd /initrd-2.6.30.9-90.fc11.x86_64.img
title Other
	rootnoverify (hd0,3)
	chainloader +1

[root@katana giles]# blkid -o udev -p /dev/mapper/VolGroup-lv_root 
ID_FS_UUID=7da72cc3-e416-4c95-89eb-99d5934f95f0
ID_FS_UUID_ENC=7da72cc3-e416-4c95-89eb-99d5934f95f0
ID_FS_VERSION=1.0
ID_FS_TYPE=ext4
ID_FS_USAGE=filesystem

[root@katana giles]# rpm -q dmraid
dmraid-1.0.0.rc16-4.fc12.x86_64
[root@katana giles]# rpm -q dracut 
dracut-002-13.4.git8f397a9b.fc12.noarch
[root@katana giles]# rpm -qf /usr/share/dracut/modules.d/* | sort -u  
dracut-002-13.4.git8f397a9b.fc12.noarch

Comment 1 giles 2009-11-19 21:19:16 UTC
*** Bug 539273 has been marked as a duplicate of this bug. ***

Comment 2 Harald Hoyer 2009-11-20 08:09:32 UTC
 root=/dev/mapping/VolGroup-lv_root

are you sure this is correct??? shouldn't it be "mapper" instead of "mapping" ??

Comment 3 giles 2009-11-20 08:46:42 UTC
(In reply to comment #2)
>  root=/dev/mapping/VolGroup-lv_root
> 
> are you sure this is correct??? shouldn't it be "mapper" instead of "mapping"
> ??  

Yes, sorry, a late night typo! It was originally 'mapper'.

Comment 4 Harald Hoyer 2009-11-20 08:53:08 UTC
try to add "rd_NO_MDIMSM"

for debugging add

"rdinfo rdshell"

or

"rdinitdebug rdshell"

you are then dropped to a shell and with rdinitdebug you can inspect "dmesg" messages and also /init.log. 
you might mount /boot by hand and copy over /init.log to attach it here.

Comment 5 giles 2009-11-20 09:04:44 UTC
OK, I added the debug command and dropped to the shell yesterday but wasn't sure where /boot was, I found /dev/sda1, but got this:

mount -t ext3 /dev/sda1 /boot 
ext3-FS no journal found.

There are no devices in /dev/mapper at this point only 'control'. No /dev/dm-x devices either.

Comment 6 Harald Hoyer 2009-11-20 09:43:21 UTC
ah, doh! your /boot is on the raid also...

one thing you can do:

1. recreate the initramfs

# dracut -f /boot/initramfs-2.6.31.5-127.fc12.x86_64.img 2.6.31.5-127.fc12.x86_64

2. boot with "rdshell"
3. get dropped to the shell and run

# dmraid -l
# dmraid -s

4. make a screenshot (camera, mobile phone)
5. attach the photo to this bugzilla

Comment 7 giles 2009-11-20 16:42:33 UTC
Created attachment 372538 [details]
No block devices found

Comment 8 giles 2009-11-20 17:20:42 UTC
Pls see attachment 'no block devices found'

Here's the output of dmraid -s when booting OK with old kernel:
[giles@katana ~]$ sudo dmraid -s
*** Active Set
name   : via_djbccigdhe
size   : 976794112
stride : 128
type   : stripe
status : ok
subsets: 0
devs   : 2
spares : 0

Comment 9 enrico.gottardo 2009-11-21 11:49:48 UTC
I've got the same problem trying to install FC12 x86_64 on a VT8237 BIOS RAID 0 (mirroring) with two 500Gb SATA disks: Anaconda doesn't recognize the HW RAID, allowing to perform the installation only on the two disks in a "standard" configuration (i.e. not-RAID).

I can just add that I didn't face this problem during the installation of FC11 x86_64

Comment 10 Harald Hoyer 2009-11-23 08:37:20 UTC
ok, can you give me the output of 

# lsmod
# dmsetup  ls --tree

from the boot with the old kernel

Comment 11 giles 2009-11-23 10:15:50 UTC
Created attachment 373057 [details]
lsmod

Comment 12 giles 2009-11-23 10:17:01 UTC
dmsetup ls --tree
via_djbccigdhep4 (253:4)
 └─via_djbccigdhe (253:0)
    ├─ (8:16)
    └─ (8:0)
VolGroup-lv_swap (253:7)
 └─via_djbccigdhep5 (253:5)
    └─via_djbccigdhep2 (253:2)
       └─via_djbccigdhe (253:0)
          ├─ (8:16)
          └─ (8:0)
via_djbccigdhep3 (253:3)
 └─via_djbccigdhe (253:0)
    ├─ (8:16)
    └─ (8:0)
VolGroup-lv_root (253:6)
 └─via_djbccigdhep5 (253:5)
    └─via_djbccigdhep2 (253:2)
       └─via_djbccigdhe (253:0)
          ├─ (8:16)
          └─ (8:0)
via_djbccigdhep1 (253:1)
 └─via_djbccigdhe (253:0)
    ├─ (8:16)
    └─ (8:0)

Comment 13 Harald Hoyer 2009-11-23 10:20:48 UTC
- boot with "rdshell"
- get dropped to the shell and run

# ls -l /dev/sd?
# modprobe sata_via
# ls -l /dev/sd?

- make a screenshot (camera, mobile phone)
- attach the photo to this bugzilla

Comment 14 giles 2009-11-23 17:21:54 UTC
Created attachment 373175 [details]
modprobe

Comment 15 giles 2009-11-23 17:36:34 UTC
Created attachment 373183 [details]
modules

Comment 16 giles 2009-11-23 17:45:18 UTC
Created attachment 373186 [details]
after boot /lib/modules

Comment 17 enrico.gottardo 2009-11-23 22:31:44 UTC
Created attachment 373261 [details]
lsmod from FC11 x86_64

Comment 18 enrico.gottardo 2009-11-23 22:33:23 UTC
Created attachment 373262 [details]
dmsetup from FC11 x86_64

Comment 19 Harald Hoyer 2009-11-24 10:49:27 UTC
you created the initramfs with the wrong kernel argument :)

please run (one line)

# dracut -f /boot/initramfs-2.6.31.5-127.fc12.x86_64.img 2.6.31.5-127.fc12.x86_64

this might fix the problem.

Or you have to rerun comment #13

Comment 20 Harald Hoyer 2009-11-24 10:50:14 UTC
# dracut -f \
   /boot/initramfs-2.6.31.5-127.fc12.x86_64.img \
   2.6.31.5-127.fc12.x86_64

Comment 21 giles 2009-11-24 11:35:14 UTC
Hi Harald, Didn't we already do this on comment #6?

Comment 22 Harald Hoyer 2009-11-24 12:33:19 UTC
according to the screenshots with the missing modules, you did it wrong ( forgot the second argument )

Comment 23 giles 2009-11-24 17:22:39 UTC
OK definitely one line now:
dracut -f /boot/initramfs-2.6.31.5-127.fc12.x86_64.img 2.6.31.5-127.fc12.x86_64
W: Possible missing firmware ql8100_fw.bin for module qla2xxx.ko
W: Possible missing firmware ql2500_fw.bin for module qla2xxx.ko
W: Possible missing firmware aic94xx-seq.fw for module aic94xx.ko

Will now repeat comment #13 and attach sshots if there's still a problem...

Comment 24 giles 2009-11-24 17:53:06 UTC
Created attachment 373499 [details]
dracut then boot into debug shell

Comment 25 giles 2009-11-24 18:31:32 UTC
Also tried rd_NO_MDIMSM kernel param and got this before the 'boot failed' message:

"dracut: rd_MD_NO_MDIMSM no MD RAID for imsm/isw raids"

Comment 26 Harald Hoyer 2009-11-25 15:07:20 UTC
hmm, repeating comment #6 with the now correctly build initramfs from comment #23 would be cool.

- boot with "rdshell"
- get dropped to the shell and run

# dmraid -l
# dmraid -s

Comment 27 giles 2009-11-25 17:27:08 UTC
Created attachment 373787 [details]
dmraid -l dmraid -s

Comment 28 Harald Hoyer 2009-11-26 09:46:44 UTC
looks great! should really boot!

if not:

- boot with "rdshell"
- get dropped to the shell and run

# blkid /dev/sda 
# blkid /dev/sdb 
# ls -l /dev/mapper
# lvm vgchange -ay
# ls -l /dev/mapper

Comment 29 Harald Hoyer 2009-11-26 09:47:07 UTC
and of course attach the screenshot, please

Comment 30 giles 2009-11-26 17:03:16 UTC
Created attachment 374038 [details]
blkid /dev/sd

Comment 31 Harald Hoyer 2009-11-26 18:15:58 UTC
oh .. wow.. no output from blkid..

no need to reboot with rdshell.. just from the running system:

# blkid -p /dev/sda
# blkid -p /dev/sdb

Comment 32 giles 2009-11-26 18:25:00 UTC
no output from this either!

# blkid -p /dev/sda
# blkid -p /dev/sdb
#

Comment 33 giles 2009-11-26 18:27:00 UTC
#blkid -p /dev/mapper/via_djbccigdhep5
/dev/mapper/via_djbccigdhep5: UUID="zsWDaI-r90B-medV-djja-dKWo-YkK7-FS7geu" VERSION="LVM2 001" TYPE="LVM2_member" USAGE="raid"

Comment 34 Harald Hoyer 2009-11-27 15:04:13 UTC
reassigning to util-linux-ng for further investigations.

Karel, it seems like blkid does not report the drives as raid members.

Comment 35 Karel Zak 2009-11-27 19:52:36 UTC
I need last few blocks from the device.

Please, from running system (you don't have to reboot with rdshell):

 # dd if=/dev/sda of=~/via.img bs=512 skip=$(($(blockdev --getsz /dev/sda)-10)) count=10

 # gzip ~/via.img

and send me the final ~/via.img.gz (or create an attachment to this BZ).

Comment 36 giles 2009-11-27 20:02:14 UTC
# dd if=/dev/sda of=~/via.img bs=512 skip=$(($(blockdev --getsz /dev/sda)-10)) count=10
10+0 records in
10+0 records out
5120 bytes (5.1 kB) copied, 0.000313716 s, 16.3 MB/s

Comment 37 giles 2009-11-27 20:03:01 UTC
Created attachment 374297 [details]
via.img

Comment 38 Fedora Update System 2009-11-30 11:38:50 UTC
util-linux-ng-2.16-10.3.fc12 has been submitted as an update for Fedora 12.
http://admin.fedoraproject.org/updates/util-linux-ng-2.16-10.3.fc12

Comment 39 Karel Zak 2009-11-30 11:46:03 UTC
Giles, thanks for the test image. The problem should be fixed in util-linux-ng-2.16-10.3.fc12.

Comment 40 giles 2009-11-30 19:38:39 UTC
Bit confused, I haven't installed the new util-linux-ng yet.

I enabled the test repo and updated thinking that the util-linux-ng update was in there.

Machine booted OK (to cmnd line at least) with new kernel: 2.6.31.6-145.fc12.x86_64

..but util-linux-ng still on 2.16-10.2.fc12

In dmesg dracut output showed something suspicious:

dracut: Scanning for dmraid devices 
dracut: Found dmraid sets:
dracut: via_djbccigdhe
dracut: Activating via_djbccigdhe
dracut: The dynamic shared library "libdmraid-events-via.so" could not be loaded:
dracut: libdmraid-events-via.so: cannot open shared object file: No such file or directory
dracut: RAID set "via_djbccigdhe" was activated
dracut: Scanning devices dm-5  for LVM volume groups 
dracut: Reading all physical volumes. This may take a while...
dracut: Found volume group "VolGroup" using metadata type lvm2
dracut: 2 logical volume(s) in volume group "VolGroup" now active

Comment 41 Fedora Update System 2009-12-01 04:42:05 UTC
util-linux-ng-2.16-10.3.fc12 has been pushed to the Fedora 12 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 42 enrico.gottardo 2009-12-01 17:45:47 UTC
I'm sorry if this is out of the topic, but i don't know where else i can ask, so here it goes..

I have a FC11 x86_64 installed on a on a VT8237 BIOS RAID with two SATA disks and everything works just fine. Problem is, when I try to install FC12 from DVD on the same machine, Anaconda does not detect the BIOS RAID.

Thank for any suggestion you may provide.

Comment 43 giles 2009-12-02 12:03:00 UTC
Hi Enrico Maybe do what I did, i.e. an online upgrade FC11->FC12 once the fixes are included in the main repo. You might have to wait longer to get an updated DVD.

Comment 44 enrico.gottardo 2009-12-03 17:23:32 UTC
Thanks Giles, I'll do that: can't wait for FC13..

Comment 45 enrico.gottardo 2009-12-04 22:45:21 UTC
(In reply to comment #41)
> util-linux-ng-2.16-10.3.fc12 has been pushed to the Fedora 12 stable
> repository.  If problems still persist, please make note of it in this bug
> report.  

Unfortunately util-linux-ng-2.16-10.3.fc12 does not fix the problem with me (I made up a DVD.iso with the updated package: during the installation anaconda still does not detect the VT 8327 BIOS RAID)


Note You need to log in before you can comment on or make changes to this bug.