Bug 504961 - unable to resolve LABEL=/boot
unable to resolve LABEL=/boot
Status: CLOSED NOTABUG
Product: Fedora
Classification: Fedora
Component: mkinitrd (Show other bugs)
11
i686 Linux
low Severity high
: ---
: ---
Assigned To: Peter Jones
Fedora Extras Quality Assurance
: Reopened
: 505291 507142 508042 (view as bug list)
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2009-06-10 04:28 EDT by Vilius Šumskas
Modified: 2009-06-26 09:43 EDT (History)
10 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2009-06-15 03:25:47 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Vilius Šumskas 2009-06-10 04:28:29 EDT
Description of problem: after upgradeing F10 to F11 one of my servers can't find /boot partition anymore. It stalls when /boot partition should be mounted and checked with fsck with an error "unable to resolve LABEL=/boot". My /boot partition is on /dev/sda1 and I have checked /dev folder for it. It is nowhere to be found, even though /dev/sda and /dev/sda2 exist. fdisk -l /dev/sda shows sda1 and sda2. DMESG during boot also shows that partition. Also I have tried partprobe which makes sda1 appear again. And I even can mount it. But as soon as I reboot sda1 cannot be mounted again.

Version-Release number of selected component (if applicable):
2.6.29.4-167.fc11.i686.PAE

How reproducible:
Always.

Actual results:
The system can't find sda1.

Expected results:
It should find it.

Additional info:
I'm not really sure that this is related to kernel, because I have tried older kernels installed on the system and it behaves the same.
Comment 1 Chuck Ebbert 2009-06-10 20:50:04 EDT
If the kernel is finding the partition this is probably a udev bug.
Comment 2 Sami Knuutinen 2009-06-11 08:00:48 EDT
I have exactly the same problem after F10->F11 upgrade on x86_64 platform.
Comment 3 Vilius Šumskas 2009-06-11 08:16:54 EDT
As this is related to udev, maybe hardware will be relevant. I'm using HP DL320 server (i386) with 2 IDE disks on fake raid integrated controller.
Comment 4 Harald Hoyer 2009-06-11 13:06:42 EDT
no udev involved in initramfs
Comment 5 Hans de Goede 2009-06-11 13:25:06 EDT
(In reply to comment #4)
> no udev involved in initramfs  

Erm, no initramfs involved in finding and mounting /boot, only in finding and mounting /.

So to get this straight,   Vilius Šumskas, the system does get to the:
"Starting Fedora" prompt and then when it tries to fsck the local systems it goes boom and drops you in a root shell to repair, or ... ?

Also what does the kernel say when scanning the disk, it normally prints something like this when scanning for partitions:

sda: sda1 sda2

What does that line from dmesg say on the system in question ?
Comment 6 Sami Knuutinen 2009-06-11 14:06:41 EDT
My system boots ok after I uncomment the sda1 and sda2 from /etc/fstab. Kernel sees all the partitions during boot, on my system the dmesg has line
sda: sda1 sda2 sda3

After the boot only the /dev/sda3 exists:
$ sudo ls -al /dev/sda*
brw-rw---- 1 root disk 8, 0 11.6. 10:56 /dev/sda
brw-rw---- 1 root disk 8, 3 11.6. 10:56 /dev/sda3

When the system is booted if I do partprobe or start and close gparted, then after that all the sda* entries are there:
$ sudo ls -al /dev/sda*
brw-rw---- 1 root disk 8, 0 11.6. 10:56 /dev/sda
brw-rw---- 1 root disk 8, 1 11.6. 20:58 /dev/sda1
brw-rw---- 1 root disk 8, 2 11.6. 20:58 /dev/sda2
brw-rw---- 1 root disk 8, 3 11.6. 10:56 /dev/sda3


In /var/log/messages there is:
---
Jun 11 10:56:35 jerry kernel: device-mapper: multipath: version 1.0.5 loaded
Jun 11 10:56:35 jerry kernel: device-mapper: table: 253:0: linear: dm-linear: Device lookup failed
Jun 11 10:56:35 jerry kernel: device-mapper: ioctl: error adding target to table
Jun 11 10:56:35 jerry kernel: EXT3 FS on sda3, internal journal
---
Could the "Device lookup failed" and "error adding target to table" be related to the problem?
Comment 7 Hans de Goede 2009-06-11 15:01:27 EDT
Hmm,

Can you please run:
/lib/udev/vol_id /dev/sda2
/lib/udev/vol_id /dev/sda3

And paste the output here ?
Comment 8 Sami Knuutinen 2009-06-12 01:18:08 EDT
The following output is right after reboot so you can see I had to do the partprobe to get the sda1 and sda2 to be available.

$ sudo /lib/udev/vol_id /dev/sda1 
[sudo] password for sknuutin: 
/dev/sda1: error opening volume
$ sudo /lib/udev/vol_id /dev/sda2
/dev/sda2: error opening volume
$ sudo /lib/udev/vol_id /dev/sda3
ID_FS_USAGE=filesystem
ID_FS_TYPE=ext3
ID_FS_VERSION=1.0
ID_FS_UUID=1de56595-7e16-42d4-a4a3-16534123e7a1
ID_FS_UUID_ENC=1de56595-7e16-42d4-a4a3-16534123e7a1
ID_FS_LABEL=/
ID_FS_LABEL_ENC=\x2f
$ sudo partprobe
$ sudo /lib/udev/vol_id /dev/sda1 
ID_FS_USAGE=filesystem
ID_FS_TYPE=ext3
ID_FS_VERSION=1.0
ID_FS_UUID=7658311a-b73e-41bc-85f4-de2d4e1ad715
ID_FS_UUID_ENC=7658311a-b73e-41bc-85f4-de2d4e1ad715
ID_FS_LABEL=/boot
ID_FS_LABEL_ENC=\x2fboot
$ sudo /lib/udev/vol_id /dev/sda2
ID_FS_USAGE=other
ID_FS_TYPE=swap
ID_FS_VERSION=2
ID_FS_UUID=869f399d-9b28-4c82-b182-adb962ee469e
ID_FS_UUID_ENC=869f399d-9b28-4c82-b182-adb962ee469e
ID_FS_LABEL=SWAP-sda2
ID_FS_LABEL_ENC=SWAP-sda2
$ sudo /lib/udev/vol_id /dev/sda3
ID_FS_USAGE=filesystem
ID_FS_TYPE=ext3
ID_FS_VERSION=1.0
ID_FS_UUID=1de56595-7e16-42d4-a4a3-16534123e7a1
ID_FS_UUID_ENC=1de56595-7e16-42d4-a4a3-16534123e7a1
ID_FS_LABEL=/
ID_FS_LABEL_ENC=\x2f
$
Comment 9 Hans de Goede 2009-06-12 02:35:57 EDT
Hmm, nothing weird there, I was thinking maybe your partitions were getting seen
as dmraid members, but they are not.

Can you directly after reboot (so while missing the /dev/sda1 and 2) do:
ls /sys/block/sda

And paste the output ?
Comment 10 Sami Knuutinen 2009-06-12 02:57:21 EDT
After reboot only sda3 exists in /sys/block/sda. When partprobe is run then also the sda1 and sda2 will appear there.

# ls /sys/block/sda
bdi         device     power  removable  size    subsystem
capability  ext_range  queue  ro         slaves  uevent
dev         holders    range  sda3       stat
# partprobe 
# ls /sys/block/sda
bdi         device     power  removable  sda2  slaves     uevent
capability  ext_range  queue  ro         sda3  stat
dev         holders    range  sda1       size  subsystem
Comment 11 Hans de Goede 2009-06-12 03:17:43 EDT
(In reply to comment #10)
> After reboot only sda3 exists in /sys/block/sda. When partprobe is run then
> also the sda1 and sda2 will appear there.
> 
> # ls /sys/block/sda
> bdi         device     power  removable  size    subsystem
> capability  ext_range  queue  ro         slaves  uevent
> dev         holders    range  sda3       stat
> # partprobe 
> # ls /sys/block/sda
> bdi         device     power  removable  sda2  slaves     uevent
> capability  ext_range  queue  ro         sda3  stat
> dev         holders    range  sda1       size  subsystem  

Ok,

So the kernel does not know about sda1 and sda2 after boot, weird. Could you try
booting with "nodmraid" on the kernel cmdline?

And if that helps run "dmraid -ay -vvv " and paste the output here ?
Comment 12 Sami Knuutinen 2009-06-12 03:33:57 EDT
The "nodmraid" on the kernel cmdline fixes the problem.

# dmraid -ay -vvv
WARN: locking /var/lock/dmraid/.lock
NOTICE: /dev/sdc: asr     discovering
NOTICE: /dev/sdc: ddf1    discovering
NOTICE: /dev/sdc: hpt37x  discovering
NOTICE: /dev/sdc: hpt45x  discovering
NOTICE: /dev/sdc: isw     discovering
NOTICE: /dev/sdc: jmicron discovering
NOTICE: /dev/sdc: lsi     discovering
NOTICE: /dev/sdc: nvidia  discovering
NOTICE: /dev/sdc: pdc     discovering
NOTICE: /dev/sdc: sil     discovering
NOTICE: /dev/sdc: via     discovering
NOTICE: /dev/sdb: asr     discovering
NOTICE: /dev/sdb: ddf1    discovering
NOTICE: /dev/sdb: hpt37x  discovering
NOTICE: /dev/sdb: hpt45x  discovering
NOTICE: /dev/sdb: isw     discovering
NOTICE: /dev/sdb: jmicron discovering
NOTICE: /dev/sdb: lsi     discovering
NOTICE: /dev/sdb: nvidia  discovering
NOTICE: /dev/sdb: pdc     discovering
NOTICE: /dev/sdb: sil     discovering
NOTICE: /dev/sdb: via     discovering
NOTICE: /dev/sda: asr     discovering
NOTICE: /dev/sda: ddf1    discovering
NOTICE: /dev/sda: hpt37x  discovering
NOTICE: /dev/sda: hpt45x  discovering
NOTICE: /dev/sda: isw     discovering
NOTICE: /dev/sda: jmicron discovering
NOTICE: /dev/sda: lsi     discovering
NOTICE: /dev/sda: nvidia  discovering
NOTICE: /dev/sda: pdc     discovering
NOTICE: /dev/sda: pdc metadata discovered
NOTICE: /dev/sda: sil     discovering
NOTICE: /dev/sda: via     discovering
NOTICE: added /dev/sda to RAID set "pdc_bdccfbiiaj"
RAID set "pdc_bdccfbiiaj" was not activated
WARN: unlocking /var/lock/dmraid/.lock
Comment 13 Hans de Goede 2009-06-12 03:52:19 EDT
(In reply to comment #12)
> The "nodmraid" on the kernel cmdline fixes the problem.
> 

I already was afraid that would be the case.

> # dmraid -ay -vvv

<snip>

> NOTICE: /dev/sda: pdc metadata discovered
> NOTICE: /dev/sda: sil     discovering
> NOTICE: /dev/sda: via     discovering
> NOTICE: added /dev/sda to RAID set "pdc_bdccfbiiaj"
> RAID set "pdc_bdccfbiiaj" was not activated
> WARN: unlocking /var/lock/dmraid/.lock  

Hmm, so it looks like your disk (once) was  part of a raidset
on a promise sata controller and when you re-used it you did not
first use the promise BIOS to remove the metadata.

Or are you still using raid on a promise controller, and are we failing to
bring the set up?

What you can try is to run:
dmraid -x

Which will scrub the pdc raid metadata from your disk. Warning this should
be ok, but better backup first!


Some explanation:
What is happening is that dmraid when it sees a raidset tells the kernel to
forget about discovered partitions on the raw disk, as we don't want applications to be accessing the raw partitions, but only their devicemapper
raid representations (so that data to a mirror gets correctly written to both disks).

This is not happening for sda3, as that is already in use at the time, so the
kernel cannot forget about it.
Comment 14 Sami Knuutinen 2009-06-12 04:48:14 EDT
Very good, the disc really was part of a raidset couple of years ago and I hadn't removed the metadata. With previous Fedora versions I hadn't noticed any problems. 

I removed the raid metadata with dmraid -r -E and now everything works good. All the sda* entries are available also without the nodmraid line on kernel cmdline.


Thank you, I really appreciate all the help I got solving the issue!
Comment 15 Hans de Goede 2009-06-12 04:57:40 EDT
(In reply to comment #14)
> Very good, the disc really was part of a raidset couple of years ago and I
> hadn't removed the metadata. With previous Fedora versions I hadn't noticed any
> problems. 
> 
> I removed the raid metadata with dmraid -r -E and now everything works good.
> All the sda* entries are available also without the nodmraid line on kernel
> cmdline.
> 
> 
> Thank you, I really appreciate all the help I got solving the issue!  

You're welcome, closing this now.
Comment 16 Harald Hoyer 2009-06-12 08:44:37 EDT
*** Bug 505291 has been marked as a duplicate of this bug. ***
Comment 17 Vilius Šumskas 2009-06-13 08:08:36 EDT
Sadly this doesn't help me.

[root@tekila ~]# dmraid -ay -vvv
WARN: locking /var/lock/dmraid/.lock
NOTICE: /dev/sdb: asr     discovering
NOTICE: /dev/sdb: ddf1    discovering
NOTICE: /dev/sdb: hpt37x  discovering
NOTICE: /dev/sdb: hpt45x  discovering
NOTICE: /dev/sdb: isw     discovering
NOTICE: /dev/sdb: jmicron discovering
NOTICE: /dev/sdb: lsi     discovering
NOTICE: /dev/sdb: lsi metadata discovered
NOTICE: /dev/sdb: nvidia  discovering
NOTICE: /dev/sdb: pdc     discovering
NOTICE: /dev/sdb: sil     discovering
NOTICE: /dev/sdb: via     discovering
NOTICE: /dev/sda: asr     discovering
NOTICE: /dev/sda: ddf1    discovering
NOTICE: /dev/sda: hpt37x  discovering
NOTICE: /dev/sda: hpt45x  discovering
NOTICE: /dev/sda: isw     discovering
NOTICE: /dev/sda: jmicron discovering
NOTICE: /dev/sda: lsi     discovering
NOTICE: /dev/sda: lsi metadata discovered
NOTICE: /dev/sda: nvidia  discovering
NOTICE: /dev/sda: pdc     discovering
NOTICE: /dev/sda: sil     discovering
NOTICE: /dev/sda: via     discovering
NOTICE: added /dev/sdb to RAID set "lsi_bhjhcichfca"
NOTICE: added /dev/sda to RAID set "lsi_bhjhcichfca"
RAID set "lsi_bhjhcichfca" was not activated
WARN: unlocking /var/lock/dmraid/.lock

[root@tekila ~]# dmraid -x

About to delete RAID set lsi_bhjhcichfca

WARNING: The metadata stored on the raidset(s) will not be accessible after deletion
Do you want to continue ? [y/n] :y
ERROR: Raid set deletion is not supported in "lsi" format


I have also tried dmraid -r -E but then my fake raid controller resets to RAID0 and I can no longer boot at all. GRUB just hangs at state1.

My sda1 is /boot partition, and sda2 and sdb1 are mirror partitions for LVM2 root volume.
Comment 18 Hans de Goede 2009-06-15 03:25:47 EDT
(In reply to comment #17)
> Sadly this doesn't help me.
> 

First of all please don't re-open unrelated bugs. The original bug was about
someone who had sta;e  motherboard raid metadata on his disk, which was no longer valid, where as if I understand you correctly, you actually have a complete
motherboard / fake raid set, and we've failed to identify it as such,
which is a *completely* different issue. So please file a new bug for this.

When filing a new bug, can you please start the F-11 installer (you don't need to actually make any changes to your system), press next at the welcome
screen you get to the screen where it shows the disks it has found and
write down which disks it sees, include this in the new report.

Also switch to tty2 (ctrl + alt + f2) and save the /tmp/anaconda.log
and /tmp/storage.log files somewhere from there (use scp for example) and attach
those to the new bug report.

Suggested summary for the new bug report: "F11 does not recognize lsi
fakeraid set"

Thanks!
Comment 19 Vilius Šumskas 2009-06-15 10:30:58 EDT
Well, original report was submitted by me, and I thought I can reopen the bug report even if someone fixed their similar bug in the process.

Anyway, here you go: https://bugzilla.redhat.com/show_bug.cgi?id=506091
Comment 20 Hans de Goede 2009-06-15 10:54:01 EDT
(In reply to comment #19)
> Well, original report was submitted by me, and I thought I can reopen the bug
> report even if someone fixed their similar bug in the process.
> 
> Anyway, here you go: https://bugzilla.redhat.com/show_bug.cgi?id=506091  

Ah, sorry, I thought Sami Knuutinen was the original reporter, my bad. Anyways
best to discuss this further in the new bug, thanks!
Comment 21 Harald Hoyer 2009-06-26 04:59:49 EDT
*** Bug 508042 has been marked as a duplicate of this bug. ***
Comment 22 Harald Hoyer 2009-06-26 05:03:12 EDT
*** Bug 507142 has been marked as a duplicate of this bug. ***

Note You need to log in before you can comment on or make changes to this bug.