Bug 901786

Summary:

After upgrade to Fedora 18 using FedUp, system does not find any LVM volumes and fails to boot

Product:

[Fedora] Fedora

Reporter:

Carl A. Adams <carlalex>

Component:

kernel

Assignee:

Kernel Maintainer List <kernel-maint>

Status:

CLOSED INSUFFICIENT_DATA

QA Contact:

Fedora Extras Quality Assurance <extras-qa>

Severity:

high

Docs Contact:

Priority:

unspecified

Version:

CC:

agk, bmarzins, bmr, carlalex, donkishoot, dwysocha, gansalmon, heinzm, itamar, jonathan, kernel-maint, l4coa3fnjplr, lvm-team, madhu.chinakonda, msnitzer, prajnoha, prockai, sam, zkabelac

Target Milestone:

---

Target Release:

---

Hardware:

x86_64

OS:

Linux

Whiteboard:

Fixed In Version:

Doc Type:

Bug Fix

Doc Text:

Story Points:

---

Clone Of:

Environment:

Last Closed:

2013-04-22 14:24:17 UTC

Type:

Bug

Regression:

---

Mount Type:

---

Documentation:

---

CRM:

Verified Versions:

Category:

---

oVirt Team:

---

RHEL 7.3 requirements from Atomic Host:

Cloudforms Team:

---

Target Upstream Version:

Embargoed:

Attachments:

Description	Flags
dmesg with errors.	none

Description Carl A. Adams 2013-01-19 07:51:40 UTC

Description of problem:
After following the instructions at http://fedoraproject.org/wiki/FedUp on an up-to-date Fedora 17 system, Fedora 18 does not boot. No errors were reported during the FedUp preparation steps. Network sources were used for installation packages, per the recommendations.

Version-Release number of selected component (if applicable):
Unknown. System does not boot.

How reproducible:
100% (Only one system upgrade attempted)

Steps to Reproduce:
1. on Fedora 17 x86_64, "yum update" system, and "yum install fedup"
2. fedup-cli --network 18 --debuglog fedupdebug.log
3. reboot
4. select upgrade from grub menu
5. system upgrades (I left the room)
6. system reboots into F18
7. F18 fails to come up.

Actual results:
Fedora 18 fails to boot up fully. Recovery mode also fails to boot. It appears to be some sort of LVM problem with the 3.7.X kernel or initrd that was installed. The following error messages appear at the end of the failure:
dracut-initqueue[252]: Warning: Could not boot.
dracut-initqueue[252]: /dev/mapper/vg_barad--dur-lv_root does not exist
dracut-initqueue[252]: /dev/vg_barad-dur/lv_root does not exist
dracut-initqueue[252]: /dev/vg_barad-dur/lv_swap does not exist
After these messages, I am dropped into a recovery shell

Expected results:
Fedora 18 should boot up with default kernel.

Additional info:

- The older Fedora 17 kernels (3.6.11) do still boot up just fine and allow me to log in, so I don't think the system is trashed, just that the F18 boot env can't find itself. The user space environment appears to be properly upgraded to F18.

- There does not appear to be any significant difference to the kernel command line parameters for the 3.6 and 3.7 kernels.

- Going through the grub upgrade steps referenced on the FedUp wiki page does not help. (I had to do this from the F17 3.6.11 booted environment)

- From the recovery shell (F18 kernel), I do not see my LVM physical or logical volumes using the lvm tool. /dev/mapper appears to be empty.

Comment 1 Carl A. Adams 2013-01-19 07:56:30 UTC

The only thing slightly weird about this system that I can think of is I have VMWare Workstation installed; but I don't see any obvious indication that this is related.

Comment 2 Carl A. Adams 2013-01-19 22:29:44 UTC

Google'd around. I attempted to regenerate the initrd with dracut per defect #827651. This did not help

Comment 3 Carl A. Adams 2013-01-19 22:42:21 UTC

Working on another hunch. I moved the disk from the internal SATA-III controller to the SATA-II controller.  This fixed my issue, and the 3.7.X kernels now boot the system.   As my disk is only a SATA-II, this really doesn't matter that much for me. 

The unrecognized SATA-III controller is a Marvell controller.  The working SATA-II controller is Intel ICH10.  Motherboard is ASUS Rampage III Gene LGA 1366 Intel X58 (http://www.newegg.com/Product/Product.aspx?Item=N82E16813131658)

There are other messages from the kernel at boot-up, and there system now has a ~30 second delay near the start of the boot sequence it used to not have.  Lots of complaining about IOMMU and broken bios.   dmesg attached.

Comment 4 Carl A. Adams 2013-01-19 22:43:10 UTC

Created attachment 683323 [details]
dmesg with errors.

Comment 5 Peter Rajnoha 2013-01-21 08:26:45 UTC

(In reply to comment #3)
> Working on another hunch. I moved the disk from the internal SATA-III
> controller to the SATA-II controller.  This fixed my issue, and the 3.7.X
> kernels now boot the system.

This does not seem to be an LVM problem, reassigning the component to kernel.

> There are other messages from the kernel at boot-up, and there system now
> has a ~30 second delay near the start of the boot sequence it used to not
> have.  Lots of complaining about IOMMU and broken bios.   dmesg attached.

Also:

[    6.226167] ata14.00: qc timeout (cmd 0xa1)
[    6.725113] ata14.00: failed to IDENTIFY (I/O error, err_mask=0x4)
[    7.532664] ata14: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
[   17.517131] ata14.00: qc timeout (cmd 0xa1)
[   18.017343] ata14.00: failed to IDENTIFY (I/O error, err_mask=0x4)
[   18.017349] ata14: limiting SATA link speed to 1.5 Gbps
[   18.824216] ata14: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
[   48.779540] ata14.00: qc timeout (cmd 0xa1)
[   49.279642] ata14.00: failed to IDENTIFY (I/O error, err_mask=0x4)
[   50.085527] ata14: SATA link up 1.5 Gbps (SStatus 113 SControl 310)

Comment 6 Neil 2013-02-10 01:44:06 UTC

I have this problem to, how I can fix it?

Comment 7 ogburton 2013-03-02 16:52:56 UTC

This appears to be an SELinux issue.  It is not that the LVM is gone, it is that the system no longer has permission to use it.  On my system the entire /dev directory had no security context.  Everything was labelled with a ?

1. On first reboot after Fed Up a GUI login prompt is presented, but users can't login.  Even root can't login either at the GUI or at a CTRL-ALT-F2 console prompt.

2. On second reboot after Fed Up attempting to boot at init 3 the boot aborts and dumps the user to dracut# after reporting that the root logical volume is not present.  From there I could see that it was in fact present, but that it had a security context of ?

3. On the third reboot after Fed Up I set selinux=0 enforcing=0 3 in GRUB to come up to a console where I could in fact login as root.

4. On the fourth reboot after Fed Up I set selinux=1 and waited for the system to relabel itself.

5. On the fifth reboot after Fed Up and selinux relabel the system came up and runs as it should.

Comment 8 Sam 2013-03-05 01:11:18 UTC

I don't think it's only an SELinux issue.  I have the same problem on my old Dell PowerEdge 2450, and adding selinux=0 enforcing=0 (I stop at level 3 on this system, console only) I still had the same problem.  And I always have SELinux disabled back several Fedora versions.

But earlier in the boot I get:

aic7xxx: no multichannel peer found!

Well, the root partition is on the AIC-7899 SCSI drive.  So I'm thinking that the aic7xxx module isn't getting loaded properly in the fedup boot, it really can't see the root partition.

The machine boots fine into Fedora 17, and did before into Fedora 16.  I'm not really sharp on grub2 yet, so I don't know how to force aic7xxx to be preloaded like when I can roll my own initrd in /boot with mkinitrd.  I can't make a custom initrd in /boot for fedup because fedup doesn't put any modules under /lib for the Fedora 18 kernel.

Caught in a trap - don't have the module I need to make an initrd so I can boot, but I need the module preloaded to boot up.  It seems.  Or maybe it's something else, but the aic7xxx error early in the boot looks ominous.

Comment 9 Marcel 2013-03-22 11:09:49 UTC

You can go further but it seems not to be a solution.

While on dracut cli do :
# blkid
check if you see some volume
# lvm vgscan
# blkid
check if you see your VG
# lvm vgchange -ay
# blkid
check if you see your logical volumes

Next you exit (#exit) dracut and fedup continue the upgrade process but it seems not to boot after :(

Comment 10 Marcel 2013-03-22 14:07:06 UTC

After you will probably have another bug where you will have to delete in /boot/grub2/grub.cfg the end of the linux boot line (all behind KEYTABLE= parameter)

Comment 11 Josh Boyer 2013-04-03 16:46:23 UTC

Are you still seeing this with the 3.8.5 kernel?

Comment 12 Josh Boyer 2013-04-22 14:24:17 UTC

This bug is being closed with INSUFFICIENT_DATA as there has not been a
response in 2 weeks.  If you are still experiencing this issue,
please reopen and attach the relevant data from the latest kernel you are
running and any data that might have been requested previously.

Comment 13 Sam 2013-04-22 14:44:51 UTC

I gave up on this. I saw no way to get the machine to boot up under Fedup when it can't see the AIC-7899 SCSI drive. I'm sitting on Fedora 17 and will probably do a full install of Fedora 18 sometime, if I care enough.