Bug 1270228

Summary: Failed to upgrade to RHEV-H7.1 to RHEV-H7.2 via RHEV-M3.5.5
Product: Red Hat Enterprise Virtualization Manager Reporter: Chaofeng Wu <cwu>
Component: ovirt-nodeAssignee: Ryan Barry <rbarry>
Status: CLOSED ERRATA QA Contact: Chaofeng Wu <cwu>
Severity: urgent Docs Contact:
Priority: high    
Version: 3.6.0CC: bmcclain, cshao, cwu, fdeutsch, gklein, huiwa, huzhao, leiwang, lsurette, mgoldboi, yaniwang, ycui, ykaul
Target Milestone: ovirt-3.6.0-rc3Keywords: TestBlocker
Target Release: 3.6.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: ovirt-node-3.3.0-0.18.20151022git82dc52c.el7ev Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-03-09 14:40:19 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Node RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
drop into dracut mode
none
grub screenshot
none
rdsosreport.txt none

Description Chaofeng Wu 2015-10-09 10:33:20 UTC
Description of problem:
Upgrade rhev-hypervisor7-7.1-20150917.0.iso to rhev-hypervisor7-7.2-20150928.0.iso failed via RHEV-M 3.5.5-0.1.el6ev. After upgrade, select RHEV-H 7.2 in the grub, but the system still boot to RHEV-H7.1, and during boot process the system finally drop into dracut mode.


Version-Release number of selected component (if applicable):
rhev-hypervisor7-7.1-20150917.0.iso
rhev-hypervisor7-7.2-20150928.0.iso
rhevm-3.5.5-0.1.el6ev.noarch

How reproducible:
100%

Steps to Reproduce:
1. Install rhev-hypervisor7-7.1-20150917.0.iso and register to RHEV-M 3.5.5-0.1.el6ev.
2. On RHEV-M web portal, set hev-hypervisor7-7.1-20150917 to maintenance, then the upgrade to rhev-hypervisor7-7.2-20150928.0.iso.

Actual results:
After step2, the system boot to RHEV-H7.1 and finally drop into dracut mode.

Expected results:
After step2, the system boot to RHEV-H7.2 successful and can be up on RHEV-M web portal.

Additional info:

Comment 1 Chaofeng Wu 2015-10-09 10:34:21 UTC
Created attachment 1081263 [details]
drop into dracut mode

Comment 2 Chaofeng Wu 2015-10-09 10:35:10 UTC
Created attachment 1081264 [details]
grub screenshot

Comment 3 Ryan Barry 2015-10-09 16:17:57 UTC
Fabian, did this include the reverted mount_live changes?

Chaofeng: I won't be able to look at this until next week. Can you attach an rdsosreport in the meantime?

Comment 4 Fabian Deutsch 2015-10-09 16:53:18 UTC
Yes. 0928 contains ovirt-node-3.3.0-0.10.20150928gite7ee3f1.el7ev which is everything up to the git tag ovirt-node-3.3.0-0.10

Comment 5 Ryan Barry 2015-10-09 16:58:22 UTC
Also, is this reproducible with other upgrade methods?

There were some other bugs about upgrading via the TUI, but none of those dropped to dracut, and the patch which resolves should be part of this build. If you try to upgrade via the TUI, does it also drop to dracut?

Comment 6 Chaofeng Wu 2015-10-10 08:12:01 UTC
Hi Ryan,

Attached rdsosreport.txt.

I also test TUI and cmd upgrade, both of two upgrade scenarios are successful, after upgrade the system can boot up correctly.

Comment 7 Chaofeng Wu 2015-10-10 08:13:14 UTC
Created attachment 1081546 [details]
rdsosreport.txt

Comment 8 Ryan Barry 2015-10-14 19:16:08 UTC
This is caused by the new initramfs generation.

It works as expected in TUI (and probably PXE) upgrades because it's running the new kernel. It does not work from RHEV-M, because RHEV-M generates an initrd for the kernel which is running when upgraded (7.1), and not the newly-installed kernel.

multipathd doesn't start, because dm-multipath can't be loaded. Actually, no modules can be loaded, because they're all for the wrong kernel version. I'm investigating a fix, and should have one out today or tomorrow.

Comment 9 Chaofeng Wu 2015-11-12 13:47:52 UTC
Verified on the rhev-hypervisor7-7.2-20151104 build.

Version-Release number of selected component (if applicable):
ovirt-node-3.6.0-0.20.20151103git3d3779a.el7ev.noarch
rhev-hypervisor7-7.2-20151104.0.iso

Steps:
1. Install rhev-hypervisor7-7.1-20151015.0.iso and register to RHEV-M 3.5.6.2-0.1.el6ev.
2. On RHEV-M web portal, set rhev-hypervisor7-7.1-20151015 to maintenance, then the upgrade to rhev-hypervisor7-7.2-20151104.0.iso.

Result:
Upgrade successful, rhevh status is up on RHEV-M portal.

This bug is fixed, so change the status to VERIFIED.

Comment 11 errata-xmlrpc 2016-03-09 14:40:19 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-0378.html