Bug 1393639

Summary: Upgrade from wrapper to wrapper failed
Product: [oVirt] ovirt-node Reporter: Qin Yuan <qiyuan>
Component: Installation & UpdateAssignee: Ryan Barry <rbarry>
Status: CLOSED CURRENTRELEASE QA Contact: Qin Yuan <qiyuan>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 4.0CC: bgraveno, bmcclain, bugs, cshao, dguo, emahoney, fdeutsch, huzhao, jiawu, lbopf, leiwang, mgoldboi, mkalinin, pbrilla, qiyuan, rbarry, weiwang, yaniwang, ycui, yzhao
Target Milestone: ovirt-4.0.5Flags: rule-engine: ovirt-4.0.z+
rule-engine: blocker+
bmcclain: planning_ack+
fdeutsch: devel_ack+
cshao: testing_ack+
Target Release: 4.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: imgbased-0.8.9-0.1.el7ev Doc Type: Known Issue
Doc Text:
Due to a problem with the filesystem layout in early versions of Red Hat Virtualization Host 4.0, subsequent upgrades will fail with a traceback from yum. For example: "Error: Package tuple ('redhat-virtualization-host-image-update', 'noarch', '0', '4.0', '20160919.0.el7_2') could not be found in rpmdb" To resolve this issue, /var/lib/yum has been relocated. However, users performing a second upgrade from Red Hat Virtualization Host 4.0 GA or 4.0.4 must run "yum clean all" or "yum clean rpmdb" before upgrading to Red Hat Virtualization Host 4.0.5.
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-01-18 07:37:51 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Node RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Attachments:
Description Flags
Error Messages printed on the screen
none
/var/log*,/tmp/*,sosreport none

Description Qin Yuan 2016-11-10 03:26:26 UTC
Created attachment 1219163 [details]
Error Messages printed on the screen

Description of problem:
Upgrade from wrapper to wrapper, the 3 builds are as followed:
Build1: redhat-virtualization-host-4.0-20160817.0
Build2: redhat-virtualization-host-4.0-20160919.0
Build3: redhat-virtualization-host-4.0-20161107.0 (the latest build)

Upgrade from Build1 to Build2 succeeded, but upgrade from Build2 to Build3 failed. The error message is :

"Error: Package tuple ('redhat-virtualization-host-image-update', 'noarch', '0', '4.0', '20160919.0.el7_2') could not be found in rpmdb"


Version-Release number of selected component (if applicable):
Build1:
redhat-virtualization-host-4.0-20160817.0
imgbased-0.8.4-0.1.el7ev.noarch
Build2:
redhat-virtualization-host-4.0-20160919.0
imgbased-0.8.5-0.1.el7ev.noarch
Build3:
redhat-virtualization-host-4.0-20161107.0
imgbased-0.8.7-0.1.el7ev.noarch


How reproducible:
100%
Not regression bug
Keywords: Upgrade


Steps to Reproduce:
1. Install Build1.
2. Download *update*.rpm of Build2, that is redhat-virtualization-host-image-update-4.0-20160919.0.el7_2.noarch.rpm.
3. Upgrade to Build2, using "yum install *update*.rpm".
4. Reboot and login in Build2.
5. Upgrade to the latest Build3, using "yum update". 


Actual results:
After step5, there is an error occurred, and the upgrading process is stopped. 

Expected results:
After step5, upgrade to Build3 successfully. 


Additional info:
Don't use wrapper to wrapper upgrading, upgrade from Build1 to Build3 or from Build2 to Build3 directly, both can succeed.

Comment 1 Qin Yuan 2016-11-10 03:28:48 UTC
Created attachment 1219164 [details]
/var/log*,/tmp/*,sosreport

Comment 2 Fabian Deutsch 2016-11-10 13:06:25 UTC
How reproducible is this?

Comment 3 Pavol Brilla 2016-11-10 17:17:53 UTC
I was able to reproduce it on 2 machines, both of them are still with it, ping me tomorrow if you want to check them

Comment 4 Ryan Barry 2016-11-10 23:46:21 UTC
For verification:

Please check that /var/lib/yum is a symlink to /usr/share/yum on redhat-virtualization-host-4.0-20161110.0

Comment 6 Ying Cui 2016-11-11 01:25:20 UTC
(In reply to Fabian Deutsch from comment #2)
> How reproducible is this?

How reproducible:
100%

Comment 7 Qin Yuan 2016-11-11 07:55:53 UTC
Test versions:
Build1:
redhat-virtualization-host-4.0-20160817.0
imgbased-0.8.4-0.1.el7ev.noarch
Build2:
redhat-virtualization-host-4.0-20160919.0
imgbased-0.8.5-0.1.el7ev.noarch
Build3(the latest build):
redhat-virtualization-host-4.0-20161110.0
imgbased-0.8.8-0.1.el7ev.noarch

Test steps:
1. Install Build1.
2. Download *update*.rpm of Build2, that is redhat-virtualization-host-image-update-4.0-20160919.0.el7_2.noarch.rpm.
3. Upgrade to Build2, using "yum install *update*.rpm".
4. Reboot and login to Build2.
5. Execute "yum clean all"
6. Upgrade to the latest Build3, using "yum update". 
7. Reboot and login to Build3.
8. check /var/lib/yum


Test results:
1. Using "yum clean all" can make wrapper to wrapper upgrading from Build1 to Build2 to Build3 succeed.
2. /var/lib/yum is not a symlink to /usr/share/yum on redhat-virtualization-host-4.0-20161110.0 after the wrapper to wrapper upgrading.

Additional info:
/var/lib/yum is a symlink to /usr/share/yum on initial installed redhat-virtualization-host-4.0-20161110.0

Comment 8 cshao 2016-11-11 08:52:42 UTC
(In reply to Pavol Brilla from comment #3)
> I was able to reproduce it on 2 machines, both of them are still with it,
> ping me tomorrow if you want to check them

Hi Pavol,

I'd like to ask you reproduced the fail via RHVM or yum?

Thanks.

Comment 9 cshao 2016-11-11 11:49:16 UTC
Hi Ryan,


Upgrade from wrapper to wrapper in rhvm side still got failed.

Test versions:
Build1:
redhat-virtualization-host-4.0-20160817.0
imgbased-0.8.4-0.1.el7ev.noarch

Build2:
redhat-virtualization-host-4.0-20160919.0
imgbased-0.8.5-0.1.el7ev.noarch

Build3(the latest build):
redhat-virtualization-host-4.0-20161110.0
imgbased-0.8.8-0.1.el7ev.noarch


Step to reproduce:
1. install build1 (edhat-virtualization-host-4.0-20160817.0 )
2. keep "rhvh.repo" in /etc/yum.repos.d/ and download redhat-virtualization-host-4.0-20160919.0.rpm
3. yum install *.rpm and finish upgrade to build2
4. reboot to bulid2
5. in rhevm side, make host to maintanence status  and then click upgrade button to upgrade
6. monitor the  /var/log/yum.log (tail -f  /var/log/yum.log) on host and focus on event on rhvm UI

Results:
After step 5, it fail to upgrade from rhvm side.

Cancel the needinfo due to QE can reproduce this issue via RHVM too.
Thanks.

Comment 10 Ryan Barry 2016-11-11 13:12:32 UTC
I suppose that relocating by itself is not good enough, and osupdater needs to move it on existing systems.

Comment 11 Qin Yuan 2016-11-14 10:33:05 UTC
The "Test versions" and "Test steps" are the same as Comment7, except the version of Buid3 is:
redhat-virtualization-host-4.0-20161111.0
imgbased-0.8.9-0.1.el7ev.noarch

Test results:
1. Wrapper to wrapper upgrading succeeded using "yum clean all".

2. /var/lib/yum is a symlink to /usr/share/yum on redhat-virtualization-host-4.0-20161111.0 after the wrapper to wrapper upgrading.

3. /var/lib/yum is a symlink to /usr/share/yum on initial installed redhat-virtualization-host-4.0-20161111.0

So the bug is fixed, change bug status to VERIFIED.