Bug 1546285

Summary: OSP10->11->12->13 undercloud upgrade fails because of failed dependencies between python-$component-tests packages and the component packages
Product: Red Hat OpenStack Reporter: Marius Cornea <mcornea>
Component: python-zaqar-tests-tempestAssignee: Chandan Kumar <chkumar>
Status: CLOSED ERRATA QA Contact: Martin Kopec <mkopec>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 13.0 (Queens)CC: apevec, ccamacho, chkumar, cschwede, dbecker, dpeacock, jschluet, knylande, lhh, markmc, mbracho, mbultel, mburns, morazi, rhel-osp-director-maint, sclewis, slinaber, srevivo, udesale
Target Milestone: betaKeywords: Triaged
Target Release: 13.0 (Queens)Flags: mbracho: needinfo+
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: python-zaqar-tests-tempest-0.0.1-0.20180323062755.f6c761d.el7ost python-keystone-tests-tempest-0.0.1-0.2.564670bgit.el7ost python-telemetry-tests-tempest-0.0.1-0.2.11277a8git.el7ost python-cinder-tests-tempest-0.0.1-1.0.aafbdf8git.el7ost Doc Type: Bug Fix
Doc Text:
Upgrades were having dependencies issues because the collection of tempest plugins were extracted from openstack-*-tests rpm subpackages during the Queens cycle. However, not all of the packaging had the right combination of Provides and Obsoletes. OSP 13 does not have the -tests (unittest sub-rpms). When attempting to do upgrades with -tests installed from prior release cause failures due to dependencies issues. To correct this issue, the Obsoletes for the older version of the -tests rpms they were extracted from have been added back.
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-06-27 13:44:45 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
undercloud_upgrade_13.log
none
yum.log none

Description Marius Cornea 2018-02-16 19:38:16 UTC
Description of problem:
OSP10->11->12->13 undercloud upgrade fails because of failed dependencies between python-$component-tests packages and the component packages:


--> Finished Dependency Resolution
Error: Package: 1:python-cinder-tests-11.0.1-2.el7ost.noarch (@rhelosp-12.0-puddle)
           Requires: openstack-cinder = 1:11.0.1-2.el7ost
           Removing: 1:openstack-cinder-11.0.1-2.el7ost.noarch (@rhelosp-12.0-puddle)
               openstack-cinder = 1:11.0.1-2.el7ost
           Updated By: 1:openstack-cinder-12.0.0-0.20180122233816.71b869c.el7ost.noarch (rhelosp-13.0-puddle)
               openstack-cinder = 1:12.0.0-0.20180122233816.71b869c.el7ost
Error: Package: python-oslo-db-tests-4.25.0-1.el7ost.noarch (@rhelosp-12.0-puddle)
           Requires: python-oslo-db = 4.25.0-1.el7ost
           Removing: python-oslo-db-4.25.0-1.el7ost.noarch (@rhelosp-12.0-puddle)
               python-oslo-db = 4.25.0-1.el7ost
           Obsoleted By: python2-oslo-db-4.33.0-0.20180117091609.ea1ec64.el7ost.noarch (rhelosp-13.0-puddle)
               python-oslo-db = 4.33.0-0.20180117091609.ea1ec64.el7ost
Error: Package: 1:python-ironic-tests-9.1.2-3.el7ost.noarch (@rhelosp-12.0-puddle)
           Requires: openstack-ironic-common = 1:9.1.2-3.el7ost
           Removing: 1:openstack-ironic-common-9.1.2-3.el7ost.noarch (@rhelosp-12.0-puddle)
               openstack-ironic-common = 1:9.1.2-3.el7ost
           Updated By: 1:openstack-ironic-common-10.0.1-0.20180123173432.2c630d9.el7ost.noarch (rhelosp-13.0-puddle)
               openstack-ironic-common = 1:10.0.1-0.20180123173432.2c630d9.el7ost
Error: Package: 1:python-neutron-lbaas-tests-11.0.1-8.el7ost.noarch (@rhelosp-12.0-puddle)
           Requires: python-neutron-lbaas = 1:11.0.1-8.el7ost
           Removing: 1:python-neutron-lbaas-11.0.1-8.el7ost.noarch (@rhelosp-12.0-puddle)
               python-neutron-lbaas = 1:11.0.1-8.el7ost
           Updated By: 1:python-neutron-lbaas-12.0.0-0.20180123055730.0dc985e.el7ost.noarch (rhelosp-13.0-puddle)
               python-neutron-lbaas = 1:12.0.0-0.20180123055730.0dc985e.el7ost
Error: Package: 1:python-neutron-tests-11.0.1-8.el7ost.noarch (@rhelosp-12.0-puddle)
           Requires: python-neutron = 1:11.0.1-8.el7ost
           Removing: 1:python-neutron-11.0.1-8.el7ost.noarch (@rhelosp-12.0-puddle)
               python-neutron = 1:11.0.1-8.el7ost
           Updated By: 1:python-neutron-12.0.0-0.20180123043113.d32ad6e.el7ost.noarch (rhelosp-13.0-puddle)
               python-neutron = 1:12.0.0-0.20180123043113.d32ad6e.el7ost
Error: Package: 1:python-heat-tests-9.0.1-3.el7ost.noarch (@rhelosp-12.0-puddle)
           Requires: openstack-heat-common = 1:9.0.1-3.el7ost
           Removing: 1:openstack-heat-common-9.0.1-3.el7ost.noarch (@rhelosp-12.0-puddle)
               openstack-heat-common = 1:9.0.1-3.el7ost
           Updated By: 1:openstack-heat-common-10.0.0-0.20180123134729.85a3e98.el7ost.noarch (rhelosp-13.0-puddle)
               openstack-heat-common = 1:10.0.0-0.20180123134729.85a3e98.el7ost
 You could try using --skip-broken to work around the problem
 You could try running: rpm -Va --nofiles --nodigest


Version-Release number of selected component (if applicable):
2018-02-14.1 build

Steps to Reproduce:
1. Upgrade undercloud from 10->11->12->13

Actual results:
Upgrade fails

Expected results:
Upgrade succeeds.

Additional info:

Comment 2 Jon Schlueter 2018-02-16 20:27:12 UTC
Sounds like related issues to splitting out the tempest plugins that happened during Queens.

Comment 13 Martin Kopec 2018-04-17 11:50:57 UTC
The dependency issue was resolved by patches tracked above as external trackers.

The packages:
 - python2-zaqar-tests-tempest-0.0.1-0.20180323062755.f6c761d.el7ost
 - python2-telemetry-tests-tempest-0.0.1-0.20180322143259.05b95d2.el7ost

are available in RHOS13's repository.

Tested on RHOS12 environments with 
 - core puddle version = 2018-04-04.2 
 - core puddle version = 2018-03-29.1 
where upgrade to RHOS13 worked without any problems. Based on that, the dependency issue was resolved and therefor marking this bug as VERIFIED.

If you encountered this issue again, don't hesitate to reopen this bug.

Comment 14 Marius Cornea 2018-04-19 00:47:06 UTC
For some reason I am still able to reproduce this issue during OSP13 undercloud upgrade but *only* when starting the initial deployment with OSP10 z4. Attaching the undercloud upgrade log showing the package conflict and yum.log

Comment 15 Marius Cornea 2018-04-19 00:47:31 UTC
Created attachment 1423833 [details]
undercloud_upgrade_13.log

Comment 16 Marius Cornea 2018-04-19 00:48:53 UTC
Created attachment 1423834 [details]
yum.log

Comment 17 Marius Cornea 2018-04-19 01:40:06 UTC
I think I know why I am seeing this error only in this scenario when I start deployment with OSP10 z4. The workflow is the following:

1/ Install OSP10 z4
2/ Minor update to OSP10 latest
3/ Run tempest
4/ Upgrade undercloud to 11/12/13

3/ Manually(infrared) installs the tests packages: https://github.com/redhat-openstack/infrared/blob/master/plugins/tempest/vars/setup/rpm.yml#L11-L23

According to the product docs[1] this is the way to run the validation so I think we still have an issue here that can only be seen when the *-tests packages have been installed on OSP10.

[1] https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/10/html-single/openstack_integration_test_suite_guide/#sec-install-tempest-packages

Comment 19 Carlos Camacho 2018-04-25 09:25:53 UTC
Yes, we are still hitting this.

Comment 21 Jon Schlueter 2018-05-02 14:13:20 UTC
python-telemetry-tests-tempest-0.0.1-0.2.dbd432egit.el7ost - has fix

Comment 22 Jon Schlueter 2018-05-02 14:18:26 UTC
correction python-telemetry-tests-tempest-0.0.1-0.2.11277a8git.el7ost

Comment 23 Jon Schlueter 2018-05-02 14:28:12 UTC
python-cinder-tests-tempest-0.0.1-1.0.aafbdf8git.el7ost - has fix

Comment 24 Jon Schlueter 2018-05-02 14:34:22 UTC
python-keystone-tests-tempest-0.0.1-0.2.564670bgit.el7ost - has fix

Comment 26 Martin Kopec 2018-05-09 07:12:28 UTC
I tested it yesterday and it's VERIFIED.

Tested on RHOS12 with core_puddle_version=2018-04-04.2
Installed tempest plugins:
$ sudo yum install python-glance-tests python-keystone-tests python-horizon-tests-tempest python-neutron-tests python-cinder-tests python-nova-tests python-swift-tests python-ceilometer-tests python-gnocchi-tests python-aodh-tests

Then the upgrade to RHOS13 was successful.

Comment 28 errata-xmlrpc 2018-06-27 13:44:45 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2018:2086