Bugzilla will be upgraded to version 5.0 on a still to be determined date in the near future. The original upgrade date has been delayed.
Bug 1546285 - OSP10->11->12->13 undercloud upgrade fails because of failed dependencies between python-$component-tests packages and the component packages
OSP10->11->12->13 undercloud upgrade fails because of failed dependencies bet...
Status: CLOSED ERRATA
Product: Red Hat OpenStack
Classification: Red Hat
Component: python-zaqar-tests-tempest (Show other bugs)
13.0 (Queens)
Unspecified Unspecified
urgent Severity urgent
: beta
: 13.0 (Queens)
Assigned To: Chandan Kumar
Martin Kopec
: Triaged
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2018-02-16 14:38 EST by Marius Cornea
Modified: 2018-06-27 09:45 EDT (History)
20 users (show)

See Also:
Fixed In Version: python-zaqar-tests-tempest-0.0.1-0.20180323062755.f6c761d.el7ost python-keystone-tests-tempest-0.0.1-0.2.564670bgit.el7ost python-telemetry-tests-tempest-0.0.1-0.2.11277a8git.el7ost python-cinder-tests-tempest-0.0.1-1.0.aafbdf8git.el7ost
Doc Type: Bug Fix
Doc Text:
Upgrades were having dependencies issues because the collection of tempest plugins were extracted from openstack-*-tests rpm subpackages during the Queens cycle. However, not all of the packaging had the right combination of Provides and Obsoletes. OSP 13 does not have the -tests (unittest sub-rpms). When attempting to do upgrades with -tests installed from prior release cause failures due to dependencies issues. To correct this issue, the Obsoletes for the older version of the -tests rpms they were extracted from have been added back.
Story Points: ---
Clone Of:
Environment:
Last Closed: 2018-06-27 09:44:45 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
mbracho: needinfo+


Attachments (Terms of Use)
undercloud_upgrade_13.log (82.92 KB, text/plain)
2018-04-18 20:47 EDT, Marius Cornea
no flags Details
yum.log (115.26 KB, text/plain)
2018-04-18 20:48 EDT, Marius Cornea
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
RDO 12581 None rpm-master: MERGED openstack/telemetry-tempest-plugin-distgit: Added obsoletes to older telemetry tempest plugins (I1a337c83ac6e5434579e887... 2018-05-02 11:00 EDT
RDO 12918 None rpm-master: MERGED openstack/neutron-tempest-plugin-distgit: Obsolete python-neutron-tests from pike and older (Ib6a87267a9cfc8110f2c289be2... 2018-05-02 11:00 EDT
RDO 12957 None rpm-master: MERGED openstack/heat-tempest-plugin-distgit: Added obsoletes to older heat tempest plugin (I6088c82db7e5ec642ccc488b27ed293ec5... 2018-05-02 11:00 EDT
RDO 12958 None rpm-master: MERGED openstack/ironic-tempest-plugin-distgit: Added obsoletes to older ironic-{, inspector} tempest plugin (I4f96ca247f738309... 2018-05-02 11:00 EDT
RDO 12959 None rpm-master: MERGED openstack/mistral-tempest-plugin-distgit: Added obsoletes to older mistral tempest plugin (Ie1c6de3e1c731fb817aa0d7c0c43... 2018-05-02 11:00 EDT
RDO 12960 None rpm-master: MERGED openstack/zaqar-tempest-plugin-distgit: Added obsoletes to older zaqar tempest plugin (I610cb50cfc09bff8e0ccbfcaadd69961... 2018-05-02 11:00 EDT
RDO 12961 None rpm-master: MERGED openstack/manila-tempest-plugin-distgit: Added obsoletes to older manila tempest plugin (I750b52169d601712a71b8bba9ae19c... 2018-05-02 11:00 EDT
RDO 12962 None rpm-master: MERGED openstack/octavia-tempest-plugin-distgit: Added obsoletes to older Octavia tempest plugin (I702f66876a3311786bbd119fb5a6... 2018-05-02 10:59 EDT
RDO 12963 None rpm-master: MERGED openstack/ec2api-tempest-plugin-distgit: Added obsoletes to older ec2api tempest plugin (I3be81077dd79e8c8977c184765b83c... 2018-05-02 10:59 EDT
RDO 12985 None queens-rdo: MERGED openstack/neutron-tempest-plugin-distgit: Obsolete python-neutron-tests from pike and older (Ib6a87267a9cfc8110f2c289be2... 2018-05-02 10:59 EDT
RDO 13018 None queens-rdo: MERGED openstack/octavia-tempest-plugin-distgit: Added obsoletes to older Octavia tempest plugin (I702f66876a3311786bbd119fb5a6... 2018-05-02 10:59 EDT
RDO 13019 None queens-rdo: MERGED openstack/telemetry-tempest-plugin-distgit: Added obsoletes to older telemetry tempest plugins (I1a337c83ac6e5434579e887... 2018-05-02 10:59 EDT
RDO 13020 None queens-rdo: MERGED openstack/manila-tempest-plugin-distgit: Added obsoletes to older manila tempest plugin (I750b52169d601712a71b8bba9ae19c... 2018-05-02 10:59 EDT
RDO 13021 None queens-rdo: MERGED openstack/cinder-tempest-plugin-distgit: Added obsoletes to older cinder tempest plugin (I2add79a77d401c0ad88acfb35795cb... 2018-05-02 10:59 EDT
RDO 13022 None queens-rdo: MERGED openstack/ec2api-tempest-plugin-distgit: Added obsoletes to older ec2api tempest plugin (I3be81077dd79e8c8977c184765b83c... 2018-05-02 10:59 EDT
RDO 13023 None queens-rdo: MERGED openstack/heat-tempest-plugin-distgit: Added obsoletes to older heat tempest plugin (I6088c82db7e5ec642ccc488b27ed293ec5... 2018-05-02 10:59 EDT
RDO 13024 None queens-rdo: MERGED openstack/ironic-tempest-plugin-distgit: Added obsoletes to older ironic-{, inspector} tempest plugin (I4f96ca247f738309... 2018-05-02 10:59 EDT
RDO 13025 None rpm-master: MERGED openstack/keystone-tempest-plugin-distgit: Added obsoletes to older keystone tempest plugin (Id62743bf73bda8ce00c7d9f65c... 2018-05-02 10:58 EDT
RDO 13026 None queens-rdo: MERGED openstack/keystone-tempest-plugin-distgit: Added obsoletes to older keystone tempest plugin (Id62743bf73bda8ce00c7d9f65c... 2018-05-02 10:58 EDT
RDO 13027 None queens-rdo: MERGED openstack/mistral-tempest-plugin-distgit: Added obsoletes to older mistral tempest plugin (Ie1c6de3e1c731fb817aa0d7c0c43... 2018-05-02 10:58 EDT
RDO 13028 None queens-rdo: MERGED openstack/zaqar-tempest-plugin-distgit: Added obsoletes to older zaqar tempest plugin (I610cb50cfc09bff8e0ccbfcaadd69961... 2018-05-02 10:58 EDT
RDO 13613 None rpm-master: MERGED openstack/keystone-tempest-plugin-distgit: Use Epoch in obsoletes to fix upgrade issue of tempest plugin (Ie99f36f1f6f2f... 2018-05-02 10:58 EDT
RDO 13614 None rpm-master: MERGED openstack/telemetry-tempest-plugin-distgit: Use Epoch in obsoletes to fix upgrade issue of tempest plugin (I64a03375884e... 2018-05-02 10:58 EDT
RDO 13615 None rpm-master: MERGED openstack/cinder-tempest-plugin-distgit: Use Epoch in obsoletes to fix upgrade issue of tempest plugin (I3d04b07506957b1... 2018-05-02 10:58 EDT
RDO 13616 None queens-rdo: NEW openstack/cinder-tempest-plugin-distgit: Use Epoch in obsoletes to fix upgrade issue of tempest plugin (I3d04b07506957b1... 2018-05-02 10:58 EDT
RDO 13617 None queens-rdo: MERGED openstack/telemetry-tempest-plugin-distgit: Use Epoch in obsoletes to fix upgrade issue of tempest plugin (I64a03375884e... 2018-05-02 10:58 EDT
RDO 13618 None queens-rdo: MERGED openstack/keystone-tempest-plugin-distgit: Use Epoch in obsoletes to fix upgrade issue of tempest plugin (Ie99f36f1f6f2f... 2018-05-02 10:57 EDT
Red Hat Product Errata RHEA-2018:2086 None None None 2018-06-27 09:45 EDT

  None (edit)
Description Marius Cornea 2018-02-16 14:38:16 EST
Description of problem:
OSP10->11->12->13 undercloud upgrade fails because of failed dependencies between python-$component-tests packages and the component packages:


--> Finished Dependency Resolution
Error: Package: 1:python-cinder-tests-11.0.1-2.el7ost.noarch (@rhelosp-12.0-puddle)
           Requires: openstack-cinder = 1:11.0.1-2.el7ost
           Removing: 1:openstack-cinder-11.0.1-2.el7ost.noarch (@rhelosp-12.0-puddle)
               openstack-cinder = 1:11.0.1-2.el7ost
           Updated By: 1:openstack-cinder-12.0.0-0.20180122233816.71b869c.el7ost.noarch (rhelosp-13.0-puddle)
               openstack-cinder = 1:12.0.0-0.20180122233816.71b869c.el7ost
Error: Package: python-oslo-db-tests-4.25.0-1.el7ost.noarch (@rhelosp-12.0-puddle)
           Requires: python-oslo-db = 4.25.0-1.el7ost
           Removing: python-oslo-db-4.25.0-1.el7ost.noarch (@rhelosp-12.0-puddle)
               python-oslo-db = 4.25.0-1.el7ost
           Obsoleted By: python2-oslo-db-4.33.0-0.20180117091609.ea1ec64.el7ost.noarch (rhelosp-13.0-puddle)
               python-oslo-db = 4.33.0-0.20180117091609.ea1ec64.el7ost
Error: Package: 1:python-ironic-tests-9.1.2-3.el7ost.noarch (@rhelosp-12.0-puddle)
           Requires: openstack-ironic-common = 1:9.1.2-3.el7ost
           Removing: 1:openstack-ironic-common-9.1.2-3.el7ost.noarch (@rhelosp-12.0-puddle)
               openstack-ironic-common = 1:9.1.2-3.el7ost
           Updated By: 1:openstack-ironic-common-10.0.1-0.20180123173432.2c630d9.el7ost.noarch (rhelosp-13.0-puddle)
               openstack-ironic-common = 1:10.0.1-0.20180123173432.2c630d9.el7ost
Error: Package: 1:python-neutron-lbaas-tests-11.0.1-8.el7ost.noarch (@rhelosp-12.0-puddle)
           Requires: python-neutron-lbaas = 1:11.0.1-8.el7ost
           Removing: 1:python-neutron-lbaas-11.0.1-8.el7ost.noarch (@rhelosp-12.0-puddle)
               python-neutron-lbaas = 1:11.0.1-8.el7ost
           Updated By: 1:python-neutron-lbaas-12.0.0-0.20180123055730.0dc985e.el7ost.noarch (rhelosp-13.0-puddle)
               python-neutron-lbaas = 1:12.0.0-0.20180123055730.0dc985e.el7ost
Error: Package: 1:python-neutron-tests-11.0.1-8.el7ost.noarch (@rhelosp-12.0-puddle)
           Requires: python-neutron = 1:11.0.1-8.el7ost
           Removing: 1:python-neutron-11.0.1-8.el7ost.noarch (@rhelosp-12.0-puddle)
               python-neutron = 1:11.0.1-8.el7ost
           Updated By: 1:python-neutron-12.0.0-0.20180123043113.d32ad6e.el7ost.noarch (rhelosp-13.0-puddle)
               python-neutron = 1:12.0.0-0.20180123043113.d32ad6e.el7ost
Error: Package: 1:python-heat-tests-9.0.1-3.el7ost.noarch (@rhelosp-12.0-puddle)
           Requires: openstack-heat-common = 1:9.0.1-3.el7ost
           Removing: 1:openstack-heat-common-9.0.1-3.el7ost.noarch (@rhelosp-12.0-puddle)
               openstack-heat-common = 1:9.0.1-3.el7ost
           Updated By: 1:openstack-heat-common-10.0.0-0.20180123134729.85a3e98.el7ost.noarch (rhelosp-13.0-puddle)
               openstack-heat-common = 1:10.0.0-0.20180123134729.85a3e98.el7ost
 You could try using --skip-broken to work around the problem
 You could try running: rpm -Va --nofiles --nodigest


Version-Release number of selected component (if applicable):
2018-02-14.1 build

Steps to Reproduce:
1. Upgrade undercloud from 10->11->12->13

Actual results:
Upgrade fails

Expected results:
Upgrade succeeds.

Additional info:
Comment 2 Jon Schlueter 2018-02-16 15:27:12 EST
Sounds like related issues to splitting out the tempest plugins that happened during Queens.
Comment 13 Martin Kopec 2018-04-17 07:50:57 EDT
The dependency issue was resolved by patches tracked above as external trackers.

The packages:
 - python2-zaqar-tests-tempest-0.0.1-0.20180323062755.f6c761d.el7ost
 - python2-telemetry-tests-tempest-0.0.1-0.20180322143259.05b95d2.el7ost

are available in RHOS13's repository.

Tested on RHOS12 environments with 
 - core puddle version = 2018-04-04.2 
 - core puddle version = 2018-03-29.1 
where upgrade to RHOS13 worked without any problems. Based on that, the dependency issue was resolved and therefor marking this bug as VERIFIED.

If you encountered this issue again, don't hesitate to reopen this bug.
Comment 14 Marius Cornea 2018-04-18 20:47:06 EDT
For some reason I am still able to reproduce this issue during OSP13 undercloud upgrade but *only* when starting the initial deployment with OSP10 z4. Attaching the undercloud upgrade log showing the package conflict and yum.log
Comment 15 Marius Cornea 2018-04-18 20:47 EDT
Created attachment 1423833 [details]
undercloud_upgrade_13.log
Comment 16 Marius Cornea 2018-04-18 20:48 EDT
Created attachment 1423834 [details]
yum.log
Comment 17 Marius Cornea 2018-04-18 21:40:06 EDT
I think I know why I am seeing this error only in this scenario when I start deployment with OSP10 z4. The workflow is the following:

1/ Install OSP10 z4
2/ Minor update to OSP10 latest
3/ Run tempest
4/ Upgrade undercloud to 11/12/13

3/ Manually(infrared) installs the tests packages: https://github.com/redhat-openstack/infrared/blob/master/plugins/tempest/vars/setup/rpm.yml#L11-L23

According to the product docs[1] this is the way to run the validation so I think we still have an issue here that can only be seen when the *-tests packages have been installed on OSP10.

[1] https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/10/html-single/openstack_integration_test_suite_guide/#sec-install-tempest-packages
Comment 19 Carlos Camacho 2018-04-25 05:25:53 EDT
Yes, we are still hitting this.
Comment 21 Jon Schlueter 2018-05-02 10:13:20 EDT
python-telemetry-tests-tempest-0.0.1-0.2.dbd432egit.el7ost - has fix
Comment 22 Jon Schlueter 2018-05-02 10:18:26 EDT
correction python-telemetry-tests-tempest-0.0.1-0.2.11277a8git.el7ost
Comment 23 Jon Schlueter 2018-05-02 10:28:12 EDT
python-cinder-tests-tempest-0.0.1-1.0.aafbdf8git.el7ost - has fix
Comment 24 Jon Schlueter 2018-05-02 10:34:22 EDT
python-keystone-tests-tempest-0.0.1-0.2.564670bgit.el7ost - has fix
Comment 26 Martin Kopec 2018-05-09 03:12:28 EDT
I tested it yesterday and it's VERIFIED.

Tested on RHOS12 with core_puddle_version=2018-04-04.2
Installed tempest plugins:
$ sudo yum install python-glance-tests python-keystone-tests python-horizon-tests-tempest python-neutron-tests python-cinder-tests python-nova-tests python-swift-tests python-ceilometer-tests python-gnocchi-tests python-aodh-tests

Then the upgrade to RHOS13 was successful.
Comment 28 errata-xmlrpc 2018-06-27 09:44:45 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2018:2086

Note You need to log in before you can comment on or make changes to this bug.