Bug 1466345 - Cannot upgrade OSP11 to OSP12: Overcloud upgrade failed due to dependency issue for openstack-dashboard-theme
Summary: Cannot upgrade OSP11 to OSP12: Overcloud upgrade failed due to dependency iss...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: python-django-horizon
Version: 12.0 (Pike)
Hardware: x86_64
OS: Linux
urgent
high
Target Milestone: beta
: 12.0 (Pike)
Assignee: Radomir Dopieralski
QA Contact: Ido Ovadia
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-06-29 13:13 UTC by Artem Hrechanychenko
Modified: 2018-02-05 19:10 UTC (History)
14 users (show)

Fixed In Version: python-django-horizon-12.0.0-0.20170821142112.el7ost.1 openstack-dashboard-theme-12.0.0-2.el7ost.1
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-12-13 21:35:30 UTC
Target Upstream Version:
Embargoed:
tvignaud: needinfo+


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2017:3462 0 normal SHIPPED_LIVE Red Hat OpenStack Platform 12.0 Enhancement Advisory 2018-02-16 01:43:25 UTC

Description Artem Hrechanychenko 2017-06-29 13:13:44 UTC
Description of problem:
On controller node:
--> Processing Dependency: openstack-dashboard = 1:11.0.1-5.el7ost for package: 1:openstack-dashboard-theme-11.0.1-5.el7ost.noarch
---> Package python-html5lib.noarch 1:0.999-6.el7ost will be installed
--> Finished Dependency Resolution
Error: Package: 1:openstack-dashboard-theme-11.0.1-5.el7ost.noarch (@rhos-11.0-signed)
           Requires: openstack-dashboard = 1:11.0.1-5.el7ost
           Removing: 1:openstack-dashboard-11.0.1-5.el7ost.noarch (@rhos-11.0-signed)
               openstack-dashboard = 1:11.0.1-5.el7ost
           Updated By: 1:openstack-dashboard-12.0.0-0.20170624124155.e8d1dd9.el7ost.noarch (rhelosp-12.0-puddle)
               openstack-dashboard = 1:12.0.0-0.20170624124155.e8d1dd9.el7ost
 You could try using --skip-broken to work around the problem
 You could try running: rpm -Va --nofiles --nodigest


Version-Release number of selected component (if applicable):
OSP12

openstack-nova-api-16.0.0-0.20170624031428.3863eca.el7ost.noarch
openstack-keystone-12.0.0-0.20170623233743.9070172.el7ost.noarch
openstack-ironic-inspector-5.1.1-0.20170622234613.05896b5.el7ost.noarch
puppet-openstack_extras-11.2.0-0.20170613085321.0b0ea62.el7ost.noarch
openstack-tripleo-common-containers-7.1.1-0.20170623115707.4ba7d56.el7ost.noarch
openstack-neutron-common-11.0.0-0.20170624003801.11acb1d.el7ost.noarch
openstack-neutron-11.0.0-0.20170624003801.11acb1d.el7ost.noarch
openstack-nova-compute-16.0.0-0.20170624031428.3863eca.el7ost.noarch
openstack-mistral-executor-5.0.0-0.20170623025911.6a1ca0f.el7ost.noarch
openstack-tempest-16.0.1-0.20170623203531.4386df8.el7ost.noarch
openstack-neutron-openvswitch-11.0.0-0.20170624003801.11acb1d.el7ost.noarch
openstack-nova-placement-api-16.0.0-0.20170624031428.3863eca.el7ost.noarch
openstack-heat-api-cfn-9.0.0-0.20170623110018.173f03a.el7ost.noarch
openstack-mistral-api-5.0.0-0.20170623025911.6a1ca0f.el7ost.noarch
openstack-ironic-api-8.0.1-0.20170621044433.f0e6a07.el7ost.noarch
python-openstack-mistral-5.0.0-0.20170623025911.6a1ca0f.el7ost.noarch
openstack-heat-common-9.0.0-0.20170623110018.173f03a.el7ost.noarch
openstack-tripleo-image-elements-7.0.0-0.20170607161959.401d861.el7ost.noarch
openstack-tripleo-puppet-elements-7.0.0-0.20170614005502.9285877.el7ost.noarch
openstack-swift-object-2.14.1-0.20170622024006.2d18ecd.el7ost.noarch
openstack-neutron-ml2-11.0.0-0.20170624003801.11acb1d.el7ost.noarch
openstack-glance-15.0.0-0.20170623215940.8188eca.el7ost.noarch
openstack-nova-scheduler-16.0.0-0.20170624031428.3863eca.el7ost.noarch
openstack-heat-engine-9.0.0-0.20170623110018.173f03a.el7ost.noarch
openstack-mistral-engine-5.0.0-0.20170623025911.6a1ca0f.el7ost.noarch
openstack-ironic-conductor-8.0.1-0.20170621044433.f0e6a07.el7ost.noarch
openstack-tripleo-common-7.1.1-0.20170623115707.4ba7d56.el7ost.noarch
python-openstackclient-3.11.0-0.20170613232431.c69304e.el7ost.noarch
openstack-ironic-common-8.0.1-0.20170621044433.f0e6a07.el7ost.noarch
openstack-swift-account-2.14.1-0.20170622024006.2d18ecd.el7ost.noarch
openstack-selinux-0.8.8-0.20170622195307.74ddc0e.el7ost.noarch
openstack-swift-container-2.14.1-0.20170622024006.2d18ecd.el7ost.noarch
openstack-nova-conductor-16.0.0-0.20170624031428.3863eca.el7ost.noarch
openstack-tripleo-validations-7.1.1-0.20170621111847.fb7346f.el7ost.noarch
openstack-tripleo-heat-templates-7.0.0-0.20170624014919.el7ost.noarch
puppet-openstacklib-11.2.0-0.20170613150439.93b8e7d.el7ost.noarch
python-openstacksdk-0.9.17-0.20170621195806.7946243.el7ost.noarch
openstack-swift-proxy-2.14.1-0.20170622024006.2d18ecd.el7ost.noarch
openstack-heat-api-9.0.0-0.20170623110018.173f03a.el7ost.noarch
openstack-zaqar-5.0.0-0.20170623003642.68eac5a.el7ost.noarch
openstack-tripleo-ui-7.1.1-0.20170624213423.cb896fd.el7ost.noarch
openstack-puppet-modules-10.0.0-1.el7ost.noarch
openstack-mistral-common-5.0.0-0.20170623025911.6a1ca0f.el7ost.noarch
openstack-nova-common-16.0.0-0.20170624031428.3863eca.el7ost.noarch

How reproducible:


Steps to Reproduce:
Steps to Reproduce:
1.Install OSP11
infrared virsh -v --host-address 10.9.76.22 --host-key ~/.ssh/id_rsa --cleanup yes && infrared virsh -v --host-address 10.9.76.22 --host-key ~/.ssh/id_rsa  --topology-nodes undercloud:1,controller:1,compute:1  -e  override.controller.cpu=8 -e override.controller.memory=16384 -e  override.undercloud.disks.disk1.size=100G && infrared tripleo-undercloud --version 11 --images-task=rpm && infrared tripleo-overcloud -v --introspect yes --tagging yes --post no --deployment-files virt --version 11 --deploy yes 

2.Upgrade undercloud and overcloud to osp11 latest+ rhel7.4
ir tripleo-undercloud -v --update-undercloud yes --mirror qeos --build 7.4-testing --osrelease 7.4 

ir tripleo-overcloud -v --updateto 7.4-testing --deployment-files virt --mirror qeos --osrelease 7.4
#Please note, during yum update stage on overcloud controller run "sudo pcs cluster start and sudo pcs cleanup resources" - w/a for https://bugzilla.redhat.com/show_bug.cgi?id=1464588

3. Upgrade undercloud and overcloud to osp12
http://etherpad.corp.redhat.com/osp12-upgrade
#please note, before updating overcloud apply w/a for https://bugzilla.redhat.com/show_bug.cgi?id=1460421
https://bugzilla.redhat.com/show_bug.cgi?id=1463227
https://bugzilla.redhat.com/show_bug.cgi?id=1466219
https://bugzilla.redhat.com/show_bug.cgi?id=1466308

Actual results:
overcloud update failed


Expected results:
overcloud update succeed

Additional info:
http://pastebin.test.redhat.com/498883

Comment 1 Red Hat Bugzilla Rules Engine 2017-06-29 13:13:49 UTC
This bugzilla has been removed from the release and needs to be reviewed and Triaged for another Target Release.

Comment 2 Jiri Stransky 2017-06-29 13:46:06 UTC
There seems to be a difference between available packages on undercloud vs. overcloud. Undercloud:

(undercloud) [stack@undercloud-0 ~]$ yum list available | grep openstack-dashboard
openstack-dashboard.noarch        1:12.0.0-0.20170624124155.e8d1dd9.el7ost
openstack-dashboard-theme.noarch  12.0.0-2.el7ost        rhelosp-12.0-puddle    


Overcloud:

[root@controller-0 ~]# yum list available | grep openstack-dashboard
openstack-dashboard.noarch        1:12.0.0-0.20170624124155.e8d1dd9.el7ost

It could be an issue with how we setup repositories on OC during upgrade initialization.

Comment 3 Artem Hrechanychenko 2017-06-29 15:38:31 UTC
yum clean all;rhos-release -x; rhos-release 12-director ; reboot on overcloud nodes didn't helped

Comment 4 Jiri Stransky 2017-06-30 08:46:34 UTC
The problem is that the openstack-dashboard and openstack-dashboard-theme versions in the latest puddle repo don't match, but it looks like they're supposed to be version-locked together, so i guess yum ignores the newer openstack-dashboard-theme. (Perhaps the reason why yum listed it on undercloud as available is that it's not yet installed on undercloud, and maybe some of the version lock checks weren't triggered then... Just guessing...)

[   ] openstack-dashboard-12.0.0-0.20170624124155.e8d1dd9.el7ost.noarch.rpm                       26-Jun-2017 10:39  3.4M  
[   ] openstack-dashboard-theme-12.0.0-2.el7ost.noarch.rpm                                        23-Jun-2017 04:38  703K  

E.g. in OSP 11 latest puddle repos, they are exactly the same version:


[   ] openstack-dashboard-11.0.1-5.el7ost.noarch.rpm                               04-Apr-2017 13:42   13M  
[   ] openstack-dashboard-theme-11.0.1-5.el7ost.noarch.rpm                         04-Apr-2017 13:42  6.2K  

Let's try adding

yum -y remove openstack-dasboard-theme

into UpgradeInitCommand as a workaround for now.

Comment 5 Jiri Stransky 2017-06-30 08:47:43 UTC
Typo fix:

yum -y remove openstack-dashboard-theme

Comment 7 Mike Burns 2017-06-30 12:22:30 UTC
It appears the issue here is the epoch.

in python-django-horizon specfile, openstack-dashboard has:

Requires:   openstack-dashboard-theme => 1:12.0

The version of openstack-dashboard-theme is 12.0.0 with no epoch.  

Note, the => may cause a problem too, though I'm not certain.  We should probably clean it up to be >=.

Comment 8 Artem Hrechanychenko 2017-06-30 12:25:26 UTC
w/a - 
create local mirror with this rpms
https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=13578817
on all oc nodes and re-start oc upgrade

Comment 9 Thierry Vignaud 2017-06-30 13:07:37 UTC
Good catch.

Comment 10 Thierry Vignaud 2017-06-30 13:09:56 UTC
Note that "=>" is automatically converted to ">=" but let's clean it for clarity/readability

Comment 14 Marius Cornea 2017-08-29 13:07:47 UTC
This issue reported initially is still showing up in the latest puddle:

[root@controller-0 heat-admin]# yum info python-django-horizon
Loaded plugins: product-id, search-disabled-repos, subscription-manager
This system is not registered with an entitlement server. You can use subscription-manager to register.
Installed Packages
Name        : python-django-horizon
Arch        : noarch
Epoch       : 1
Version     : 11.0.3
Release     : 3.el7ost
Size        : 2.9 M
Repo        : installed
From repo   : rhos-11.0-signed
Summary     : Django application for talking to Openstack
URL         : http://horizon.openstack.org/
License     : ASL 2.0 and BSD
Description : Horizon is a Django application for providing Openstack UI components.
            : It allows performing site administrator (viewing account resource usage,
            : configuring users, accounts, quotas, flavors, etc.) and end user
            : operations (start/stop/delete instances, create/restore snapshots, view
            : instance VNC console, etc.)

Available Packages
Name        : python-django-horizon
Arch        : noarch
Epoch       : 1
Version     : 12.0.0
Release     : 0.20170821142112.el7ost
Size        : 697 k
Repo        : rhelosp-12.0-puddle/x86_64
Summary     : Django application for talking to Openstack
URL         : http://horizon.openstack.org/
License     : ASL 2.0 and BSD
Description : Horizon is a Django application for providing Openstack UI components.
            : It allows performing site administrator (viewing account resource usage,
            : configuring users, accounts, quotas, flavors, etc.) and end user
            : operations (start/stop/delete instances, create/restore snapshots, view
            : instance VNC console, etc.)



--> Finished Dependency Resolution
Error: Package: 1:openstack-dashboard-theme-11.0.3-3.el7ost.noarch (@rhos-11.0-signed)
           Requires: openstack-dashboard = 1:11.0.3-3.el7ost
           Removing: 1:openstack-dashboard-11.0.3-3.el7ost.noarch (@rhos-11.0-signed)
               openstack-dashboard = 1:11.0.3-3.el7ost
           Updated By: 1:openstack-dashboard-12.0.0-0.20170821142112.el7ost.noarch (rhelosp-12.0-puddle)
               openstack-dashboard = 1:12.0.0-0.20170821142112.el7ost
 You could try using --skip-broken to work around the problem
 You could try running: rpm -Va --nofiles --nodigest

Comment 16 Thierry Vignaud 2017-08-30 13:44:10 UTC
Could you attach the full yum logs?

Comment 17 Thierry Vignaud 2017-08-30 14:51:18 UTC
I suspect a repo issue

Comment 18 Thierry Vignaud 2017-08-30 15:09:39 UTC
From some manual tests with dnf (bad I know) & rpm, I think the issue is that EVR of the new openstack-dashboard-theme is lower than the one in osp11 (VR is newer but epoch isn't)
I think we might need to add back the Epoch in the splitted openstack-dashboard-theme.
I'll test that theory

Comment 19 Thierry Vignaud 2017-08-30 15:27:55 UTC
This seems to do the trick (openstack-dashboard-theme-12.0.0-2.el7ost.1)
(I should have bump the release to 3 instead of appending ".1" btw)
We now need to adjust the requires in python-django-horizon so that it includes the epoch again

Comment 24 errata-xmlrpc 2017-12-13 21:35:30 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:3462


Note You need to log in before you can comment on or make changes to this bug.