Bug 1354046 - rhel-osp-director: 8.0->9.0 upgrade, Keystone WSGI migration step fails, httpd resource is down on the cluster and fails to start.
Summary: rhel-osp-director: 8.0->9.0 upgrade, Keystone WSGI migration step fails, htt...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates
Version: 9.0 (Mitaka)
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ga
: 9.0 (Mitaka)
Assignee: Michele Baldessari
QA Contact: Rodrigo Duarte
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-07-08 20:05 UTC by Alexander Chuzhoy
Modified: 2016-11-11 07:09 UTC (History)
15 users (show)

Fixed In Version: openstack-tripleo-heat-templates-2.0.0-21.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-08-11 11:35:52 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2016:1599 normal SHIPPED_LIVE Red Hat OpenStack Platform 9 director Release Candidate Advisory 2016-08-11 15:25:37 UTC
OpenStack gerrit 338879 None None None 2016-07-13 13:19:47 UTC

Internal Links: 1365225

Description Alexander Chuzhoy 2016-07-08 20:05:16 UTC
rhel-osp-director:  8.0->9.0 upgrade, Keystone WSGI migration step fails, httpd resource is down on the cluster and fails to start.


Environment:
httpd-tools-2.4.6-40.el7_2.1.x86_64
python-keystoneclient-1.7.2-1.el7ost.noarch
python-keystoneauth1-1.1.0-4.el7ost.noarch
python-keystone-8.0.1-1.el7ost.noarch
httpd-2.4.6-40.el7_2.1.x86_64
python-keystonemiddleware-2.3.1-1.el7ost.noarch
openstack-keystone-8.0.1-1.el7ost.noarch
openstack-tripleo-heat-templates-kilo-2.0.0-13.el7ost.noarch
openstack-tripleo-heat-templates-2.0.0-13.el7ost.noarch
openstack-tripleo-heat-templates-liberty-2.0.0-13.el7ost.noarch
openstack-puppet-modules-8.1.2-1.el7ost.noarch
instack-undercloud-4.0.0-5.el7ost.noarch



Steps to reproduce:
1. Deploy 8.0 with:
openstack overcloud deploy --templates --control-scale 3 --compute-scale 1   --neutron-network-type vxlan --neutron-tunnel-types vxlan  --ntp-server clock.redhat.com --timeout 90 -e /usr/share/openstack-tripleo-heat-templates/environments/storage-environment.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml -e network-environment.yaml --ceph-storage-scale 1

2. Upgrade the undercloud 
3. Successfully pass the step with "-e /usr/share/openstack-tripleo-heat-templates/environments/major-upgrade-aodh.yaml"
4. Attempt to do the keystone wsgi migration step (includes  -e /usr/share/openstack-tripleo-heat-templates/environments/major-upgrade-keystone-liberty-mitaka.yaml)
Result:
2016-07-08 19:23:50 [ControllerDeployment]: SIGNAL_COMPLETE Unknown
2016-07-08 19:23:52 [1]: SIGNAL_COMPLETE Unknown
2016-07-08 19:23:54 [1]: SIGNAL_COMPLETE Unknown
Stack overcloud UPDATE_FAILED
Deployment failed:  Heat Stack update failed.

Checked the pcs resource status. Several resources were stopped.
ran "pcs resource cleanup"
All but one resource successfully started. 

[root@overcloud-controller-1 ~]# pcs status|grep -B1 -i stop
 Clone Set: httpd-clone [httpd]
     Stopped: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ]


Was unable to start this resource:
(98)Address already in use: AH00072: make_sock: could not bind to address 192.168.100.12:5000



turned out "/usr/bin/python2/usr/bin/keystone-all" was already using port 5000.

Comment 2 Alexander Chuzhoy 2016-07-08 20:33:25 UTC
The workaround is to stop the openstack-keystone.service on all controllers and then to start the httpd resource.

Comment 19 errata-xmlrpc 2016-08-11 11:35:52 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2016-1599.html


Note You need to log in before you can comment on or make changes to this bug.