Bug 1321036 - Upgrade needs update identifier set to prevent down services
Summary: Upgrade needs update identifier set to prevent down services
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates
Version: 8.0 (Liberty)
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ga
: 8.0 (Liberty)
Assignee: Marios Andreou
QA Contact: Omri Hochman
URL:
Whiteboard:
: 1320825 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-03-24 14:14 UTC by mathieu bultel
Modified: 2016-04-20 13:04 UTC (History)
6 users (show)

Fixed In Version: openstack-tripleo-heat-templates-0.8.14-4.el7ost
Doc Type: Bug Fix
Doc Text:
Previously, the final stage of the upgrades process did not ensure that pacemaker cluster resources were being restarted properly. This would cause some resources to remain down after the upgrade was complete. For example, updating the password of the rabbit messaging service as part of an upgrade would mean that you would also need to restart all the services that communicate with this service such as neutron-server, but only after having restarted the rabbit service itself. Now, pacemaker resources are restarted during the final part of the upgrade, and the upgrade completes successfully with all pacemaker resources reported as "Started".
Clone Of:
Environment:
Last Closed: 2016-04-20 13:04:27 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
OpenStack gerrit 297175 0 'None' 'MERGED' 'Set UpdateIdentifier for upgrade converge, to prevent services down' 2019-11-29 15:56:28 UTC
Red Hat Product Errata RHBA-2016:0653 0 normal SHIPPED_LIVE Red Hat Enterprise Linux OSP 8 director Bug Fix Advisory 2016-04-20 17:02:07 UTC

Description mathieu bultel 2016-03-24 14:14:12 UTC 
Description of problem:

Upgrade 7 to 8-director.

Upgrade command:
openstack overcloud deploy --templates tripleo-heat-templates -e tripleo-heat-templates/overcloud-resource-registry-puppet.yaml -e tripleo-heat-templates/environments/puppet-pacemaker.yaml -e tripleo-heat-templates/environments/major-upgrade-pacemaker-converge.yaml

Environment: Virt, 1 controller, 1 compute, 1 ceph, network isolation.

After an Overcloud upgrade, when applying the environments/major-upgrade-pacemaker-converge.yaml templates, neutron-server was down.
Setting the update_identifier, allow the upgrade command to trigger the restart of haproxy and let neutron-server available again.



Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:
Comment 2 mathieu bultel 2016-03-24 14:14:33 UTC 
Proposed review :
https://review.openstack.org/297175
Comment 3 Mike Burns 2016-03-24 14:28:15 UTC 
*** Bug 1320825 has been marked as a duplicate of this bug. ***
Comment 9 Omri Hochman 2016-04-15 20:29:48 UTC 
verified with openstack-tripleo-heat-templates-kilo-0.8.14-7.el7ost.noarch

the code is merged from: 
https://github.com/openstack/tripleo-heat-templates/blob/a12087715f0fe4251a95ab67120023d553c24a45/extraconfig/tasks/pacemaker_resource_restart.sh#L11

to: /usr/share/openstack-tripleo-heat-templates/extraconfig/tasks/pacemaker_resource_restart.sh

the upgrade command included major-upgrade-pacemaker-converge.yaml:

openstack overcloud deploy   --templates /usr/share/openstack-tripleo-heat-templates -e    /usr/share/openstack-tripleo-heat-templates/overcloud-resource-registry-puppet.yaml   -e   /usr/share/openstack-tripleo-heat-templates/environments/puppet-pacemaker.yaml   -e   /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml   -e   /usr/share/openstack-tripleo-heat-templates/environments/net-single-nic-with-vlans.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/storage-environment.yaml -e /home/stack/ssl-heat-templates/environments/enable-tls.yaml -e /home/stack/ssl-heat-templates/environments/inject-trust-anchor.yaml  -e network-environment.yaml -e  /usr/share/openstack-tripleo-heat-templates/environments/major-upgrade-pacemaker-converge.yaml

[heat-admin@overcloud-controller-1 ~]$ openstack-status  | grep neutron-server
neutron-server:                         active    (disabled on boot)


pcs status view:
-----------------
  Started: [ overcloud-controller-0 overcloud-controller-1 overcloud-controller-2 ]
 Clone Set: neutron-server-clone [neutron-server]
Comment 11 errata-xmlrpc 2016-04-20 13:04:27 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-0653.html
Note You need to log in before you can comment on or make changes to this bug.