Bug 1535489 - OSP10 -> OSP13 FFU upgrade: upgrade_steps_playbook.yaml: fails if run multiple times because cinder-volume pcs disable tasks are not idempotent
Summary: OSP10 -> OSP13 FFU upgrade: upgrade_steps_playbook.yaml: fails if run multipl...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates
Version: 13.0 (Queens)
Hardware: Unspecified
OS: Unspecified
high
urgent
Target Milestone: beta
: 13.0 (Queens)
Assignee: Emilien Macchi
QA Contact: Marius Cornea
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-01-17 14:10 UTC by Marius Cornea
Modified: 2018-06-27 13:43 UTC (History)
14 users (show)

Fixed In Version: openstack-tripleo-heat-templates-8.0.0-0.20180227121938.e0f59ee.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-06-27 13:42:25 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Launchpad 1743789 None None None 2018-01-17 14:10:21 UTC
OpenStack gerrit 539249 None MERGED Add pacemaker upgrade_tasks for P..Q major upgrade 2020-08-21 04:40:31 UTC
Red Hat Product Errata RHEA-2018:2086 None None None 2018-06-27 13:43:05 UTC

Description Marius Cornea 2018-01-17 14:10:22 UTC
Description of problem:
Newton -> Queens FFU upgrade: upgrade_steps_playbook.yaml: fails if run multiple times because openstack-cinder-volume pcs disable tasks are not idempotent

Disabling and deleting the openstack-cinder-volume pacemaker resource tasks are not idempotent:
https://github.com/openstack/tripleo-heat-templates/blob/master/docker/services/pacemaker/cinder-volume.yaml#L241-L258

if the upgrade upgrade_steps_playbook.yaml is run multiple times and the openstack-cinder-volume was deleted in the first run then the 2nd time it fails with:

TASK [Disable the openstack-cinder-volume cluster resource] ***********************************************************************************************************************************************************************************
FAILED - RETRYING: Disable the openstack-cinder-volume cluster resource (5 retries left).
FAILED - RETRYING: Disable the openstack-cinder-volume cluster resource (4 retries left).
FAILED - RETRYING: Disable the openstack-cinder-volume cluster resource (3 retries left).
FAILED - RETRYING: Disable the openstack-cinder-volume cluster resource (2 retries left).
FAILED - RETRYING: Disable the openstack-cinder-volume cluster resource (1 retries left).
fatal: [192.168.24.9]: FAILED! => {"attempts": 5, "changed": false, "error": "Error: resource/clone/master/group/bundle 'openstack-cinder-volume' does not exist\n", "failed": true, "msg": "Failed, to set the resource openstack-cinder-volume to the state disable", "output": "", "rc": 1}

We should make the delete tasks idempotent to allow running the playbook multiple times to be able to recover in case of failure.

Comment 11 errata-xmlrpc 2018-06-27 13:42:25 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2018:2086


Note You need to log in before you can comment on or make changes to this bug.