Bug 1535489

Summary: OSP10 -> OSP13 FFU upgrade: upgrade_steps_playbook.yaml: fails if run multiple times because cinder-volume pcs disable tasks are not idempotent
Product: Red Hat OpenStack Reporter: Marius Cornea <mcornea>
Component: openstack-tripleo-heat-templatesAssignee: Emilien Macchi <emacchi>
Status: CLOSED ERRATA QA Contact: Marius Cornea <mcornea>
Severity: urgent Docs Contact:
Priority: high    
Version: 13.0 (Queens)CC: dbecker, emacchi, jfrancoa, jschluet, mbracho, mbultel, mburns, mcornea, morazi, rhel-osp-director-maint, rhos-flags, sathlang, sclewis, tshefi
Target Milestone: betaKeywords: Triaged
Target Release: 13.0 (Queens)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: openstack-tripleo-heat-templates-8.0.0-0.20180227121938.e0f59ee.el7ost Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-06-27 13:42:25 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Marius Cornea 2018-01-17 14:10:22 UTC
Description of problem:
Newton -> Queens FFU upgrade: upgrade_steps_playbook.yaml: fails if run multiple times because openstack-cinder-volume pcs disable tasks are not idempotent

Disabling and deleting the openstack-cinder-volume pacemaker resource tasks are not idempotent:
https://github.com/openstack/tripleo-heat-templates/blob/master/docker/services/pacemaker/cinder-volume.yaml#L241-L258

if the upgrade upgrade_steps_playbook.yaml is run multiple times and the openstack-cinder-volume was deleted in the first run then the 2nd time it fails with:

TASK [Disable the openstack-cinder-volume cluster resource] ***********************************************************************************************************************************************************************************
FAILED - RETRYING: Disable the openstack-cinder-volume cluster resource (5 retries left).
FAILED - RETRYING: Disable the openstack-cinder-volume cluster resource (4 retries left).
FAILED - RETRYING: Disable the openstack-cinder-volume cluster resource (3 retries left).
FAILED - RETRYING: Disable the openstack-cinder-volume cluster resource (2 retries left).
FAILED - RETRYING: Disable the openstack-cinder-volume cluster resource (1 retries left).
fatal: [192.168.24.9]: FAILED! => {"attempts": 5, "changed": false, "error": "Error: resource/clone/master/group/bundle 'openstack-cinder-volume' does not exist\n", "failed": true, "msg": "Failed, to set the resource openstack-cinder-volume to the state disable", "output": "", "rc": 1}

We should make the delete tasks idempotent to allow running the playbook multiple times to be able to recover in case of failure.

Comment 11 errata-xmlrpc 2018-06-27 13:42:25 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2018:2086