Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1792901

Summary: [OSP14] Manila upgrade tasks failing with: The conditional check 'manila_share_pcs_res|succeeded' failed.
Product: Red Hat OpenStack Reporter: Jose Luis Franco <jfrancoa>
Component: openstack-tripleo-heat-templatesAssignee: Jose Luis Franco <jfrancoa>
Status: CLOSED EOL QA Contact: Sasha Smolyak <ssmolyak>
Severity: high Docs Contact:
Priority: high    
Version: 14.0 (Rocky)CC: mburns, rrasouli
Target Milestone: ---Keywords: Triaged, ZStream
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1792890 Environment:
Last Closed: 2020-01-20 14:17:44 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1792890    
Bug Blocks:    

Description Jose Luis Franco 2020-01-20 10:51:44 UTC
+++ This bug was initially created as a clone of Bug #1792890 +++
+++ In OSP14 only the ovn-dbs service is impacted by this issue +++

Description of problem:

Bugzilla opened from an issue identified in the case https://access.redhat.com/support/cases/#/case/02406707.

During the upgrade of the controllers from OSP12 to OSP13, one of the upgrade tasks failed with the following error:

2020-01-18 01:59:07,802 p=136626 u=mistral |  TASK [Disable the Manila-Share cluster resource before container upgrade] ******
2020-01-18 01:59:07,802 p=136626 u=mistral |  Saturday 18 January 2020  01:59:07 +0000 (0:00:00.372)       0:05:48.717 ******
2020-01-18 01:59:07,895 p=136626 u=mistral |  skipping: [fws-shk01-fcn03] => {"changed": false, "skip_reason": "Conditional result was False"}
2020-01-18 01:59:07,991 p=136626 u=mistral |  skipping: [fws-shk01-fcn02] => {"changed": false, "skip_reason": "Conditional result was False"}
2020-01-18 01:59:08,005 p=136626 u=mistral |  [DEPRECATION WARNING]: Using tests as filters is deprecated. Instead of using
`result|succeeded` use `result is succeeded`. This feature will be removed in
version 2.9. Deprecation warnings can be disabled by setting
deprecation_warnings=False in ansible.cfg.
2020-01-18 01:59:08,009 p=136626 u=mistral |  fatal: [fws-shk01-fcn01]: FAILED! => {"msg": "The conditional check 'manila_share_pcs_res|succeeded' failed. The error was: The failed test expects a dictionary\n\nThe error appears to have been in '/var/lib/mistral/527eac46-145c-4244-a399-e8f20cb697e8/Controller/upgrade_tasks.yaml': line 1010, column 5, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n- block:\n  - name: Disable the Manila-Share cluster resource before container upgrade\n    ^ here\n"}
2020-01-18 01:59:08,096 p=136626 u=mistral |  TASK [Update the Manila-Share bundle to use the new container image name] ******
2020-01-18 01:59:08,096 p=136626 u=mistral |  Saturday 18 January 2020  01:59:08 +0000 (0:00:00.294)       0:05:49.011 ******


Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1. Deploy an OSP12 environment with the manilla service enabled
2. Upgrade the Undercloud to 13
3. Upgrade the Overcloud Controller's role

Actual results:

The upgrade procedure fails for the Controller.

Expected results:


Additional info:

--- Additional comment from Jose Luis Franco on 2020-01-20 10:38:37 UTC ---

The upgrade tasks for the manila-share service are incorrect when evaluating the manila_share_pcs_res variable:

        - name: Update openstack-manila-share pcs resource bundle for new container image
          when:
            - step|int == 1
            - manila_share_containerized|bool
            - is_bootstrap_node
            - manila_share_pcs_res|succeeded
          block:
            - name: Disable the Manila-Share cluster resource before container upgrade
              pacemaker_resource:
                resource: openstack-manila-share
                state: disable
                wait_for_resource: true
              register: output
              retries: 5
              until: output.rc == 0

https://github.com/openstack/tripleo-heat-templates/blob/stable/queens/docker/services/pacemaker/manila-share.yaml#L330

The manila_share_pcs_res is a boolean:

2020-01-18 01:54:07,460 p=136626 u=mistral |  TASK [Set fact manila_share_pcs_res] *******************************************
2020-01-18 01:54:07,460 p=136626 u=mistral |  Saturday 18 January 2020  01:54:07 +0000 (0:00:01.193)       0:00:48.375 ******
2020-01-18 01:54:07,477 p=136626 u=mistral |  [DEPRECATION WARNING]: Using tests as filters is deprecated. Instead of using
`result|succeeded` use `result is succeeded`. This feature will be removed in
version 2.9. Deprecation warnings can be disabled by setting
deprecation_warnings=False in ansible.cfg.
2020-01-18 01:54:07,556 p=136626 u=mistral |  ok: [fws-shk01-fcn03] => {"ansible_facts": {"manila_share_pcs_res": true}, "changed": false}
2020-01-18 01:54:07,565 p=136626 u=mistral |  [DEPRECATION WARNING]: Using tests as filters is deprecated. Instead of using
`result|succeeded` use `result is succeeded`. This feature will be removed in
version 2.9. Deprecation warnings can be disabled by setting
deprecation_warnings=False in ansible.cfg.
2020-01-18 01:54:07,650 p=136626 u=mistral |  ok: [fws-shk01-fcn02] => {"ansible_facts": {"manila_share_pcs_res": true}, "changed": false}
2020-01-18 01:54:07,662 p=136626 u=mistral |  [DEPRECATION WARNING]: Using tests as filters is deprecated. Instead of using
`result|succeeded` use `result is succeeded`. This feature will be removed in
version 2.9. Deprecation warnings can be disabled by setting
deprecation_warnings=False in ansible.cfg.
2020-01-18 01:54:07,686 p=136626 u=mistral |  ok: [fws-shk01-fcn01] => {"ansible_facts": {"manila_share_pcs_res": true}, "changed": false}


Therefore, we should be using the |bool filter instead.
The same issue is occuring in the ovn-dbs.yaml template: https://github.com/openstack/tripleo-heat-templates/blob/stable/queens/docker/services/pacemaker/ovn-dbs.yaml#L353

Comment 1 Jose Luis Franco 2020-01-20 11:37:30 UTC
I guess it does not make much sense to fix this for 14 as it is already EOL. Anyway, I'll leave the bugzilla opened until the 13 equivalent gets solved and verified.