Bug 1872851

Summary: [FFWD] add the ceph_nfs short name to hiera data for the upgrade task fails
Product: Red Hat OpenStack Reporter: Yogev Rabl <yrabl>
Component: tripleo-ansibleAssignee: Lukas Bezdicka <lbezdick>
Status: CLOSED CURRENTRELEASE QA Contact: Sasha Smolyak <ssmolyak>
Severity: high Docs Contact:
Priority: high    
Version: 16.1 (Train)CC: aschultz, gfidente, gouthamr, jpretori, lbezdick, vhariria
Target Milestone: z2Keywords: Triaged
Target Release: 16.1 (Train on RHEL 8.2)   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: openstack-tripleo-heat-templates-11.3.2-1.20200914170156.el8ost Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-03-16 13:26:29 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Yogev Rabl 2020-08-26 18:02:40 UTC
Description of problem:
The CephNFS hiera update fails with the error:
TASK [add the ceph_nfs short name to hiera data for the upgrade.] **************
Wednesday 26 August 2020  13:38:09 -0400 (0:00:00.108)       0:00:54.214 ****** 
ERROR! the role 'tripleo_upgrade_hiera' was not found in /var/lib/mistral/0989ea80-9eb4-4b7d-bfe5-317e3952f6ac/roles:/var/lib/mistral/0989ea80-9eb4-4b7d-bfe5-317e3952f6ac/roles:/root/.ansible/roles:/usr/share/ansible/tripleo-roles:/usr/share/ansible/roles:/etc/ansible/roles:/usr/share/openstack-tripleo-validations/roles:/var/lib/mistral/0989ea80-9eb4-4b7d-bfe5-317e3952f6ac:/var/lib/mistral/0989ea80-9eb4-4b7d-bfe5-317e3952f6ac

The error appears to be in '/var/lib/mistral/0989ea80-9eb4-4b7d-bfe5-317e3952f6ac/ControllerStorageNfs/upgrade_tasks_step1.yaml': line 69, column 13, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

  - include_role:
      name: tripleo_upgrade_hiera
            ^ here

The role is located in /usr/share/ansible/roles/tripleo-upgrade-hiera


Version-Release number of selected component (if applicable):
tripleo-ansible-0.5.1-0.20200611113659.34b8fcc.el8ost.noarch


How reproducible:
100%

Steps to Reproduce:
1. upgrade an OSP13 with CephNFS to OSP 16.1

Actual results:
openstack overcloud upgrade run \
        --stack overcloud \
        --tags system_upgrade \
        --limit controller-0 \
        | tee oc-c0-system-upgrade.log

fails

Expected results:
The stage passes

Additional info:

Comment 1 Yogev Rabl 2020-08-27 13:30:29 UTC
A workround

$ cd /usr/share/ansible/roles
$ sudo ln -s tripleo-upgrade-hiera tripleo_upgrade_hiera

Comment 2 Jose Luis Franco 2020-08-31 16:32:02 UTC
Hello Yogev,

I think I've seen the bug here:

[jfrancoa@localhost tripleo-heat-templates]$ grep -r "tripleo_upgrade_hiera" *
deployment/ceph-ansible/ceph-nfs.yaml:                name: tripleo_upgrade_hiera
deployment/ceph-ansible/ceph-nfs.yaml:                name: tripleo_upgrade_hiera
deployment/pacemaker/pacemaker-remote-baremetal-puppet.yaml:                name: tripleo_upgrade_hiera
deployment/pacemaker/pacemaker-remote-baremetal-puppet.yaml:                name: tripleo_upgrade_hiera
deployment/pacemaker/pacemaker-remote-baremetal-puppet.yaml:            name: tripleo_upgrade_hiera

[jfrancoa@localhost tripleo-heat-templates]$ grep -r "tripleo-upgrade-hiera" *
deployment/cinder/cinder-volume-pacemaker-puppet.yaml:                name: tripleo-upgrade-hiera
deployment/cinder/cinder-volume-pacemaker-puppet.yaml:                name: tripleo-upgrade-hiera
deployment/cinder/cinder-volume-pacemaker-puppet.yaml:                name: tripleo-upgrade-hiera
deployment/cinder/cinder-backup-pacemaker-puppet.yaml:                name: tripleo-upgrade-hiera
deployment/cinder/cinder-backup-pacemaker-puppet.yaml:                name: tripleo-upgrade-hiera
deployment/cinder/cinder-backup-pacemaker-puppet.yaml:                name: tripleo-upgrade-hiera
deployment/database/mysql-pacemaker-puppet.yaml:                name: tripleo-upgrade-hiera
deployment/database/mysql-pacemaker-puppet.yaml:                name: tripleo-upgrade-hiera
deployment/database/mysql-pacemaker-puppet.yaml:                name: tripleo-upgrade-hiera
deployment/database/redis-pacemaker-puppet.yaml:                name: tripleo-upgrade-hiera
deployment/database/redis-pacemaker-puppet.yaml:                name: tripleo-upgrade-hiera
deployment/ovn/ovn-dbs-pacemaker-puppet.yaml:                name: tripleo-upgrade-hiera
deployment/ovn/ovn-dbs-pacemaker-puppet.yaml:                name: tripleo-upgrade-hiera
deployment/haproxy/haproxy-pacemaker-puppet.yaml:                name: tripleo-upgrade-hiera
deployment/haproxy/haproxy-pacemaker-puppet.yaml:                name: tripleo-upgrade-hiera
deployment/pacemaker/pacemaker-baremetal-puppet.yaml.bkp:                name: tripleo-upgrade-hiera
deployment/pacemaker/pacemaker-baremetal-puppet.yaml.bkp:                name: tripleo-upgrade-hiera
deployment/pacemaker/pacemaker-baremetal-puppet.yaml.bkp:                name: tripleo-upgrade-hiera
deployment/pacemaker/pacemaker-baremetal-puppet.yaml:                name: tripleo-upgrade-hiera
deployment/pacemaker/pacemaker-baremetal-puppet.yaml:                name: tripleo-upgrade-hiera
deployment/pacemaker/pacemaker-baremetal-puppet.yaml:                name: tripleo-upgrade-hiera
deployment/rabbitmq/rabbitmq-messaging-pacemaker-puppet.yaml:                name: tripleo-upgrade-hiera
deployment/rabbitmq/rabbitmq-messaging-pacemaker-puppet.yaml:                name: tripleo-upgrade-hiera
deployment/rabbitmq/rabbitmq-messaging-rpc-pacemaker-puppet.yaml:                name: tripleo-upgrade-hiera
deployment/rabbitmq/rabbitmq-messaging-rpc-pacemaker-puppet.yaml:                name: tripleo-upgrade-hiera
deployment/rabbitmq/rabbitmq-messaging-rpc-pacemaker-puppet.yaml:                name: tripleo-upgrade-hiera
deployment/rabbitmq/rabbitmq-messaging-notify-pacemaker-puppet.yaml:                name: tripleo-upgrade-hiera
deployment/rabbitmq/rabbitmq-messaging-notify-pacemaker-puppet.yaml:                name: tripleo-upgrade-hiera
deployment/rabbitmq/rabbitmq-messaging-notify-pacemaker-puppet.yaml:                name: tripleo-upgrade-hiera
deployment/manila/manila-share-pacemaker-puppet.yaml:                name: tripleo-upgrade-hiera
deployment/manila/manila-share-pacemaker-puppet.yaml:                name: tripleo-upgrade-hiera
deployment/manila/manila-share-pacemaker-puppet.yaml:                name: tripleo-upgrade-hiera

This templates use the wrong role identifier:

https://github.com/openstack/tripleo-heat-templates/blob/90021a6d3e63c2555f0e917dc5e939de5b605dac/deployment/ceph-ansible/ceph-nfs.yaml#L103-L105
https://github.com/openstack/tripleo-heat-templates/blob/stable/train/deployment/pacemaker/pacemaker-remote-baremetal-puppet.yaml#L158-L160

I will prepare a fix for this.

Comment 4 Jose Luis Franco 2021-01-13 13:23:51 UTC
Patch present in rhos/rhos-16.1-trunk-patches 70f52b99225507d92add0398a044ed9928b4abad