Bug 1556992

Summary: FFU: post upgrading an environment; the hieradata is not refreshed on stack updates
Product: Red Hat OpenStack Reporter: Marius Cornea <mcornea>
Component: openstack-tripleo-heat-templatesAssignee: Marios Andreou <mandreou>
Status: CLOSED ERRATA QA Contact: Marius Cornea <mcornea>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 13.0 (Queens)CC: gfidente, jschluet, mandreou, mbultel, mburns, pgrist, rhel-osp-director-maint, scohen, shardy
Target Milestone: betaKeywords: Triaged
Target Release: 13.0 (Queens)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: openstack-tripleo-common-8.6.1-3.el7ost openstack-tripleo-heat-templates-8.0.2-3.el7ost Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-06-27 13:46:53 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On:    
Bug Blocks: 1485413, 1488566    

Description Marius Cornea 2018-03-15 17:30:09 UTC
Description of problem:
FFU: post upgrading an environment with ceph osd nodes the controller nodes are missing ceph-mgr required firewall rules

After upgrade checking the ceph services related iptables rules on controller nodes:

[root@controller-0 heat-admin]# iptables -nL | grep ceph
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            multiport dports 6789 /* 110 ceph_mon */ state NEW

[root@controller-0 heat-admin]# docker ps | grep ceph
ae2d122f614c        registry.access.redhat.com/rhceph/rhceph-3-rhel7:latest                                                "/entrypoint.sh"         19 hours ago        Up 19 hours                                   ceph-mgr-controller-0
55aad6e09f2a        registry.access.redhat.com/rhceph/rhceph-3-rhel7:latest                                                "/entrypoint.sh"         19 hours ago        Up 19 hours                                   ceph-mon-controller-0

Expected results:
There should be an additional iptables rule installed for the ceph-mgr service allowing access to ports tcp 6800:7300 per https://github.com/openstack/tripleo-heat-templates/blob/master/docker/services/ceph-ansible/ceph-mgr.yaml#L59-L62

Comment 2 Giulio Fidente 2018-03-15 17:32:38 UTC
Looks like none of the Ceph services appear in Heat's list of enabled_services.

Comment 3 Giulio Fidente 2018-03-15 17:51:21 UTC
By inspecting the Heat stack, I can see CephMgr as one of the deployed stacks and resource-show shows [1] it's correctly mapped to docker/services/ceph-ansible/ceph-mgr.yaml

This might be an issue with the templates not refreshing [2] on upgrade.

Note that for FFU we explicitly disable Ceph services for the initial stack update [3], then we enable it back as one of the last upgrade steps.

1. http://ix.io/Xtq
2. https://github.com/openstack/tripleo-heat-templates/blob/master/overcloud.j2.yaml#L450-L460
3. https://github.com/openstack/tripleo-heat-templates/blob/master/environments/fast-forward-upgrade.yaml

Comment 16 errata-xmlrpc 2018-06-27 13:46:53 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2018:2086