Bug 1834901
| Summary: | Re-running overcloud deploy ignores heat templates | ||
|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Jakub Libosvar <jlibosva> |
| Component: | openstack-tripleo-heat-templates | Assignee: | Emilien Macchi <emacchi> |
| Status: | CLOSED ERRATA | QA Contact: | Roman Safronov <rsafrono> |
| Severity: | urgent | Docs Contact: | |
| Priority: | high | ||
| Version: | 16.1 (Train) | CC: | aschultz, beagles, emacchi, jpretori, lbezdick, mbarnett, mburns, ramishra, sclewis, spower |
| Target Milestone: | rc | Keywords: | Regression, Triaged |
| Target Release: | 16.1 (Train on RHEL 8.2) | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | openstack-tripleo-heat-templates-11.3.2-0.20200616081527.396affd.el8ost | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2020-07-29 07:52:44 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
|
Description
Jakub Libosvar
2020-05-12 15:52:02 UTC
Looking at the neutron metadata agent for example, the json files for the service containers appear to be on the host under /var/lib/tripleo-config and the timestamps seem to match the other files indicating that they might be being regenerated. However, the hieradata for the metadata agent is missing. Perhaps some aspect of the deployment is dropping the service, but it hasn't been removed from some other data source created and used by the deployment framework? When you set a service to OS::Heat::None, the services are not removed from the host that were previously running. You need to have something to do that clean up. Usually we recommend switch it from the real service, to a service that describes all the removal actions. Can you please provide a full set of templates that were used and the command that were run? Currently there is not enough information to understand the order of actions or what is actually performed. (In reply to Alex Schultz from comment #2) > When you set a service to OS::Heat::None, the services are not removed from > the host that were previously running. You need to have something to do that > clean up. Usually we recommend switch it from the real service, to a > service that describes all the removal actions. The services used to be removed in OSP 16 GA when set to None. But this is not the problem this BZ aims on. The real problem is that even when I remove the service manually and I set it to None in templates, then it still gets configured regardless of the template settings. I will provide the full templates once I have the env back and I re-run the migration. Jakub where you able to reproduce this? I think the issue is that we shouldn't be setting them to None, but rather to a disabled service that is basically a noop service. We've done this in the past for services that we've removed in order to ensure they get properly handled for things like FFU or just a basic upgrade. Example https://opendev.org/openstack/tripleo-heat-templates/src/branch/stable/queens/puppet/services/disabled/ceilometer-api-disabled.yaml Ok so this is a regression from at least OSP13 and likely OSP16. Previously when you set a service to OS::Heat::None it would stop managing the service but leave it in place. This was likely caused by the fix for Bug 1726606 since we're likely removing the service definition which causes heat not to recognize that it should be removed from the stack. We likely need to check if there is a stack and a service is defined in the stack, do not remove the OS::Heat::None service. The work around would be to create a dummy/empty service to use instead of defining OS::Heat::None when you are removing a services. This issue shows up in the ml2->ovn migration because we're running something externally to the deployment to do the migration rather than properly handling it during a deploy/update/upgrade procedure via deployment steps/host prep tasks/external tasks or soemthing to that effect. openstack-tripleo-common-11.3.3-0.20200403044649.56c0fd5.el8ost.noarch/openstack-tripleo-common-containers-11.3.3-0.20200403044649.56c0fd5.el8ost.noarch look pretty old to have the fix for Bug 1726606 and cause the issue. So I attempted to reproduce this with 16.1 by deploying an overcloud, then disabling OS::TripleO::Services::Chrony by setting it to OS::Heat::None. It didn't reproduce as the chrony tasks were not present on the subsequent update. I'll now try with the ml2 -> ovn process with 16.0 to see if it's specific to that. I think https://review.opendev.org/#/c/737337/ will fix the issue, where it'll clean up all containers that aren't supposed to be on a host or re-run once FFU is finished. I checked the ovn migration and the old containers are started up but they aren't in the ansible playbook so it looks like Heat is doing the correct thing and it's likely the bug resolved via https://review.opendev.org/#/c/737340/ Verified on RHOS-16.1-RHEL-8-20200625.n.0 with openstack-tripleo-heat-templates-11.3.2-0.20200616081529.396affd.el8ost.noarch Verified that ml2ovs services are not running after migration to ml2ovn. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:3148 |