Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1876153

Summary: Using split-stack leads to large number of duplicates in /etc/hosts for undercloud.ctlplane over time
Product: Red Hat OpenStack Reporter: Kellen Gattis <kgattis>
Component: python-tripleoclientAssignee: Emilien Macchi <emacchi>
Status: CLOSED ERRATA QA Contact: David Rosenfeld <drosenfe>
Severity: medium Docs Contact:
Priority: medium    
Version: 16.1 (Train)CC: cjeanner, emacchi, gregraka, hbrock, jamsmith, jslagle, lmiccini, mbultel, mburns, michele
Target Milestone: ---Keywords: Triaged, ZStream
Target Release: ---Flags: kgattis: needinfo-
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: python-tripleoclient-12.3.2-1.20200815213404.29686a6.el8ost Doc Type: If docs needed, set a value
Doc Text:
The entry in `/etc/hosts` for the undercloud duplicates anytime the Compute stack is updated on the undercloud and overcloud nodes. This occurs for split-stack deployments where the Controllers and Compute nodes are divided into multiple stacks. + Other indications of this problem are the following: + ** mysql reporting errors about packets exceeding their maximum size. ** The Orchestration service (heat) warning that templates are exceeding their maximum size. ** The Workflow service (mistral) warning that fields are exceeding their maximum size. As a workaround, in the file generated by running the `openstack overcloud export` command that is included in the Compute stack, under `ExtraHostFileEntries`, remove the erroneous entry for the undercloud.
 Story Points: ---
Clone Of: Environment:
Last Closed: 2020-10-28 15:39:36 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Kellen Gattis 2020-09-05 21:00:05 UTC 
Description of problem:
When using a split-stack deployment where the Controllers and Computes are split into multiple stacks, the entry in /etc/hosts for the undercloud starts duplicating with each subsequent compute stack update on the undercloud and overcloud nodes.

Some symptoms of this include an entry in /etc/hosts that could look like this:
192.168.10.11 undercloud.ctlplane.example.org undercloud.ctlplane undercloud.ctlplane undercloud.ctlplane undercloud.ctlplane undercloud.ctlplane undercloud.ctlplane undercloud.ctlplane undercloud.ctlplane undercloud.ctlplane undercloud.ctlplane undercloud.ctlplane undercloud.ctlplane undercloud.ctlplane undercloud.ctlplane undercloud.ctlplane undercloud.ctlplane undercloud.ctlplane undercloud.ctlplane 

Other signs that this is happening include mysql complaining about packets exceeding max size, heat complaining about templates exceeding max size, and mistral complaining about fields exceeding max size.

When looking at the file generated by running "openstack overcloud export" that is included in the compute stack, I noticed that there was an entry in "ExtraHostFileEntries" for the undercloud.  Removing this entry from the file stopped the /etc/hosts entry storm.

As a workaround, one can simply delete the undercloud entry in ExtraHostFileEntries whenever the central site export file is generated and everything works as expected.


Version-Release number of selected component (if applicable):
python3-tripleoclient-12.3.2-0.20200615103427.6f877f6.el8ost.noarch

How reproducible:
Very

Steps to Reproduce:
1. Deploy a controller stack
2. Export controller (central) stack to template file
Docs: https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/16.1/html/distributed_compute_node_and_storage_deployment/configuring_and_installing_the_hub_undercloud#deploying-edge-nodes-without-storage
3. Deploy a compute stack using exported file from step #2

Actual results:
/etc/hosts grows with each compute stack update

Expected results:
/etc/hosts has one entry for the undercloud that is stable

Additional info:
https://bugzilla.redhat.com/show_bug.cgi?id=1842919 seems like a similar user experience
Comment 1 Michele Baldessari 2020-09-07 06:46:08 UTC 
Does applying https://review.opendev.org/#/c/740543/ to your undercloud fix it in your case?
Comment 3 Kellen Gattis 2020-09-10 01:32:55 UTC 
(In reply to Michele Baldessari from comment #1)
> Does applying https://review.opendev.org/#/c/740543/ to your undercloud fix
> it in your case?

I applied the patch and it definitely helped to stop undercloud.ctlplane from multiplying in /etc/hosts - thanks for the recommendation.

The only oddity after applying the patch was that I ended up with two identical lines in /etc/hosts for the undercloud entry on both the undercloud and the overcloud compute stack nodes after I ran the compute stack creation/update.  I never saw more than two undercloud lines appear regardless of how many stack updates I ran and I didn't notice any misbehaving operation as a result, so pretty harmless overall.  I removed the undercloud entry from the central export file's ExtraHostFileEntries (same procedure as the workaround mentioned in the bugzilla's description of the problem) and the double entry went away when I performed a subsequent stack update.
Comment 11 errata-xmlrpc 2020-10-28 15:39:36 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat OpenStack Platform 16.1 bug fix and enhancement advisory), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2020:4284