Bug 1784706

Summary:	Duplicate horizon, keystone containers getting launched during overcloud deploy process
Product:	Red Hat OpenStack	Reporter:	shyam.biradar
Component:	tripleo-ansible	Assignee:	Emilien Macchi <emacchi>
Status:	CLOSED INSUFFICIENT_DATA	QA Contact:	Sasha Smolyak <ssmolyak>
Severity:	urgent	Docs Contact:
Priority:	unspecified
Version:	14.0 (Rocky)	CC:	aschultz, emacchi, shyam.biradar
Target Milestone:	---	Keywords:	Triaged, ZStream
Target Release:	---
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2020-01-06 15:04:02 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description shyam.biradar 2019-12-18 06:37:53 UTC

Description of problem:
During overcloud deploy command execution, duplicate containers getting launched for horizon and keystone.



Version-Release number of selected component (if applicable):
RHOSP14


How reproducible:
Always reproducible

Steps to Reproduce:
1. Run "openstack overcloud deploy..." command, overcloud stack create completes but deployment fails at ansible step x(1-5)
2. Run "openstack overcloud deploy --config-download-only..." command to just apply ansible configuration
3. The above command creates duplicate horizon and keystone containers with different names(horizon, horizon_<random_string>) trying to listen on the same port. So one horizon container keeps restarting with error "address already in use", while other one is in healthy state.



Actual results:
Openstack overcloud deploy command should create only one container of a given service and on the next cloud update,  if it finds that the container already exists, it should not launch new duplicate container. It should skip container creation part.

Expected results:
openstack overcloud deploy command creating duplicate containers for same service listening on the same port


Additional info:

Command showing duplicate containers(Note: I have manually stopped duplicate containers which were in 'restarting' state):

[root@overcloud-controller-0 ~]# docker ps -a | grep horizon
5d89f5fd3faf 172.172.3.151:8787/rhosp14/openstack-horizon:14.0-124 "kolla_start" About an hour ago Exited (1) 18 minutes ago horizon-eilkqsif
f07170d17117 172.172.3.151:8787/rhosp14/openstack-horizon:14.0-124 "kolla_start" 3 hours ago Exited (1) 2 hours ago horizon-slbfkwmf
aeeea02c6125 172.172.3.151:8787/rhosp14/openstack-horizon:14.0-124 "kolla_start" 3 hours ago Up 3 hours horizon
c9b3020ee979 172.172.3.151:8787/rhosp14/openstack-horizon:14.0-124 "/bin/bash -c 'tou..." 16 hours ago Exited (0) 16 hours ago horizon_fix_perms

[root@overcloud-controller-0 heat-admin]# docker ps -a | grep keystone
98be6a894c3e 172.172.3.151:8787/rhosp14/openstack-keystone:14.0-119 "/bin/bash -c '/us..." About an hour ago Up About an hour keystone_cron-u3l5opvw
277bc5a9c47b 172.172.3.151:8787/rhosp14/openstack-keystone:14.0-119 "/bin/bash -c '/us..." About an hour ago Up About an hour keystone_cron
0bb4a84baac9 172.172.3.151:8787/rhosp14/openstack-keystone:14.0-119 "kolla_start" 3 hours ago Exited (1) 2 hours ago keystone-8qsm5jpl
1852c3e4d116 172.172.3.151:8787/rhosp14/openstack-keystone:14.0-119 "kolla_start" 3 hours ago Up 3 hours (healthy) keystone
1e61d427c852 172.172.3.151:8787/rhosp14/openstack-keystone:14.0-119 "/usr/bin/bootstra..." 3 hours ago Exited (1) 3 hours ago keystone_db_sync-lzkfykbg
287314b2f2fa 172.172.3.151:8787/rhosp14/openstack-keystone:14.0-119 "/usr/bin/bootstra..." 3 hours ago Exited (0) 3 hours ago keystone_db_sync
e4242f3af966 172.172.3.151:8787/rhosp14/openstack-keystone:14.0-119 "/bin/bash -c 'cho..." 15 hours ago Exited (0) 15 hours ago keystone_init_log

Comment 1 shyam.biradar 2019-12-18 06:42:22 UTC

The impact is our overcloud deployment consistently failing.

Comment 2 Emilien Macchi 2019-12-18 13:11:15 UTC

Please provide the templates and roles on how you deployed the cloud, it's probably a template or user error.

Comment 3 Alex Schultz 2020-01-06 15:04:02 UTC

Please provide the templates or additional logs if this is still an issue. Thanks.

Comment 4 Red Hat Bugzilla 2023-09-14 05:48:54 UTC

The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days