Bug 1263801

Summary:	Postdeployment does not complete if overcloud deploy command in background
Product:	Red Hat OpenStack	Reporter:	Dan Sneddon <dsneddon>
Component:	python-tripleoclient	Assignee:	Ben Nemec <bnemec>
Status:	CLOSED ERRATA	QA Contact:	Gurenko Alex <agurenko>
Severity:	unspecified	Docs Contact:
Priority:	low
Version:	7.0 (Kilo)	CC:	achernet, augol, bnemec, dmatthew, hbrock, jcoufal, joea, jschluet, jslagle, lbopf, mburns, rhel-osp-director-maint, rybrown
Target Milestone:	rc	Keywords:	TestOnly, Triaged
Target Release:	11.0 (Ocata)
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	python-tripleoclient-6.1.0-5.el7ost	Doc Type:	Bug Fix
Doc Text:	Cause: Previous versions of OSP director had issues doing post-deployment steps if the deploy command was run in the background. Consequence: Deployments would hang during post-deployment until the command was brought into the foreground of the session. Fix: Post-deployment was re-designed so that the problematic steps are no longer done. Result: Deployment in the background now works as expected.	Story Points:	---
Clone Of:		Environment:
Last Closed:	2017-05-17 19:23:56 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Dan Sneddon 2015-09-16 17:57:15 UTC

Description of problem:
If you run openstack overcloud deploy with &> logfile, the process goes into the background. This results in the deployment hanging on one of the last steps in postdeployment, and the endpoints and security settings don't get completed.

Version-Release number of selected component (if applicable):
OSP 7 GA (probably affects Director 7.1 as well)

How reproducible:
100%

Steps to Reproduce:
1. Run the openstack overcloud deploy in the background
2.
3.

Actual results:
The stack gets to CREATE_COMPLETE, but in the postdeployment steps the script will hang.

Expected results:
The postdeployment should complete.

Additional info:
When the script was in this state, ps showed it as "stopped". When we foregrounded the process, the postdeployment immediately completed and the endpoints and security for the admin user was set up. This *might* have something to do with an SSH call requiring a TTY, but definitely only works when the process is in the foreground. This causes a lot of confusion, since the stack goes to CREATE_COMPLETE but the overcloud is not functional.

A simple workaround is just to make sure that the overcloud deploy process happens in the foreground. The output can be piped to tee to write it to a log file while keeping it in the foreground.

Comment 3 Ryan Brown 2015-09-18 17:10:25 UTC

Hi, I've put together a patchset to resolve this upstream - it's somewhat large so I'd want to put together a less invasive set of patches if we want this to make it into y2. 

https://review.openstack.org/#/q/status:open+project:openstack/python-tripleoclient+branch:master+topic:soloPostDeploy,n,z

Comment 4 Ben Nemec 2015-12-17 15:44:36 UTC

I'm aware of this happening in at least a couple of instances, but I also have a report that someone was able to deploy successfully with the process in the background, so it appears this isn't 100% reproducible.  Still unclear what the cause is.

Comment 5 Mike Burns 2016-04-07 20:50:54 UTC

This bug did not make the OSP 8.0 release.  It is being deferred to OSP 10.

Comment 7 Ben Nemec 2016-10-06 13:50:30 UTC

Note that this may be fixed in OSP 10, since we don't do post-deployment over ssh anymore.

Comment 9 Emilien Macchi 2017-04-20 20:26:38 UTC

*** Bug 1329343 has been marked as a duplicate of this bug. ***

Comment 11 Amit Ugol 2017-04-24 06:08:58 UTC

I couldn't reproduce it for a while and personally I always deploy to the background.

Comment 14 errata-xmlrpc 2017-05-17 19:23:56 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1245

Comment 15 Ben Nemec 2017-06-16 21:08:31 UTC

This did not show up in my doc-text query so I missed it.  Adding it now in case it's still useful.