Bug 1263801 - Postdeployment does not complete if overcloud deploy command in background
Summary: Postdeployment does not complete if overcloud deploy command in background
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: python-tripleoclient
Version: 7.0 (Kilo)
Hardware: Unspecified
OS: Unspecified
low
unspecified
Target Milestone: rc
: 11.0 (Ocata)
Assignee: Ben Nemec
QA Contact: Gurenko Alex
URL:
Whiteboard:
: 1329343 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-09-16 17:57 UTC by Dan Sneddon
Modified: 2017-06-16 21:08 UTC (History)
13 users (show)

Fixed In Version: python-tripleoclient-6.1.0-5.el7ost
Doc Type: Bug Fix
Doc Text:
Cause: Previous versions of OSP director had issues doing post-deployment steps if the deploy command was run in the background. Consequence: Deployments would hang during post-deployment until the command was brought into the foreground of the session. Fix: Post-deployment was re-designed so that the problematic steps are no longer done. Result: Deployment in the background now works as expected.
Clone Of:
Environment:
Last Closed: 2017-05-17 19:23:56 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
OpenStack gerrit 225247 0 None None None Never
Red Hat Product Errata RHEA-2017:1245 0 normal SHIPPED_LIVE Red Hat OpenStack Platform 11.0 Bug Fix and Enhancement Advisory 2017-05-17 23:01:50 UTC

Description Dan Sneddon 2015-09-16 17:57:15 UTC
Description of problem:
If you run openstack overcloud deploy with &> logfile, the process goes into the background. This results in the deployment hanging on one of the last steps in postdeployment, and the endpoints and security settings don't get completed.

Version-Release number of selected component (if applicable):
OSP 7 GA (probably affects Director 7.1 as well)

How reproducible:
100%

Steps to Reproduce:
1. Run the openstack overcloud deploy in the background
2.
3.

Actual results:
The stack gets to CREATE_COMPLETE, but in the postdeployment steps the script will hang.

Expected results:
The postdeployment should complete.

Additional info:
When the script was in this state, ps showed it as "stopped". When we foregrounded the process, the postdeployment immediately completed and the endpoints and security for the admin user was set up. This *might* have something to do with an SSH call requiring a TTY, but definitely only works when the process is in the foreground. This causes a lot of confusion, since the stack goes to CREATE_COMPLETE but the overcloud is not functional.

A simple workaround is just to make sure that the overcloud deploy process happens in the foreground. The output can be piped to tee to write it to a log file while keeping it in the foreground.

Comment 3 Ryan Brown 2015-09-18 17:10:25 UTC
Hi, I've put together a patchset to resolve this upstream - it's somewhat large so I'd want to put together a less invasive set of patches if we want this to make it into y2. 

https://review.openstack.org/#/q/status:open+project:openstack/python-tripleoclient+branch:master+topic:soloPostDeploy,n,z

Comment 4 Ben Nemec 2015-12-17 15:44:36 UTC
I'm aware of this happening in at least a couple of instances, but I also have a report that someone was able to deploy successfully with the process in the background, so it appears this isn't 100% reproducible.  Still unclear what the cause is.

Comment 5 Mike Burns 2016-04-07 20:50:54 UTC
This bug did not make the OSP 8.0 release.  It is being deferred to OSP 10.

Comment 7 Ben Nemec 2016-10-06 13:50:30 UTC
Note that this may be fixed in OSP 10, since we don't do post-deployment over ssh anymore.

Comment 9 Emilien Macchi 2017-04-20 20:26:38 UTC
*** Bug 1329343 has been marked as a duplicate of this bug. ***

Comment 11 Amit Ugol 2017-04-24 06:08:58 UTC
I couldn't reproduce it for a while and personally I always deploy to the background.

Comment 14 errata-xmlrpc 2017-05-17 19:23:56 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1245

Comment 15 Ben Nemec 2017-06-16 21:08:31 UTC
This did not show up in my doc-text query so I missed it.  Adding it now in case it's still useful.


Note You need to log in before you can comment on or make changes to this bug.