Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1245826

Summary: Overcloud package update cmd not waiting for hooks
Product: Red Hat OpenStack Reporter: Omri Hochman <ohochman>
Component: python-rdomanager-oscpluginAssignee: Jan Provaznik <jprovazn>
Status: CLOSED WONTFIX QA Contact: Shai Revivo <srevivo>
Severity: high Docs Contact:
Priority: high    
Version: unspecifiedCC: dmacpher, hbrock, jcoufal, jslagle, mburns, rhel-osp-director-maint, zbitter
Target Milestone: ---Keywords: Reopened
Target Release: 10.0 (Newton)   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Known Issue
Doc Text:
The "openstack overcloud update stack" command returns immediately despite ongoing operations in the background. The command seems to run forever because it's not interactive. In these situations, run the command with the "-i" flag. This prompts the user for any manual interaction needs.
 Story Points: ---
Clone Of: Environment:
Last Closed: 2016-10-05 19:36:10 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
instack-messages none

Description Omri Hochman 2015-07-22 21:18:15 UTC 
Attempt to update the overcloud results with UPDATE_IN_PROGRESS forever. 


Environment:
------------
instack-undercloud-2.1.2-21.el7ost.noarch
python-rdomanager-oscplugin-0.0.8-41.el7ost.noarch
openstack-tripleo-puppet-elements-0.0.1-4.el7ost.noarch
openstack-tripleo-heat-templates-0.8.6-44.el7ost.noarch
openstack-tripleo-common-0.0.1.dev6-1.git49b57eb.el7ost.noarch
openstack-tripleo-0.0.7-0.1.1664e566.el7ost.noarch
openstack-tripleo-image-elements-0.9.6-6.el7ost.noarch



Steps : 
--------- 
(1) deployed HA with network-isolation on Bare-Metal - with 3 X controller, 1 X compute , 1 x Ceph

(2) After Overcloud complete successfully attempt to Update the overcloud by running  : 

openstack overcloud update stack overcloud --plan overcloud -e /home/stack/network-environment.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml   


Results: 
----------
(1) the CLI return quickly ( with status: COMPLETE ) 

[stack@rhos-compute-node-18 ~]$ openstack overcloud update stack overcloud --plan overcloud -e /home/stack/network-environment.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml
starting package update on stack overcloud
stack overcloud status: COMPLETE


(2) stack remain in UPDATE_IN_PROGRESS forever, 
[stack@rhos-compute-node-18 ~]$ heat stack-list
+--------------------------------------+------------+--------------------+----------------------+
| id                                   | stack_name | stack_status       | creation_time        |
+--------------------------------------+------------+--------------------+----------------------+
| 67d49c4e-da3f-4f7d-962d-ee12c10289e4 | overcloud  | UPDATE_IN_PROGRESS | 2015-07-22T19:07:51Z |
+--------------------------------------+------------+--------------------+----------------------+



[stack@rhos-compute-node-18 ~]$ heat hook-poll -n5 overcloud
+------------------+--------------------------------------+------------------------------------------------+-----------------+----------------------+----------------------------------------------------+
| resource_name    | id                                   | resource_status_reason                         | resource_status | event_time           | stack_name                                         |
+------------------+--------------------------------------+------------------------------------------------+-----------------+----------------------+----------------------------------------------------+
| UpdateDeployment | 36d1b943-7cb6-4cf6-b448-9c762686cb41 | UPDATE paused until Hook pre-update is cleared | CREATE_COMPLETE | 2015-07-22T19:47:58Z | overcloud-Controller-767fuwebsqx5-2-6dm23hpmj7ak   |
| UpdateDeployment | 126782ac-0fce-48ae-bd63-b3e9164afb09 | UPDATE paused until Hook pre-update is cleared | CREATE_COMPLETE | 2015-07-22T19:48:29Z | overcloud-Controller-767fuwebsqx5-1-2t52zx7tmwe4   |
| UpdateDeployment | 0109f189-7c58-444f-a8a0-9570e27c7257 | UPDATE paused until Hook pre-update is cleared | CREATE_COMPLETE | 2015-07-22T19:47:23Z | overcloud-Ceph-Storage-y33mejyglqa6-0-xwsyeyco2sjn |
| UpdateDeployment | 54cf3e44-b8c9-4e97-87f0-26b137ff0f98 | UPDATE paused until Hook pre-update is cleared | CREATE_COMPLETE | 2015-07-22T19:48:13Z | overcloud-Controller-767fuwebsqx5-0-64ifaeobmahg   |
| UpdateDeployment | 1359aee8-9ae3-48f6-b88b-eb36e8badf6f | UPDATE paused until Hook pre-update is cleared | CREATE_COMPLETE | 2015-07-22T19:47:58Z | overcloud-Compute-rwbgku7pcpk6-0-tdapm5n53gus      |
+------------------+--------------------------------------+------------------------------------------------+-----------------+----------------------+----------------------------------------------------+
Comment 1 Omri Hochman 2015-07-22 21:25:26 UTC 
Created attachment 1055078 [details]
instack-messages
Comment 2 Zane Bitter 2015-07-22 21:26:14 UTC 
Basically the CLI command is exiting immediately instead of giving the user the chance to interactively clear the hooks. Since the stack is not guaranteed to move to UPDATE_IN_PROGRESS before returning from the heat stack-update command, most likely we are seeing the COMPLETE status from a previous operation and misinterpreting it as the update being complete.

Note that the issue is likely in openstack-tripleo-common, but we don't have a bugzilla component for that.
Comment 3 Jan Provaznik 2015-07-23 06:07:34 UTC 
You can use '-i' (interactive) param to run through whole package update process and wait for hooks ('-i' is in doc too). Minor bug is that "COMPLETE" status is returned right after triggering the update (because of the reason described by Zane) but this shouldn't have an impact on the update process itself.
Comment 4 Jan Provaznik 2015-07-23 12:05:50 UTC 
If you abort already running "openstack overcloud update stack overcloud -i" you can anytime get back to it just by re-running the command again (the command checks if update is already in progress and if so it continues with the existing update).

IMO this can be closed as "not a bug" if you agree?
Comment 5 Zane Bitter 2015-07-23 12:34:34 UTC 
We could probably expand the docs to specifically mention -i and why it is needed (and what to do if you forget to add it). I can't actually imagine any reason that you'd not want to specify that option.
Comment 6 Jan Provaznik 2015-07-23 14:21:48 UTC 
doc patch https://review.gerrithub.io/241146 adds explicit note about using interactive mode
Comment 7 Mike Burns 2015-08-19 16:29:02 UTC 
The docs are updated to state that -i is required, so closing this bug.
Comment 8 Mike Burns 2016-04-07 20:43:53 UTC 
This bug did not make the OSP 8.0 release.  It is being deferred to OSP 10.
Comment 10 Jaromir Coufal 2016-10-05 19:36:10 UTC 
Obsolete. Please re-open if still valid.