Red Hat Bugzilla – Bug 1245826
Overcloud package update cmd not waiting for hooks
Last modified: 2016-10-05 15:36:10 EDT
Attempt to update the overcloud results with UPDATE_IN_PROGRESS forever. Environment: ------------ instack-undercloud-2.1.2-21.el7ost.noarch python-rdomanager-oscplugin-0.0.8-41.el7ost.noarch openstack-tripleo-puppet-elements-0.0.1-4.el7ost.noarch openstack-tripleo-heat-templates-0.8.6-44.el7ost.noarch openstack-tripleo-common-0.0.1.dev6-1.git49b57eb.el7ost.noarch openstack-tripleo-0.0.7-0.1.1664e566.el7ost.noarch openstack-tripleo-image-elements-0.9.6-6.el7ost.noarch Steps : --------- (1) deployed HA with network-isolation on Bare-Metal - with 3 X controller, 1 X compute , 1 x Ceph (2) After Overcloud complete successfully attempt to Update the overcloud by running : openstack overcloud update stack overcloud --plan overcloud -e /home/stack/network-environment.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml Results: ---------- (1) the CLI return quickly ( with status: COMPLETE ) [stack@rhos-compute-node-18 ~]$ openstack overcloud update stack overcloud --plan overcloud -e /home/stack/network-environment.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml starting package update on stack overcloud stack overcloud status: COMPLETE (2) stack remain in UPDATE_IN_PROGRESS forever, [stack@rhos-compute-node-18 ~]$ heat stack-list +--------------------------------------+------------+--------------------+----------------------+ | id | stack_name | stack_status | creation_time | +--------------------------------------+------------+--------------------+----------------------+ | 67d49c4e-da3f-4f7d-962d-ee12c10289e4 | overcloud | UPDATE_IN_PROGRESS | 2015-07-22T19:07:51Z | +--------------------------------------+------------+--------------------+----------------------+ [stack@rhos-compute-node-18 ~]$ heat hook-poll -n5 overcloud +------------------+--------------------------------------+------------------------------------------------+-----------------+----------------------+----------------------------------------------------+ | resource_name | id | resource_status_reason | resource_status | event_time | stack_name | +------------------+--------------------------------------+------------------------------------------------+-----------------+----------------------+----------------------------------------------------+ | UpdateDeployment | 36d1b943-7cb6-4cf6-b448-9c762686cb41 | UPDATE paused until Hook pre-update is cleared | CREATE_COMPLETE | 2015-07-22T19:47:58Z | overcloud-Controller-767fuwebsqx5-2-6dm23hpmj7ak | | UpdateDeployment | 126782ac-0fce-48ae-bd63-b3e9164afb09 | UPDATE paused until Hook pre-update is cleared | CREATE_COMPLETE | 2015-07-22T19:48:29Z | overcloud-Controller-767fuwebsqx5-1-2t52zx7tmwe4 | | UpdateDeployment | 0109f189-7c58-444f-a8a0-9570e27c7257 | UPDATE paused until Hook pre-update is cleared | CREATE_COMPLETE | 2015-07-22T19:47:23Z | overcloud-Ceph-Storage-y33mejyglqa6-0-xwsyeyco2sjn | | UpdateDeployment | 54cf3e44-b8c9-4e97-87f0-26b137ff0f98 | UPDATE paused until Hook pre-update is cleared | CREATE_COMPLETE | 2015-07-22T19:48:13Z | overcloud-Controller-767fuwebsqx5-0-64ifaeobmahg | | UpdateDeployment | 1359aee8-9ae3-48f6-b88b-eb36e8badf6f | UPDATE paused until Hook pre-update is cleared | CREATE_COMPLETE | 2015-07-22T19:47:58Z | overcloud-Compute-rwbgku7pcpk6-0-tdapm5n53gus | +------------------+--------------------------------------+------------------------------------------------+-----------------+----------------------+----------------------------------------------------+
Created attachment 1055078 [details] instack-messages
Basically the CLI command is exiting immediately instead of giving the user the chance to interactively clear the hooks. Since the stack is not guaranteed to move to UPDATE_IN_PROGRESS before returning from the heat stack-update command, most likely we are seeing the COMPLETE status from a previous operation and misinterpreting it as the update being complete. Note that the issue is likely in openstack-tripleo-common, but we don't have a bugzilla component for that.
You can use '-i' (interactive) param to run through whole package update process and wait for hooks ('-i' is in doc too). Minor bug is that "COMPLETE" status is returned right after triggering the update (because of the reason described by Zane) but this shouldn't have an impact on the update process itself.
If you abort already running "openstack overcloud update stack overcloud -i" you can anytime get back to it just by re-running the command again (the command checks if update is already in progress and if so it continues with the existing update). IMO this can be closed as "not a bug" if you agree?
We could probably expand the docs to specifically mention -i and why it is needed (and what to do if you forget to add it). I can't actually imagine any reason that you'd not want to specify that option.
doc patch https://review.gerrithub.io/241146 adds explicit note about using interactive mode
The docs are updated to state that -i is required, so closing this bug.
This bug did not make the OSP 8.0 release. It is being deferred to OSP 10.
Obsolete. Please re-open if still valid.