Bug 1274840 - [RFE] 'openstack overcloud update' should produce messages that truly reflect if update process is running.
[RFE] 'openstack overcloud update' should produce messages that truly reflect...
Status: ASSIGNED
Product: Red Hat OpenStack
Classification: Red Hat
Component: rhosp-director (Show other bugs)
7.0 (Kilo)
x86_64 Linux
medium Severity medium
: ---
: ---
Assigned To: Julie Pichon
Ola Pavlenko
NeedsAllocation
: FutureFeature
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-10-23 11:44 EDT by Omri Hochman
Modified: 2018-02-16 05:25 EST (History)
9 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Omri Hochman 2015-10-23 11:44:40 EDT
'openstack overcloud update' should produce messages that truly reflect if update process is running.

Environment :
--------------
instack-undercloud-2.1.2-29.el7ost.noarch
instack-0.0.7-1.el7ost.noarch
openstack-heat-templates-0-0.6.20150605git.el7ost.noarch
openstack-tripleo-heat-templates-0.8.6-71.el7ost.noarc

Description:
-------------
I'm trying to update undercloud and overcloud from 7.0 to 7.1 : 

Currently when running 'openstack overcloud update' even in case nothing is happens the command shows IN_PROGRESS ... IN_PROGRESS ... IN_PROGRESS...  until time out..  

[stack@instack ~]$ openstack overcloud update stack overcloud -i --templates -e /usr/share/openstack-tripleo-heat-templates/overcloud-resource-registry-puppet.yaml -e /home/stack/update.yaml
IN_PROGRESS
IN_PROGRESS
IN_PROGRESS
IN_PROGRESS
IN_PROGRESS
IN_PROGRESS
IN_PROGRESS
IN_PROGRESS
IN_PROGRESS
IN_PROGRESS
IN_PROGRESS
IN_PROGRESS
IN_PROGRESS
IN_PROGRESS

We should have messages which are more useful that shows the user :
 
 - What update action is running  ? on which host? 
 - The message should truly reflect if the process is running 
 - In case of failure to run 'yum update' on the overcloud nodes, 
   the process should stop and not wait for timeout. 

suggestions ^ fix not have to answer all criteria
Comment 2 Jan Provaznik 2015-10-23 12:13:34 EDT
Sounds like a heat bug in this case:
[stack@instack ~]$ heat stack-list 
+--------------------------------------+------------+--------------------+----------------------+
| id                                   | stack_name | stack_status       | creation_time        |
+--------------------------------------+------------+--------------------+----------------------+
| 83983e64-ef46-4a3c-89e3-7410653e9380 | overcloud  | UPDATE_IN_PROGRESS | 2015-10-22T21:33:16Z |
+--------------------------------------+------------+--------------------+----------------------+

[stack@instack ~]$ heat stack-list -n|grep PROG
| 83983e64-ef46-4a3c-89e3-7410653e9380 | overcloud                                                                                                     | UPDATE_IN_PROGRESS | 2015-10-22T21:33:16Z | None                                 |
[stack@instack ~]$ 

All resources are in OCMPLETE state (progress didn't start). In heat log:
-- Logs begin at Thu 2015-10-22 16:27:37 EDT, end at Fri 2015-10-23 12:11:18 EDT. --
Oct 23 10:54:17 instack.localdomain heat-engine[30198]: Traceback (most recent call last):
Oct 23 10:54:17 instack.localdomain heat-engine[30198]: File "/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 457, in fire_timers
Oct 23 10:54:17 instack.localdomain heat-engine[30198]: timer()
Oct 23 10:54:17 instack.localdomain heat-engine[30198]: File "/usr/lib/python2.7/site-packages/eventlet/hubs/timer.py", line 58, in __call__
Oct 23 10:54:17 instack.localdomain heat-engine[30198]: cb(*args, **kw)
Oct 23 10:54:17 instack.localdomain heat-engine[30198]: File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 214, in main
Oct 23 10:54:17 instack.localdomain heat-engine[30198]: result = function(*args, **kwargs)
Oct 23 10:54:17 instack.localdomain heat-engine[30198]: File "/usr/lib/python2.7/site-packages/heat/engine/service.py", line 111, in _start_with_trace
Oct 23 10:54:17 instack.localdomain heat-engine[30198]: profiler.init(**trace)
Oct 23 10:54:17 instack.localdomain heat-engine[30198]: File "/usr/lib/python2.7/site-packages/osprofiler/profiler.py", line 105, in wrapper
Oct 23 10:54:17 instack.localdomain heat-engine[30198]: return f(*args, **kwargs)
Oct 23 10:54:17 instack.localdomain heat-engine[30198]: File "/usr/lib/python2.7/site-packages/heat/engine/stack.py", line 867, in update
Oct 23 10:54:17 instack.localdomain heat-engine[30198]: @scheduler.wrappertask
Oct 23 10:54:17 instack.localdomain heat-engine[30198]: File "/usr/lib/python2.7/site-packages/heat/engine/scheduler.py", line 174, in __call__
Oct 23 10:54:17 instack.localdomain heat-engine[30198]: self.start(timeout=timeout)
Oct 23 10:54:17 instack.localdomain heat-engine[30198]: File "/usr/lib/python2.7/site-packages/heat/engine/scheduler.py", line 200, in start
Oct 23 10:54:17 instack.localdomain heat-engine[30198]: self.step()
Oct 23 10:54:17 instack.localdomain heat-engine[30198]: File "/usr/lib/python2.7/site-packages/heat/engine/scheduler.py", line 223, in step
Oct 23 10:54:17 instack.localdomain heat-engine[30198]: next(self._runner)
Oct 23 10:54:17 instack.localdomain heat-engine[30198]: File "/usr/lib/python2.7/site-packages/heat/engine/scheduler.py", line 289, in wrapper
Oct 23 10:54:17 instack.localdomain heat-engine[30198]: subtask = next(parent)
Oct 23 10:54:17 instack.localdomain heat-engine[30198]: File "/usr/lib/python2.7/site-packages/heat/engine/stack.py", line 918, in update_task
Oct 23 10:54:17 instack.localdomain heat-engine[30198]: updater.start(timeout=self.timeout_secs())
Oct 23 10:54:17 instack.localdomain heat-engine[30198]: File "/usr/lib/python2.7/site-packages/heat/engine/scheduler.py", line 200, in start
Oct 23 10:54:17 instack.localdomain heat-engine[30198]: self.step()
Oct 23 10:54:17 instack.localdomain heat-engine[30198]: File "/usr/lib/python2.7/site-packages/heat/engine/scheduler.py", line 223, in step
Oct 23 10:54:17 instack.localdomain heat-engine[30198]: next(self._runner)
Oct 23 10:54:17 instack.localdomain heat-engine[30198]: File "/usr/lib/python2.7/site-packages/heat/engine/scheduler.py", line 289, in wrapper
Oct 23 10:54:17 instack.localdomain heat-engine[30198]: subtask = next(parent)
Oct 23 10:54:17 instack.localdomain heat-engine[30198]: File "/usr/lib/python2.7/site-packages/heat/engine/update.py", line 60, in __call__
Oct 23 10:54:17 instack.localdomain heat-engine[30198]: self.dependencies(),
Oct 23 10:54:17 instack.localdomain heat-engine[30198]: File "/usr/lib/python2.7/site-packages/heat/engine/update.py", line 185, in dependencies
Oct 23 10:54:17 instack.localdomain heat-engine[30198]: if res_name in self.previous_stack:
Oct 23 10:54:17 instack.localdomain heat-engine[30198]: File "/usr/lib/python2.7/site-packages/heat/engine/stack.py", line 240, in dependencies
Oct 23 10:54:17 instack.localdomain heat-engine[30198]: File "/usr/lib/python2.7/site-packages/heat/engine/stack.py", line 312, in _get_dependencies
Oct 23 10:54:17 instack.localdomain heat-engine[30198]: return deps
Oct 23 10:54:17 instack.localdomain heat-engine[30198]: File "/usr/lib/python2.7/site-packages/heat/engine/resources/openstack/neutron/port.py", line 263, in add_dependencies
Oct 23 10:54:17 instack.localdomain heat-engine[30198]: if res.has_interface('OS::Neutron::Subnet'):
Oct 23 10:54:17 instack.localdomain heat-engine[30198]: File "/usr/lib/python2.7/site-packages/heat/engine/resource.py", line 319, in has_interface
Oct 23 10:54:17 instack.localdomain heat-engine[30198]: def type(self):
Oct 23 10:54:17 instack.localdomain heat-engine[30198]: AttributeError: 'NoneType' object has no attribute 'name'
Comment 3 Omri Hochman 2015-10-23 13:23:15 EDT
(In reply to Jan Provaznik from comment #2)
> Sounds like a heat bug in this case:

Opened another Bz for the heat issue : 
https://bugzilla.redhat.com/show_bug.cgi?id=1274859
Comment 4 Zane Bitter 2015-11-02 13:04:47 EST
Yeah, as Jan said the immediate cause of the failure is a Heat bug, and the cause of the failure to report the failure is another Heat bug (upstream: https://bugs.launchpad.net/heat/+bug/1492433). The client is doing the best it can with the information it's getting from Heat here.

Ideally we'd be able to stop using breakpoints for this altogether so we can get rid of the interactive part of the client and let Heat handle everything, but that depends on new Heat features in Liberty so it will have to wait for OSP8.
Comment 6 Mike Burns 2016-04-07 16:54:03 EDT
This bug did not make the OSP 8.0 release.  It is being deferred to OSP 10.
Comment 8 Jason E. Rist 2016-10-14 18:52:16 EDT
This fix is merged upstream, can we please get verification as to whether this is still an issue?
Comment 9 Omri Hochman 2017-01-18 12:11:38 EST
When user is running the update process from command line - the user will get on the screen something such : 

IN_PROGRESS
IN_PROGRESS
IN_PROGRESS
IN_PROGRESS
IN_PROGRESS
IN_PROGRESS
IN_PROGRESS
IN_PROGRESS


I believe that the original request of this bug, since the update can take a while (hours) , is to provide the user output messages with more meaningful information than ' IN_PROGRESS'    - something that indicates :  - What update action is running  ? on which host? etc.. 

But i also think it's more of an RFE than Bz, and that a PM should prioritize this according to how the update procedure should be reflected, in the UI.. or if we would want to improve those "messages" that displays during OSP minor-update when been executed from command-line.

Note You need to log in before you can comment on or make changes to this bug.