Bug 1290303 - TUI crash when check Hosted Engine VM status while ovirt-ha-agent.service is inactive
TUI crash when check Hosted Engine VM status while ovirt-ha-agent.service is ...
Status: CLOSED ERRATA
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-node-plugin-hosted-engine (Show other bugs)
3.6.0
Unspecified Unspecified
high Severity urgent
: ovirt-3.6.2
: 3.6.0
Assigned To: Douglas Schilling Landgraf
wanghui
node
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-12-10 01:46 EST by wanghui
Modified: 2016-03-09 09:44 EST (History)
9 users (show)

See Also:
Fixed In Version: ovirt-node-plugin-hosted-engine-0.3.0-6.el7ev
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-03-09 09:44:08 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: Node
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
ovirt-node.log file (1.08 MB, application/x-gzip)
2015-12-10 01:46 EST, wanghui
no flags Details
varlog_rhevh_20151221.1. (1.07 MB, application/x-gzip)
2015-12-24 05:01 EST, Ying Cui
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
oVirt gerrit 51406 ovirt-3.6 MERGED hosted_engine_page: try/except for he --vm-status 2016-01-06 06:24 EST
oVirt gerrit 51413 ovirt-3.6 MERGED hosted_engine_page: try/except for he --vm-status 2016-01-06 10:16 EST

  None (edit)
Description wanghui 2015-12-10 01:46:47 EST
Created attachment 1104238 [details]
ovirt-node.log file

Description of problem:
TUI crash when check Hosted Engine VM status while ovirt-ha-agent.service is inactive. It will report error as follows.

An error appeared in the UI: CalledProcessError()
Press ENTER to logout ...
or enter 's' to drop to shell
Version-Release number of selected component (if applicable):

How reproducible:
rhev-hypervisor7-7.2-20151201.2.el7ev
ovirt-node-3.6.0-0.23.20151201git5eed7af.el7ev.noarch
ovirt-node-plugin-hosted-engine-0.3.0-4.el7ev.noarch

Steps to Reproduce:
1. TUI install rhev-hypervisor7-7.2-20151201.2.el7ev
2. Setup Hosted Engine
3. After finished setup Hosted Engine, click <Hosted Engine VM status> to check vm status immediately.
4. Run #hosted-engine --vm-status in Shell
5. Check ovirt-ha-agent.service

Actual results:
1. TUI crash as reports as follows.

An error appeared in the UI: CalledProcessError()
Press ENTER to logout ...
or enter 's' to drop to shell

2. No putput after run#hosted-engine --vm-status
3. After step5, ovirt-ha-agent.service is inactive.

Expected results:
1. It should return vm status without error.

Additional info:
Comment 1 Fabian Deutsch 2015-12-22 08:47:10 EST
Before beta2.2 ovirt-ha-agent and -broker were not coming up reliably.

With the next release however, these services should be up reliably.
Comment 2 Ying Cui 2015-12-24 04:57:57 EST
Tested the latest RHEV-H beta 2.2 build rhev-hypervisor7-7.2-20151221.1.iso, still encountered this issue.

Test version:
# rpm -qa ovirt-node ovirt-hosted-engine-ha vdsm ovirt-hosted-engine-setup
ovirt-node-3.6.0-0.24.20151209gitc0fa931.el7ev.noarch
vdsm-4.17.13-1.el7ev.noarch
ovirt-hosted-engine-setup-1.3.1.3-1.el7ev.noarch
ovirt-hosted-engine-ha-1.3.3.6-1.el7ev.noarch
# cat /etc/redhat-release 
Red Hat Enterprise Virtualization Hypervisor (Beta) release 7.2 (20151221.1.el7ev)

Steps to Reproduce:
1. TUI install rhev-hypervisor7-7.2-20151221.1.iso
2. Setup Hosted Engine
3. During HE on RHEV-H deploying, whatever how make the ovirt-ha-agent.service inactive.
  just one of examples:
  Cancel(Ctrl+C) automatically engine setup on the rhevm-appliance during deploying HE on RHEV-H
4. Back to HE-TUI
5. Click on < Hosted Engine VM status >    

Then TUI crash display ERROR:

An error appeared in the UI: CalledProcessError()
Press ENTER to logout ...
or enter 's' to drop to shell

# cat ovirt-node.log
---
2015-12-24 09:44:52,318       INFO Failed to connect to broker: [Errno 2] No such file or directory
2015-12-24 09:44:52,318       INFO Retrying broker connection in '0' seconds
2015-12-24 09:44:52,319      ERROR Failed to connect to broker, the number of errors has exceeded the limit (1)
2015-12-24 09:44:52,320       INFO Failed to connect to broker: [Errno 2] No such file or directory
2015-12-24 09:44:52,320       INFO Retrying broker connection in '0' seconds
2015-12-24 09:44:52,320      ERROR Failed to connect to broker, the number of errors has exceeded the limit (1)
2015-12-24 09:44:52,321       INFO Failed to connect to broker: [Errno 2] No such file or directory
2015-12-24 09:44:52,321       INFO Retrying broker connection in '0' seconds
2015-12-24 09:44:52,321      ERROR Failed to connect to broker, the number of errors has exceeded the limit (1)
2015-12-24 09:44:52,322       INFO Current page is 'Hosted Engine'
2015-12-24 09:44:55,158       INFO Failed to connect to broker: [Errno 2] No such file or directory
2015-12-24 09:44:55,158       INFO Retrying broker connection in '0' seconds
2015-12-24 09:44:55,158      ERROR Failed to connect to broker, the number of errors has exceeded the limit (1)
2015-12-24 09:44:55,159       INFO Failed to connect to broker: [Errno 2] No such file or directory
2015-12-24 09:44:55,159       INFO Retrying broker connection in '0' seconds
2015-12-24 09:44:55,159      ERROR Failed to connect to broker, the number of errors has exceeded the limit (1)
2015-12-24 09:47:38,244       INFO Current page is 'RHEV-M'
2015-12-24 09:47:38,529       INFO Current page is 'Performance'
2015-12-24 09:47:39,539       INFO Current page is 'RHEV-M'
2015-12-24 09:47:39,821       INFO Current page is 'Performance'
2015-12-24 09:47:42,159       INFO Current page is 'RHEV-M'
2015-12-24 09:47:42,870       INFO Success, id 140031838868816
2015-12-24 09:47:42,898       INFO Success, id 140031838868432
2015-12-24 09:47:42,924       INFO Success, id 140031838868112
2015-12-24 09:47:42,950       INFO Current page is 'Hosted Engine'
2015-12-24 09:47:45,520       INFO Success, id 79797904
2015-12-24 09:47:45,546       INFO Success, id 79797968
2015-12-24 09:47:47,625      ERROR An error appeared in the UI: CalledProcessError()
2015-12-24 09:47:47,625       INFO Exception:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/ovirt/node/app.py", line 304, in run
    self.ui.run()
  File "/usr/lib/python2.7/site-packages/ovirt/node/ui/urwid_builder.py", line 441, in run
    self.__loop.run()
  File "/usr/lib64/python2.7/site-packages/urwid/main_loop.py", line 271, in run
    self.screen.run_wrapper(self._run)
  File "/usr/lib64/python2.7/site-packages/urwid/raw_display.py", line 241, in run_wrapper
    return fn()
  File "/usr/lib64/python2.7/site-packages/urwid/main_loop.py", line 336, in _run
    self.event_loop.run()
  File "/usr/lib64/python2.7/site-packages/urwid/main_loop.py", line 707, in run
    self._loop()
  File "/usr/lib64/python2.7/site-packages/urwid/main_loop.py", line 786, in _loop
    self._watch_files[fd]()
  File "/usr/lib64/python2.7/site-packages/urwid/main_loop.py", line 387, in _update
    self.process_input(keys)
  File "/usr/lib64/python2.7/site-packages/urwid/main_loop.py", line 487, in process_input
    k = self._topmost_widget.keypress(self.screen_size, k)
  File "/usr/lib64/python2.7/site-packages/urwid/container.py", line 1102, in keypress
    return self.body.keypress( (maxcol, remaining), key )
  File "/usr/lib64/python2.7/site-packages/urwid/container.py", line 1559, in keypress
    key = self.focus.keypress(tsize, key)
  File "/usr/lib64/python2.7/site-packages/urwid/container.py", line 2240, in keypress
    key = w.keypress((mc,) + size[1:], key)
  File "/usr/lib64/python2.7/site-packages/urwid/container.py", line 1102, in keypress
    return self.body.keypress( (maxcol, remaining), key )
  File "/usr/lib64/python2.7/site-packages/urwid/decoration.py", line 618, in keypress
    return self._original_widget.keypress(maxvals, key)
  File "/usr/lib64/python2.7/site-packages/urwid/container.py", line 1559, in keypress
    key = self.focus.keypress(tsize, key)
  File "/usr/lib64/python2.7/site-packages/urwid/container.py", line 1559, in keypress
    key = self.focus.keypress(tsize, key)
  File "/usr/lib64/python2.7/site-packages/urwid/decoration.py", line 618, in keypress
    return self._original_widget.keypress(maxvals, key)
  File "/usr/lib64/python2.7/site-packages/urwid/container.py", line 1102, in keypress
    return self.body.keypress( (maxcol, remaining), key )
  File "/usr/lib/python2.7/site-packages/ovirt/node/ui/widgets.py", line 759, in keypress
    return super(TabablePile, self).keypress(size, key)
  File "/usr/lib64/python2.7/site-packages/urwid/container.py", line 1559, in keypress
    key = self.focus.keypress(tsize, key)
  File "/usr/lib64/python2.7/site-packages/urwid/container.py", line 1559, in keypress
    key = self.focus.keypress(tsize, key)
  File "/usr/lib64/python2.7/site-packages/urwid/decoration.py", line 618, in keypress
    return self._original_widget.keypress(maxvals, key)
  File "/usr/lib64/python2.7/site-packages/urwid/wimp.py", line 534, in keypress
    self._emit('click')
  File "/usr/lib64/python2.7/site-packages/urwid/widget.py", line 463, in _emit
    signals.emit_signal(self, name, self, *args)
  File "/usr/lib64/python2.7/site-packages/urwid/signals.py", line 120, in emit
    result |= bool(callback(*args_copy))
  File "/usr/lib/python2.7/site-packages/ovirt/node/ui/widgets.py", line 542, in on_click_cb
    urwid.emit_signal(self, 'click', self)
  File "/usr/lib64/python2.7/site-packages/urwid/signals.py", line 120, in emit
    result |= bool(callback(*args_copy))
  File "/usr/lib/python2.7/site-packages/ovirt/node/ui/urwid_builder.py", line 111, in on_widget_click_cb
    ui_button.on_activate(change)
  File "/usr/lib/python2.7/site-packages/ovirt/node/base.py", line 103, in __call__
    self.emit(userdata)
  File "/usr/lib/python2.7/site-packages/ovirt/node/base.py", line 85, in emit
    if cb(self.target, userdata) is False:
  File "/usr/lib/python2.7/site-packages/ovirt/node/ui/__init__.py", line 202, in __call__
    r = self.callback(userdata)
  File "/usr/lib/python2.7/site-packages/ovirt/node/app.py", line 176, in call_on_ui_save
    self.current_plugin()._on_ui_save()
  File "/usr/lib/python2.7/site-packages/ovirt/node/plugins.py", line 406, in _on_ui_save
    successfull_merge = self.on_merge(effective_changes)
  File "/usr/lib/python2.7/site-packages/ovirt/node/setup/hostedengine/hosted_engine_page.py", line 151, in on_merge
    stderr=utils.process.STDOUT)
  File "/usr/lib/python2.7/site-packages/ovirt/node/utils/process.py", line 159, in check_output
    stdout = subprocess.check_output(*args, **kwargs)
  File "/usr/lib64/python2.7/subprocess.py", line 575, in check_output
    raise CalledProcessError(retcode, cmd, output=output)
CalledProcessError: Command '['hosted-engine', '--vm-status']' returned non-zero exit status 1
Comment 3 Ying Cui 2015-12-24 05:01 EST
Created attachment 1109163 [details]
varlog_rhevh_20151221.1.
Comment 5 wanghui 2016-02-03 02:59:21 EST
Test version:
rhev-hypervisor7-7.2-20160126.0
ovirt-node-3.6.1-5.0.el7ev.noarch
ovirt-node-plugin-hosted-engine-0.3.0-6.el7ev.noarch

Test steps:
1. TUI install rhev-hypervisor7-7.2-20151201.2.el7ev
2. Setup Hosted Engine
3. After finished setup Hosted Engine, click <Hosted Engine VM status> to check vm status immediately.
4. Run #hosted-engine --vm-status in Shell
5. Check ovirt-ha-agent.service

Test result:
1. After steo3, it shows as follows. No TUI crash anymore.
"Failed to collect hosted engine vm status, check ovirt-ha-broker logs.

So this issue is fixed in ovirt-node-plugin-hosted-engine-0.3.0-6.el7ev.noarch.
Comment 7 errata-xmlrpc 2016-03-09 09:44:08 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-0378.html

Note You need to log in before you can comment on or make changes to this bug.