Bug 1292089 - ha-broker failed to start after upgrade of RHEV-H from 3.5 to 3.6
Summary: ha-broker failed to start after upgrade of RHEV-H from 3.5 to 3.6
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: ovirt-hosted-engine-ha
Classification: oVirt
Component: Broker
Version: 1.3.3.2
Hardware: x86_64
OS: Linux
high
urgent
Target Milestone: ovirt-3.6.1
: 1.3.3.4
Assignee: Martin Sivák
QA Contact: Artyom
URL:
Whiteboard: sla
Depends On:
Blocks: RHEV3.6Upgrade
TreeView+ depends on / blocked
 
Reported: 2015-12-16 12:53 UTC by Artyom
Modified: 2016-02-10 19:19 UTC (History)
11 users (show)

Fixed In Version: ovirt-hosted-engine-ha-1.3.3.4
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-01-13 14:40:06 UTC
oVirt Team: SLA
Embargoed:
rule-engine: ovirt-3.6.z+
rule-engine: blocker+
mgoldboi: planning_ack+
dfediuck: devel_ack+
gklein: testing_ack+


Attachments (Terms of Use)
sosreport from RHEV-H 7.2 (5.89 MB, application/x-xz)
2015-12-16 12:53 UTC, Artyom
no flags Details


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 50662 0 None None None 2015-12-17 14:29:05 UTC

Description Artyom 2015-12-16 12:53:33 UTC
Created attachment 1106388 [details]
sosreport from RHEV-H 7.2

Description of problem:
ha-broker failed to start after upgrade of RHEV-H from 3.5 to 3.6 with exception
MainThread::CRITICAL::2015-12-16 09:56:14,885::broker::108::ovirt_hosted_engine_ha.broker.broker.Broker::(run) Could not start ha-broker
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/broker.py", line 102, in run
    self._run_server()
  File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/broker.py", line 155, in _run_server
    self._monitor_instance = self._get_monitor()
  File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/broker.py", line 173, in _get_monitor
    return monitor.Monitor()
  File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/monitor.py", line 33, in __init__
    self._discover_submonitors()
  File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/monitor.py", line 41, in _discover_submonitors
    for filename in (f for f in os.listdir(smdir) if
  File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/broker.py", line 147, in _handle_quit
    self._listener.close_connections()
AttributeError: 'NoneType' object has no attribute 'close_connections'

Version-Release number of selected component (if applicable):
3.5 - Red Hat Enterprise Virtualization Hypervisor release 7.1 (20151015.0.el7ev)
=================================
vdsm-4.16.27-1.el7ev.x86_64
ovirt-hosted-engine-setup-1.2.6.1-1.el7ev.noarch
ovirt-hosted-engine-ha-1.2.7.2-1.el7ev.noarch

3.6 - Red Hat Enterprise Virtualization Hypervisor (Beta) release 7.2 (20151210.1.el7ev)
=================================
vdsm-4.17.13-1.el7ev.noarch
ovirt-hosted-engine-setup-1.3.1.2-1.el7ev.noarch
ovirt-hosted-engine-ha-1.3.3.3-1.el7ev.noarch

How reproducible:
Always

Steps to Reproduce:
1. Deploy 3.5 HE environment on one host
2. Put host to global maintenance and stop engine vm(hosted-engine --set-maintenance --mode=global && hosted-engine --vm-poweroff)
3. Reboot host and upgrade it to 3.6
4. Check hosted-engine services

Actual results:
ovirt-ha-broker service down

Expected results:
ovirt-ha-broker service up

Additional info:

Comment 1 Martin Sivák 2015-12-17 14:26:52 UTC
Can you please retest with the latest version? (1.3.3.4-1) We committed a fix recently that might have fixed this: https://gerrit.ovirt.org/#/c/50662/

Comment 2 Red Hat Bugzilla Rules Engine 2015-12-17 14:34:57 UTC
Fixed bug tickets must have target milestone set prior to fixing them. Please set the correct milestone and move the bugs back to the previous status after this is corrected.

Comment 3 Artyom 2015-12-17 15:54:38 UTC
I checked it with patch above and looks like error not appear anymore and ovirt-ha-broker succeed to start after upgrade. But need to check it on RHEV-H with new package to be sure if all works correct.

Comment 5 Red Hat Bugzilla Rules Engine 2015-12-21 07:17:04 UTC
Bug tickets that are moved to testing must have target release set to make sure tester knows what to test. Please set the correct target release before moving to ON_QA.

Comment 6 Artyom 2015-12-22 15:54:47 UTC
Verified
Upgrade from 3.5 - Red Hat Enterprise Virtualization Hypervisor release 7.1 (20151015.0.el7ev)
==============================================================================================
ovirt-hosted-engine-ha-1.2.7.2-1.el7ev.noarch
ovirt-hosted-engine-setup-1.2.6.1-1.el7ev.noarch
vdsm-4.16.27-1.el7ev.x86_64

to 3.6 - Red Hat Enterprise Virtualization Hypervisor (Beta) release 7.2 (20151221.1.el7ev)
==============================================================================================
ovirt-hosted-engine-setup-1.3.1.3-1.el7ev.noarch
ovirt-hosted-engine-ha-1.3.3.6-1.el7ev.noarch
vdsm-4.17.13-1.el7ev.noarch

1) Install RHEV-H 3.5
2) Deploy hosted-engine on one host with NFS storage
3) Enable global maintenance(on host hosted-engine --set-maintenance --mode=global)
4) Upgrade engine to 3.6
5) Power off engine vm(hosted-engine --vm-poweroff)
6) Upgrade host to RHEV-H 3.6 via usb-key

Upgrade succeed and after host upgrade ovirt-he-agent and ovirt-ha-broker up.

Comment 7 Sandro Bonazzola 2016-01-13 14:40:06 UTC
oVirt 3.6.1 has been released, closing current release


Note You need to log in before you can comment on or make changes to this bug.