Bug 1252801

Summary: /var/log/messages is spammed with hosted engine errors on RHEV-H 3.5.4
Product: Red Hat Enterprise Virtualization Manager Reporter: rhev-integ
Component: ovirt-hosted-engine-haAssignee: Martin Sivák <msivak>
Status: CLOSED ERRATA QA Contact: Artyom <alukiano>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: 3.5.4CC: alukiano, amureini, bcholler, bmcclain, cshao, danken, ecohen, gklein, huiwa, istein, juwu, leiwang, lsurette, mburman, mgoldboi, msivak, pstehlik, pzhukov, rgolan, sapandit, sbonazzo, ycui, yeylon
Target Milestone: ovirt-3.5.6Keywords: Regression, Triaged, ZStream
Target Release: 3.5.6Flags: juwu: needinfo-
mgoldboi: Triaged+
Hardware: x86_64   
OS: Linux   
Whiteboard: sla
Fixed In Version: ovirt-hosted-engine-ha-1.2.6.1 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 1248539 Environment:
Last Closed: 2015-10-26 21:39:32 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: SLA RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1248539    
Bug Blocks:    

Comment 4 Yaniv Lavi 2015-08-31 14:46:32 UTC
Why was this moved to 3.5.4?

Comment 5 Martin Sivák 2015-08-31 15:00:11 UTC
Because it is present there as part of the RHEVH fix list.

Comment 6 Artyom 2015-09-02 10:17:01 UTC
Still can see this kind of errors under vdsm and message logs(appear after I add host to engine):

/var/log/messages.log
Sep  2 08:55:24 cyan-vdsf vdsm vds ERROR failed to retrieve Hosted Engine HA info#012Traceback (most recent call last):#012  File "/usr/share/vdsm/API.py", line 1703, in _getHaInfo#012  File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/client/client.py", line 100, in get_all_stats#012  File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/client/client.py", line 173, in _configure_broker_conn#012  File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/env/config.py", line 129, in get#012Exception: Configuration value not found: file=/etc/ovirt-hosted-engine/hosted-engine.conf, key=sdUUID

/var/log/vdsm/vdsm.log
Thread-404::ERROR::2015-09-02 08:56:10,184::API::1723::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
Traceback (most recent call last):
  File "/usr/share/vdsm/API.py", line 1703, in _getHaInfo
  File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/client/client.py", line 100, in get_all_stats
  File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/client/client.py", line 173, in _configure_broker_conn
  File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/env/config.py", line 129, in get
Exception: Configuration value not found: file=/etc/ovirt-hosted-engine/hosted-engine.conf, key=sdUUID

HOST:
Red Hat Enterprise Virtualization Hypervisor release 6.7 (20150828.0.el6ev)
vdsm-4.16.26-1.el6ev.x86_64
ovirt-hosted-engine-ha-1.2.6-3.el6ev.noarch

I see that one of patches placed under vdsm, so maybe we miss correct vdsm package

Comment 7 Nikolai Sednev 2015-09-02 10:26:18 UTC
Works for me on these components:
ovirt-node-branding-rhev-3.2.3-20.el6.noarch
vdsm-4.16.26-1.el6ev.x86_64
ovirt-node-plugin-hosted-engine-0.2.0-18.0.el6ev.noarch
sanlock-2.8-2.el6_5.x86_64
libvirt-client-0.10.2-54.el6.x86_64
mom-0.4.1-5.el6ev.noarch
ovirt-node-plugin-rhn-3.2.3-20.el6.noarch
ovirt-node-3.2.3-20.el6.noarch
qemu-kvm-rhev-0.12.1.2-2.479.el6.x86_64
ovirt-hosted-engine-setup-1.2.5.3-1.el6ev.noarch
ovirt-host-deploy-offline-1.3.0-3.el6ev.x86_64
ovirt-node-plugin-vdsm-0.2.0-26.el6ev.noarch
ovirt-node-plugin-cim-3.2.3-20.el6.noarch
ovirt-node-selinux-3.2.3-20.el6.noarch
ovirt-hosted-engine-ha-1.2.6-3.el6ev.noarch
ovirt-node-plugin-snmp-3.2.3-20.el6.noarch
ovirt-host-deploy-1.3.2-1.el6ev.noarch


Engine used:
3.5.4.2-1.3.el6ev

Comment 8 Nikolai Sednev 2015-09-02 10:29:17 UTC
My bad, rechecked the vdsm log on my host and found:
Thread-296::ERROR::2015-09-02 10:27:18,061::API::1723::vds::(_getHaInfo) failed to retrieve Hosted Engine HA info
Traceback (most recent call last):
  File "/usr/share/vdsm/API.py", line 1703, in _getHaInfo
  File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/client/client.py", line 100, in get_all_stats
  File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/client/client.py", line 173, in _configure_broker_conn
  File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/env/config.py", line 129, in get
Exception: Configuration value not found: file=/etc/ovirt-hosted-engine/hosted-engine.conf, key=sdUUID

Comment 9 Martin Sivák 2015-09-02 12:12:56 UTC
Yep, the error in comment #6 happens because the vdsm part is not in. It did not seem necessary at first and nobody wanted to respin vdsm with just this patch.

We will need a VDSM clone of this bug with all the acks to be able to get it there.

Comment 10 Ilanit Stein 2015-09-02 15:10:05 UTC
The problem seem to be RHEV-H specific.
It was not seen on rhevm 3.5.4 rhel7.1/rhel6.7 (tested by alukiano)

Comment 11 Sandro Bonazzola 2015-09-02 15:12:01 UTC
(In reply to Martin Sivák from comment #9)
> Yep, the error in comment #6 happens because the vdsm part is not in. It did
> not seem necessary at first and nobody wanted to respin vdsm with just this
> patch.
> 
> We will need a VDSM clone of this bug with all the acks to be able to get it
> there.

So I think this bug should be on modified and the new bug on post.

Comment 13 Michael Burman 2015-09-03 05:46:50 UTC
I still see this bug on Red Hat Enterprise Virtualization Hypervisor release 6.7 (20150826.0.el6ev)
vdsm-4.16.26-1.el6ev.x86_64
ovirt-node-3.2.3-20.el6.noarch

- The log still spammed with this errors.

Comment 14 Yaniv Lavi 2015-09-08 10:42:54 UTC
*** Bug 1258853 has been marked as a duplicate of this bug. ***

Comment 18 Artyom 2015-10-19 16:15:45 UTC
Verified on:
ovirt-hosted-engine-setup-1.2.6.1-1.el6ev.noarch
ovirt-hosted-engine-ha-1.2.7.2-1.el6ev.noarch
vdsm-4.16.27-1.el6ev.x86_64

Red Hat Enterprise Virtualization Hypervisor release 6.7 (20151015.1.el6ev)

1) Install RHEV-H (20151015.1.el6ev) on host
2) Reboot host
3) Add host to engine

No any flood error messages in vdsm.log or messages.log

Comment 19 Julie 2015-10-21 22:20:44 UTC
If this bug requires doc text for errata release, please provide draft text in the doc text field in the following format:

Cause:
Consequence:
Fix:
Result:

The documentation team will review, edit, and approve the text.

If this bug does not require doc text, please set the 'requires_doc_text' flag to -.

Comment 22 errata-xmlrpc 2015-10-26 21:39:32 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-1939.html

Comment 24 Artyom 2016-04-05 11:48:16 UTC
Just log issue, so do not think that it is relevant to add it to QE coverage.