Bug 1331514

Summary: agent.log log file is flooded with error messages as - "Unable to get vm.conf from OVF_STORE, falling back to initial vm.conf"
Product: [oVirt] ovirt-hosted-engine-ha Reporter: SATHEESARAN <sasundar>
Component: AgentAssignee: Simone Tiraboschi <stirabos>
Status: CLOSED NOTABUG QA Contact: SATHEESARAN <sasundar>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 1.3.5.3CC: bugs, sabose, stirabos
Target Milestone: ovirt-4.0.0-rcFlags: sabose: ovirt-4.0.0?
sasundar: planning_ack?
sasundar: devel_ack?
sasundar: testing_ack?
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
RHEV RHGS HCI RHEL 7.2 RHEV 3.6.5
Last Closed: 2016-05-20 12:19:40 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Integration RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1277939    
Attachments:
Description Flags
agent.log
none
sosreport from the node
none
broker.log
none
vdsm.log
none
supervdsm.log none

Description SATHEESARAN 2016-04-28 17:25:39 UTC 
Description of problem:
-----------------------
the log file agent.log was flooded with error messages - "Unable to get vm.conf from OVF_STORE, falling back to initial vm.conf"

Version-Release number of selected component (if applicable):
-------------------------------------------------------------
RHEV 3.6.5.3-1
ovirt-host-deploy-1.4.1-1.el7ev.noarch
ovirt-hosted-engine-ha-1.3.5.3-1.el7ev.noarch
ovirt-hosted-engine-setup-1.3.5.0-1.el7ev.noarch

How reproducible:
-----------------
Always

Steps to Reproduce:
-------------------
1. On RHEL 7.2 perform deploy hosted-engine ( hosted-engine --deploy )
2. Use glusterfs volume ( replica 3 + sharded volume ) for the hosted-engine backend
3. After hosted-engine deployment is successful, check for agent logs ( /var/log/ovirt-hosted-engine/ha/agent.log )

Actual results:
---------------
There are many error messages

Expected results:
-----------------
There shouldn't be any error messages


Additional info:
-----------------
<snip>
MainThread::ERROR::2016-04-28 22:47:10,958::config::234::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_local_conf_file) Unable to get vm.conf from OVF_STORE, falling back to initial vm.conf
MainThread::ERROR::2016-04-28 22:47:50,964::config::234::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_local_conf_file) Unable to get vm.conf from OVF_STORE, falling back to initial vm.conf
MainThread::ERROR::2016-04-28 22:48:26,908::config::234::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_local_conf_file) Unable to get vm.conf from OVF_STORE, falling back to initial vm.conf
MainThread::ERROR::2016-04-28 22:49:01,852::config::234::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_local_conf_file) Unable to get vm.conf from OVF_STORE, falling back to initial vm.conf
</snip>

[root@ ~]# grep ERROR /var/log/ovirt-hosted-engine-ha/agent.log  | wc -l
333
[root@ ~]# grep ERROR /var/log/ovirt-hosted-engine-ha/agent.log  | wc -l
342
Comment 1 SATHEESARAN 2016-04-28 17:26:35 UTC 
Created attachment 1152003 [details]
agent.log
Comment 2 SATHEESARAN 2016-04-29 08:28:49 UTC 
Created attachment 1152161 [details]
sosreport from the node
Comment 3 SATHEESARAN 2016-04-29 08:29:40 UTC 
Created attachment 1152162 [details]
broker.log
Comment 4 SATHEESARAN 2016-04-29 08:43:04 UTC 
Created attachment 1152168 [details]
vdsm.log
Comment 5 SATHEESARAN 2016-04-29 08:43:54 UTC 
Created attachment 1152169 [details]
supervdsm.log
Comment 6 SATHEESARAN 2016-04-29 09:12:31 UTC 
I have also found various error messages in vdsm.log[1] and and also in broker.log[2]. I have raise separate bugs for those

[1] - https://bugzilla.redhat.com/show_bug.cgi?id=1331503
[2] - https://bugzilla.redhat.com/show_bug.cgi?id=1331526
Comment 7 SATHEESARAN 2016-04-29 09:14:38 UTC 
There is no functional loss, and so lowering the severity
Comment 8 Yaniv Lavi 2016-05-09 10:53:24 UTC 
Moving to first RC, since things should not be targeted to second one at this point.
Comment 9 Simone Tiraboschi 2016-05-20 12:19:40 UTC 
The engine creates the OVF_STORE volume, which contains the engine VM description, only when the datacenter goes up but to do the the user as to manually create the first storage domain for regular VMs.

ovirt-ha-agent will try to periodically parse the engine VM configuration from the OVF_STORE volume, if it's not there, ovirt-ha-agent can fail back to the initial (deployment time) vm.conf which can be out of sync and so it's worth to signal it.

Creating the first regular storage domain should make the datacenter up, after one our the engine will create the OVF_STORE, ovirt-ha-agent will start parsing it and this message will disappear.