Bug 1630346

Summary: Node zero hosted-engine deployment fails with RHVH based on RHEL 7.6
Product: Red Hat Enterprise Virtualization Manager Reporter: SATHEESARAN <sasundar>
Component: ovirt-hosted-engine-setupAssignee: Simone Tiraboschi <stirabos>
Status: CLOSED DUPLICATE QA Contact: meital avital <mavital>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 4.2.7CC: lsurette
Target Milestone: ovirt-4.2.7Keywords: ZStream
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-09-20 06:53:25 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Integration RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
engine.log
none
localvm bootstrap log
none
sosreport from the physical host none

Description SATHEESARAN 2018-09-18 12:38:36 UTC
Description of problem:
-----------------------
This testing is done for RHHI 2.0 validation with RHVH based on RHEL 7.6

While starting with the node-zero deployment,  the ansible deployment failed and there were exceptions found in the engine.log while configuring management network

Version-Release number of selected component (if applicable):
--------------------------------------------------------------
RHVH ISO - RHVH-4.2-20180828.0-RHVH-x86_64-dvd1.iso
redhat-virtualization-host - redhat-virtualization-host-image-update-4.2-20180917.0.el7_6.noarch.rpm

How reproducible:
------------------
Always

Steps to Reproduce:
-------------------

1. Create gluster volume for HE VM
2. Start with Node zero deployment from cockpit with following configurations:
     Storage: gluster replica 3 volume
     HE with DHCP
     Host is having 2 networks, one for ovirtmgmt and other for gluster data
     
Actual results:
---------------
The local HE VM was running but failed to configure management network

Expected results:
------------------
HE deployment should be successful

Comment 1 SATHEESARAN 2018-09-18 12:43:56 UTC
1. localvm bootstrap log
---------------------------
Here is the error from the localvm bootstrap logs
<snip>
2018-09-18 17:47:03,997+0530 INFO ansible ok {'status': 'OK', 'ansible_task': u'Wait for the host to be up', 'ansible_host': u'localhost', 'ansible_playbook': u'/usr/share/ovirt-hosted-engine-setup/ansible/bootstrap_local_vm.yml', 'ansible_type': 'task'}
2018-09-18 17:47:03,997+0530 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7fc293f16750> kwargs 
2018-09-18 17:47:04,724+0530 INFO ansible task start {'status': 'OK', 'ansible_task': u'debug', 'ansible_playbook': u'/usr/share/ovirt-hosted-engine-setup/ansible/bootstrap_local_vm.yml', 'ansible_type': 'task'}
2018-09-18 17:47:04,724+0530 DEBUG ansible on_any args TASK: debug kwargs is_conditional:False 
2018-09-18 17:47:05,413+0530 INFO ansible ok {'status': 'OK', 'ansible_task': u'', 'ansible_host': u'localhost', 'ansible_playbook': u'/usr/share/ovirt-hosted-engine-setup/ansible/bootstrap_local_vm.yml', 'ansible_type': 'task'}
2018-09-18 17:47:05,414+0530 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7fc293ef9050> kwargs 
2018-09-18 17:47:06,136+0530 INFO ansible task start {'status': 'OK', 'ansible_task': u'Check host status', 'ansible_playbook': u'/usr/share/ovirt-hosted-engine-setup/ansible/bootstrap_local_vm.yml', 'ansible_type': 'task'}
2018-09-18 17:47:06,137+0530 DEBUG ansible on_any args TASK: Check host status kwargs is_conditional:False 
2018-09-18 17:47:06,832+0530 DEBUG var changed: host "localhost" var "ansible_failed_result" type "<type 'dict'>" value: "{
    "_ansible_no_log": false, 
    "changed": false, 
    "failed": true, 
    "msg": "The host has been set in non_operational status, please check engine logs, fix accordingly and re-deploy.\n"
}"
</snip>


2. vdsm.log
------------
There are some errors in vdsm.log

<snip>
2018-09-18 17:41:25,223+0530 INFO  (vm/1021e453) [virt.vm] (vmId='1021e453-2656-4186-aaa1-453fdde27719') Starting connection (guestagent:248)
2018-09-18 17:41:25,224+0530 ERROR (vm/1021e453) [virt.vm] (vmId='1021e453-2656-4186-aaa1-453fdde27719') Failed to connect to guest agent channel (vm:2514)
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 2512, in _vmDependentInit
    self.guestAgent.start()
  File "/usr/lib/python2.7/site-packages/vdsm/virt/guestagent.py", line 249, in start
    self._prepare_socket()
  File "/usr/lib/python2.7/site-packages/vdsm/virt/guestagent.py", line 291, in _prepare_socket
    supervdsm.getProxy().prepareVmChannel(self._socketName)
  File "/usr/lib/python2.7/site-packages/vdsm/common/supervdsm.py", line 55, in __call__
    return callMethod()
  File "/usr/lib/python2.7/site-packages/vdsm/common/supervdsm.py", line 53, in <lambda>
    **kwargs)
  File "<string>", line 2, in prepareVmChannel
  File "/usr/lib64/python2.7/multiprocessing/managers.py", line 773, in _callmethod
    raise convert_to_error(kind, result)
OSError: [Errno 2] No such file or directory: '/var/lib/libvirt/qemu/channels/1021e453-2656-4186-aaa1-453fdde27719.com.redhat.rhevm.vdsm'

</snip>

Comment 2 SATHEESARAN 2018-09-18 12:48:08 UTC
Created attachment 1484349 [details]
engine.log

Comment 3 SATHEESARAN 2018-09-18 12:49:17 UTC
Created attachment 1484350 [details]
localvm bootstrap log

Comment 4 SATHEESARAN 2018-09-18 12:50:24 UTC
Created attachment 1484358 [details]
sosreport from the physical host

Comment 5 SATHEESARAN 2018-09-18 12:56:14 UTC
The additional information is that the testing used the rhvm-appliance - rhvm-appliance-4.2-20180828.0.el7.noarch.rpm - which is based on RHEL 7.5, should that caused this issue by any chance ? I hope not though !!

Comment 6 Simone Tiraboschi 2018-09-18 13:15:11 UTC
I think it's just a duplicate of 1626960

Comment 7 Sandro Bonazzola 2018-09-20 06:53:25 UTC
Closing as duplicate of bug #1626960

*** This bug has been marked as a duplicate of bug 1626960 ***