Bug 1221221

Summary: hosted-engine fails deploying the second host cause it keep the firewall configuration untouched
Product: [Retired] oVirt Reporter: Simone Tiraboschi <stirabos>
Component: ovirt-hosted-engine-setupAssignee: Sandro Bonazzola <sbonazzo>
Status: CLOSED NOTABUG QA Contact: meital avital <mavital>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 3.5CC: alonbl, bazulay, dougsland, ecohen, gklein, lsurette, rbalakri, yeylon
Target Milestone: ---Keywords: Reopened
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-05-13 14:37:40 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Attachments:
Description Flags
engine logs
none
host-deploy logs
none
hosted-engine logs none

Description Simone Tiraboschi 2015-05-13 14:09:28 UTC
Description of problem:
I'm trying to deploy oVirt 3.5.2 on centos 7.1.
It fails adding the second host.

[ INFO  ] Stage: Closing up
[ INFO  ] Waiting for the host to become operational in the engine. This may take several minutes...
[ INFO  ] Still waiting for VDSM host to become operational...
[ INFO  ] Still waiting for VDSM host to become operational...
[ ERROR ] The VDSM host was found in a failed state. Please check engine and bootstrap installation logs.
[ ERROR ] Unable to add hosted_engine_2 to the manager

I checked host-deploy logs on the engine VM.
Host deploy correctly completed but the engine is not able to contact the host.
2015-05-13 15:07:31,924 INFO  [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation c71he352: Starting vdsm
2015-05-13 15:07:31,930 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 65291b0a, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Starting vdsm.
2015-05-13 15:07:33,705 INFO  [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation c71he352: Stage: Pre-termination
2015-05-13 15:07:33,710 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 65291b0a, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Stage: Pre-termination.
2015-05-13 15:07:33,740 INFO  [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation c71he352: Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20150513150733-c71he352-65291b0a.log'
2015-05-13 15:07:33,745 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 65291b0a, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20150513150733-c71he352-65291b0a.log'.
2015-05-13 15:07:33,944 INFO  [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation c71he352: Stage: Termination
2015-05-13 15:07:33,958 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 65291b0a, Call Stack: null, Custom Event ID: -1, Message: Installing Host hosted_engine_2. Stage: Termination.
2015-05-13 15:07:34,108 INFO  [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to c71he352/192.168.1.130
2015-05-13 15:07:34,116 WARN  [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed
2015-05-13 15:07:34,118 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (org.ovirt.thread.pool-8-thread-24) Exception during connection
2015-05-13 15:07:34,120 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.TimeBoundPollVDSCommand] (org.ovirt.thread.pool-8-thread-8) [65291b0a] java.util.concurrent.ExecutionException: org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcRunTimeException: Connection issues during send request

On the host firewalld is active and iptables no.
iptables-services is there.

[root@c71he352 ~]# systemctl status firewalld; systemctl status iptables
firewalld.service - firewalld - dynamic firewall daemon
   Loaded: loaded (/usr/lib/systemd/system/firewalld.service; enabled)
   Active: active (running) since Wed 2015-05-13 11:20:17 CEST; 4h 48min ago
 Main PID: 735 (firewalld)
   CGroup: /system.slice/firewalld.service
           └─735 /usr/bin/python -Es /usr/sbin/firewalld --nofork --nopid

May 13 11:20:17 c71_he35_2 systemd[1]: Started firewalld - dynamic firewall daemon.
iptables.service - IPv4 firewall with iptables
   Loaded: loaded (/usr/lib/systemd/system/iptables.service; disabled)
   Active: inactive (dead)



Version-Release number of selected component (if applicable):
On the engine VM, ovirt-host-deploy.noarch        1.3.1-1.fc20        @ovirt-3.5

How reproducible:
Seen once

Steps to Reproduce:
1. Deploy hosted-engine on centos 7.1
2.
3.

Actual results:
It fails adding the second host cause is not reachable due to firewalld 

Expected results:
It succeeds

Additional info:

Comment 1 Simone Tiraboschi 2015-05-13 14:12:13 UTC
Created attachment 1025073 [details]
engine logs

Comment 2 Simone Tiraboschi 2015-05-13 14:12:41 UTC
Created attachment 1025074 [details]
host-deploy logs

Comment 3 Simone Tiraboschi 2015-05-13 14:13:10 UTC
Created attachment 1025075 [details]
hosted-engine logs

Comment 4 Alon Bar-Lev 2015-05-13 14:15:53 UTC
you do not enable iptables management:

2015-05-13 15:07:33 DEBUG otopi.context context.dumpEnvironment:500 ENV NETWORK/iptablesEnable=bool:'False'

so iptables/firewalld are untouched.

Comment 5 Simone Tiraboschi 2015-05-13 14:30:26 UTC
It's not on host-deploy but it's still a bug

Comment 6 Alon Bar-Lev 2015-05-13 14:32:10 UTC
(In reply to Simone Tiraboschi from comment #5)
> It's not on host-deploy but it's still a bug

you should open a new one, please revert.

you should not hijack bugs.

Comment 7 Simone Tiraboschi 2015-05-13 14:37:40 UTC
Ok, sorry