Bug 1227735 - Failed to deploy additional host due to iptables configured to accept only the SSH in/out traffic
Summary: Failed to deploy additional host due to iptables configured to accept only th...
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-hosted-engine-setup
Version: 3.6.0
Hardware: x86_64
OS: Linux
unspecified
urgent
Target Milestone: ---
: ---
Assignee: Sandro Bonazzola
QA Contact: Nikolai Sednev
URL:
Whiteboard: integration
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-06-03 12:02 UTC by Nikolai Sednev
Modified: 2015-06-22 06:09 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-06-04 17:13:10 UTC
oVirt Team: ---
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
additional host's logs (6.08 KB, application/x-gzip)
2015-06-03 12:02 UTC, Nikolai Sednev
no flags Details
sosreport-blue-vdsc.qa.lab.tlv.redhat.com-20150531210307.tar.xz (5.66 MB, application/x-xz)
2015-06-03 12:03 UTC, Nikolai Sednev
no flags Details

Description Nikolai Sednev 2015-06-03 12:02:56 UTC
Created attachment 1034263 [details]
additional host's logs

Description of problem:
Can't add additional host to HE environment if only SSH in/out iptables configured before the deployment of HE started at additional host.

[root@blue-vdsc ~]# iptables -A INPUT -p tcp --dport 22 -j ACCEPT
[root@blue-vdsc ~]# iptables -A OUTPUT -p tcp --sport 22 -j ACCEPT
[root@blue-vdsc ~]# iptables -A INPUT -j DROP                     
[root@blue-vdsc ~]# iptables -A OUTPUT -j DROP                    
[root@blue-vdsc ~]# iptables -L
Chain INPUT (policy ACCEPT)    
target     prot opt source               destination
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:ssh
DROP       all  --  anywhere             anywhere

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
ACCEPT     tcp  --  anywhere             anywhere             tcp spt:ssh
DROP       all  --  anywhere             anywhere
[root@blue-vdsc ~]# hosted-engine --deploy
[ INFO  ] Stage: Initializing
[ INFO  ] Generating a temporary VNC password.
[ INFO  ] Stage: Environment setup
          Continuing will configure this host for serving as hypervisor and create a VM where you have to install oVirt Engine afterwards.
          Are you sure you want to continue? (Yes, No)[Yes]:
          Configuration files: []
          Log file: /var/log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-20150531204600-uxf6cj.log
          Version: otopi-1.3.2 (otopi-1.3.2-1.el7ev)
          It has been detected that this program is executed through an SSH connection without using screen.
          Continuing with the installation may lead to broken installation if the network connection fails.
          It is highly recommended to abort the installation and run it inside a screen session using command "screen".
          Do you want to continue anyway? (Yes, No)[No]: yes
[ INFO  ] Hardware supports virtualization
[ INFO  ] Stage: Environment packages setup
[ INFO  ] Stage: Programs detection
[ INFO  ] Stage: Environment setup
[ INFO  ] Waiting for VDSM hardware info
[ INFO  ] Waiting for VDSM hardware info
[ INFO  ] Waiting for VDSM hardware info
[ INFO  ] Waiting for VDSM hardware info
[ INFO  ] Waiting for VDSM hardware info
[ INFO  ] Waiting for VDSM hardware info
[ INFO  ] Waiting for VDSM hardware info
[ INFO  ] Waiting for VDSM hardware info
[ INFO  ] Waiting for VDSM hardware info
[ INFO  ] Waiting for VDSM hardware info
[ INFO  ] Generating libvirt-spice certificates
[ ERROR ] Failed to execute stage 'Environment setup': timed out
[ INFO  ] Stage: Clean up
[ INFO  ] Generating answer file '/var/lib/ovirt-hosted-engine-setup/answers/answers-20150531205814.conf'
[ INFO  ] Stage: Pre-termination
[ INFO  ] Stage: Termination
[root@blue-vdsc ~]#

Version-Release number of selected component (if applicable):
libvirt-python-1.2.8-7.el7_1.1.x86_64
libvirt-daemon-driver-nodedev-1.2.8-16.el7_1.3.x86_64
mom-0.4.1-5.el7ev.noarch
vdsm-4.16.18-1.el7ev.x86_64
sanlock-3.2.2-2.el7.x86_64
sanlock-lib-3.2.2-2.el7.x86_64
sanlock-python-3.2.2-2.el7.x86_64
ovirt-host-deploy-1.3.0-2.el7ev.noarch
libvirt-client-1.2.8-16.el7_1.3.x86_64
libvirt-daemon-driver-nwfilter-1.2.8-16.el7_1.3.x86_64
libvirt-daemon-config-nwfilter-1.2.8-16.el7_1.3.x86_64
libvirt-daemon-driver-interface-1.2.8-16.el7_1.3.x86_64
libvirt-daemon-driver-secret-1.2.8-16.el7_1.3.x86_64
libvirt-daemon-driver-qemu-1.2.8-16.el7_1.3.x86_64
libvirt-daemon-driver-storage-1.2.8-16.el7_1.3.x86_64
ovirt-hosted-engine-ha-1.2.6-2.el7ev.noarch
qemu-kvm-rhev-2.1.2-23.el7_1.3.x86_64
libvirt-daemon-1.2.8-16.el7_1.3.x86_64
libvirt-lock-sanlock-1.2.8-16.el7_1.3.x86_64
libvirt-daemon-driver-network-1.2.8-16.el7_1.3.x86_64
libvirt-daemon-kvm-1.2.8-16.el7_1.3.x86_64
ovirt-hosted-engine-setup-1.2.4-2.el7ev.noarch
qemu-kvm-common-rhev-2.1.2-23.el7_1.3.x86_64

How reproducible:
100%

Steps to Reproduce:
1.Reprovision 2 hosts to RHEL7.1.
2.Deploy HE on first host with default iptables (select yet to question for configuring iptbles on host during deployment on first host) RHEL6.6 as engine's OS.
3.Configure iptables at additional host (second host) as follows:
[root@blue-vdsc ~]# iptables -A INPUT -p tcp --dport 22 -j ACCEPT
[root@blue-vdsc ~]# iptables -A OUTPUT -p tcp --sport 22 -j ACCEPT
[root@blue-vdsc ~]# iptables -A INPUT -j DROP                     
[root@blue-vdsc ~]# iptables -A OUTPUT -j DROP                    
[root@blue-vdsc ~]# iptables -L
Chain INPUT (policy ACCEPT)    
target     prot opt source               destination
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:ssh
DROP       all  --  anywhere             anywhere

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
ACCEPT     tcp  --  anywhere             anywhere             tcp spt:ssh
DROP       all  --  anywhere             anywhere

4.Add second host to HE e.g. hosted-engine --deploy.

Actual results:
Deployment fails with [ ERROR ] Failed to execute stage 'Environment setup': timed out.

Expected results:
Deployment should succeed.

Additional info:
Logs from second host attached.

Comment 1 Nikolai Sednev 2015-06-03 12:03:54 UTC
Created attachment 1034264 [details]
sosreport-blue-vdsc.qa.lab.tlv.redhat.com-20150531210307.tar.xz

Comment 2 Sandro Bonazzola 2015-06-04 17:13:10 UTC
This is valid for every kind of setup.
If iptables is set for closing all the outgoing connections the setup tool can't connect to vdsm and any other kind of connection can't be established.

The setup tool is supposed to be transactional (in the configuration part at least) so we can't alter iptables before starting the setup process.

I was tempted to move this to vdsm since if anything should take care of checking vdsm connectivity it should be vdsm-tool configure called when initializing vdsm.
But at the end, I don't think that also vdsm-tool shouldn't be responsible of changing such weird iptables configuration.

Closing as wontfix for now. Feel free to repoen if there is any valid reason for handling this case.

Comment 3 Nikolai Sednev 2015-06-21 15:22:17 UTC
(In reply to Sandro Bonazzola from comment #2)
> This is valid for every kind of setup.
> If iptables is set for closing all the outgoing connections the setup tool
> can't connect to vdsm and any other kind of connection can't be established.
> 
> The setup tool is supposed to be transactional (in the configuration part at
> least) so we can't alter iptables before starting the setup process.
> 
> I was tempted to move this to vdsm since if anything should take care of
> checking vdsm connectivity it should be vdsm-tool configure called when
> initializing vdsm.
> But at the end, I don't think that also vdsm-tool shouldn't be responsible
> of changing such weird iptables configuration.
> 
> Closing as wontfix for now. Feel free to repoen if there is any valid reason
> for handling this case.

Hi Sandro,
It wasn't closed on all port, ssh was opened, as a matter of fact, I was still connected to the host via ssh.
Does connection to vdsm established not over ssh? If connection established not over ssh, then it's not a bug, otherwise it is a bug.
Can you describe this in more details?

Comment 4 Sandro Bonazzola 2015-06-22 06:09:06 UTC
(In reply to Nikolai Sednev from comment #3)
> Hi Sandro,
> It wasn't closed on all port, ssh was opened, as a matter of fact, I was
> still connected to the host via ssh.
> Does connection to vdsm established not over ssh? If connection established
> not over ssh, then it's not a bug, otherwise it is a bug.
> Can you describe this in more details?

I'm not going to give a networking lesson in bugzilla. 
You closed all outbound connections except those with source port 22 and inbound connections with destination port 22, this means you can just accept 1 ssh connection and you can't have any other connection on the host.
For further references: https://www.google.it/search?q=networking+tutorials


Note You need to log in before you can comment on or make changes to this bug.