Bug 998938

Summary: GuideMe AddHost does not pass OverrideFirewall
Product: Red Hat Enterprise Virtualization Manager Reporter: Rejy M Cyriac <rcyriac>
Component: ovirt-engine-webadmin-portalAssignee: Alexander Wels <awels>
Status: CLOSED CURRENTRELEASE QA Contact: Tareq Alayan <talayan>
Severity: high Docs Contact:
Priority: high    
Version: 3.2.0CC: aberezin, acathrow, alonbl, awels, ebenahar, ecohen, iheim, pstehlik, rcyriac, Rhev-m-bugs, sasundar, sbonazzo, yeylon
Target Milestone: ---Keywords: Reopened, ZStream
Target Release: 3.4.0   
Hardware: All   
OS: Linux   
Whiteboard: ux
Fixed In Version: av1 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1067417 (view as bug list) Environment:
Last Closed: 2014-06-12 14:04:44 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1067417, 1069847, 1078909, 1142926    
Attachments:
Description Flags
installation logs of RHS server added through 'Configure Host' Dialogue Box
none
installation logs of RHS server added through 'New Host' Dialogue Box
none
rhevm-log-collector logs from RHEVM
none
screenshots none

Description Rejy M Cyriac 2013-08-20 11:35:47 UTC
Description of problem:
Immediately following the creation of a Gluster Enabled Cluster, if a Red Hat Storage (RHS) server is added through the 'Configure Host' option of the 'New Cluster - Guide Me' Dialogue Box, the iptables firewall rules are not being added during bootstrapping.

If the RHS server is added after creation of the Gluster Enabled Cluster, through the 'New Host' dialogue box, the iptables firewall rules are being added during bootstrapping, as expected.

snippet from installation logs of RHS server added through the 'Configure Host' option of the 'New Cluster - Guide Me' Dialogue Box, is given below

--------------------------------------------------------------

2013-08-20 14:51:10 DEBUG otopi.plugins.otopi.dialog.machine dialog.__logString:215 DIALOG:SEND       ***Q:STRING CUSTOMIZATION_COMMAND
2013-08-20 14:51:10 DEBUG otopi.plugins.otopi.dialog.machine dialog.__logString:215 DIALOG:SEND       ###
2013-08-20 14:51:10 DEBUG otopi.plugins.otopi.dialog.machine dialog.__logString:215 DIALOG:SEND       ### Customization phase, use 'install' to proceed
2013-08-20 14:51:10 DEBUG otopi.plugins.otopi.dialog.machine dialog.__logString:215 DIALOG:SEND       ### COMMAND> 
2013-08-20 14:51:10 DEBUG otopi.plugins.otopi.dialog.machine dialog.__logString:215 DIALOG:RECEIVE    env-query -k NETWORK/iptablesEnable
2013-08-20 14:51:10 DEBUG otopi.plugins.otopi.dialog.machine dialog.__logString:215 DIALOG:SEND       ***Q:VALUE NETWORK/iptablesEnable
2013-08-20 14:51:10 DEBUG otopi.plugins.otopi.dialog.machine dialog.__logString:215 DIALOG:SEND       ###
2013-08-20 14:51:10 DEBUG otopi.plugins.otopi.dialog.machine dialog.__logString:215 DIALOG:SEND       ### Please specify value for 'NETWORK/iptablesEnable':
2013-08-20 14:51:10 DEBUG otopi.plugins.otopi.dialog.machine dialog.__logString:215 DIALOG:SEND       ### Response is VALUE NETWORK/iptablesEnable=type:value or ABORT NETWORK/iptablesEnable
2013-08-20 14:51:10 DEBUG otopi.plugins.otopi.dialog.machine dialog.__logString:215 DIALOG:RECEIVE    VALUE NETWORK/iptablesEnable=bool:false

--------------------------------------------------------------

snippet from installation logs of RHS server added through the 'New Host' dialogue box, after creation of the Gluster Enabled Cluster, is given below
--------------------------------------------------------------

2013-08-20 16:18:30 DEBUG otopi.plugins.otopi.dialog.machine dialog.__logString:215 DIALOG:SEND       ***Q:STRING CUSTOMIZATION_COMMAND
2013-08-20 16:18:30 DEBUG otopi.plugins.otopi.dialog.machine dialog.__logString:215 DIALOG:SEND       ###
2013-08-20 16:18:30 DEBUG otopi.plugins.otopi.dialog.machine dialog.__logString:215 DIALOG:SEND       ### Customization phase, use 'install' to proceed
2013-08-20 16:18:30 DEBUG otopi.plugins.otopi.dialog.machine dialog.__logString:215 DIALOG:SEND       ### COMMAND> 
2013-08-20 16:18:30 DEBUG otopi.plugins.otopi.dialog.machine dialog.__logString:215 DIALOG:RECEIVE    env-query -k NETWORK/iptablesEnable
2013-08-20 16:18:30 DEBUG otopi.plugins.otopi.dialog.machine dialog.__logString:215 DIALOG:SEND       ***Q:VALUE NETWORK/iptablesEnable
2013-08-20 16:18:30 DEBUG otopi.plugins.otopi.dialog.machine dialog.__logString:215 DIALOG:SEND       ###
2013-08-20 16:18:30 DEBUG otopi.plugins.otopi.dialog.machine dialog.__logString:215 DIALOG:SEND       ### Please specify value for 'NETWORK/iptablesEnable':
2013-08-20 16:18:30 DEBUG otopi.plugins.otopi.dialog.machine dialog.__logString:215 DIALOG:SEND       ### Response is VALUE NETWORK/iptablesEnable=type:value or ABORT NETWORK/iptablesEnable
2013-08-20 16:18:30 DEBUG otopi.plugins.otopi.dialog.machine dialog.__logString:215 DIALOG:RECEIVE    VALUE NETWORK/iptablesEnable=bool:true

--------------------------------------------------------------

Version-Release number of selected component (if applicable):
Red Hat Enterprise Virtualization Manager Version: 3.2.2-0.41.el6ev
RHS 2.1 (glusterfs-server-3.4.0.20rhs-2.el6rhs.x86_64)

How reproducible:


Steps to Reproduce:
1.Create Data Center with POSIX compliant FS Storage Type - compatibility version 3.2
2.Create Gluster Enabled Cluster
3.Choose to 'Configure Host' at the second screen during creation of Gluster Enabled Cluster, and add RHS server
4.Examine firewall rules after bootstrapping and reboot of RHS server

Actual results:
The iptables firewall rules are not being added during bootstrapping of RHS server, added through the 'Configure Host' option, of the 'New Cluster - Guide Me' Dialogue Box.

Expected results:
The iptables firewall rules should be added during bootstrapping of RHS server, added through the 'Configure Host' option, of the 'New Cluster - Guide Me' Dialogue Box.

Additional info:

Comment 1 Rejy M Cyriac 2013-08-20 11:38:22 UTC
Created attachment 788452 [details]
installation logs of RHS server added through 'Configure Host' Dialogue Box

Comment 2 Rejy M Cyriac 2013-08-20 11:39:35 UTC
Created attachment 788453 [details]
installation logs of RHS server added through 'New Host' Dialogue Box

Comment 3 Rejy M Cyriac 2013-08-20 11:40:24 UTC
Created attachment 788454 [details]
rhevm-log-collector logs from RHEVM

Comment 4 Itamar Heim 2013-08-20 17:30:50 UTC
dup of bug 961645?

Comment 5 Rejy M Cyriac 2013-08-20 17:44:51 UTC
(In reply to Itamar Heim from comment #4)
> dup of bug 961645?

I had seen that BZ, but it is not clear whether the issue reported there, is due to the absence of specific ports not being opened during bootstrapping (as in BZ 991145), or failure to set any firewall rules at all during bootstrapping as is the case here.

Moreover, the issue reported here is also unique in that the firewall rules are not being added during bootstrapping of RHS server, only when the RHS server is added through the 'Configure Host' option, of the 'New Cluster - Guide Me' Dialogue Box. If the RHS server is added after creation of the Gluster Enabled Cluster, through the 'New Host' dialogue box, the iptables firewall rules are being added during bootstrapping, as expected.

I think further analysis by devel would be good to ensure whether issues reported in these BZs are caused by same bugs or different ones.

Comment 6 Alon Bar-Lev 2013-08-22 06:16:01 UTC
The guide me dialog is probably not setting OverrideFirewall of the InstallVdsCommand parameters.

I am unsure who should handle that...

Comment 8 David Botzer 2013-09-30 08:58:09 UTC
I used 3.3/is16
I created new DC - POSIX compliant FS Storage Type 3.3 - Via the guideme
I created a cluster (cpu intel) 
  Enabled Gluster Service
          Checkbox - import existing Gluster config (Inserted my RHS info)
I added new Host with Firewall checkbox checked

Host is added correctly with iptables and rules

RHS didnt rebooted, nor has any rules on it

Please advise,

Comment 9 Alexander Wels 2013-09-30 12:12:21 UTC
As far as I know this is just configuring the rules/iptables on the host, I don't think it should alter any firewall settings on the RHS box.

Comment 10 David Botzer 2013-10-01 13:13:58 UTC
3.3/is16
I installed setup with host that is able to add gluster SD
What should I check now  ?

Comment 11 Einav Cohen 2013-10-04 17:29:50 UTC
(In reply to David Botzer from comment #10)
> 3.3/is16
> I installed setup with host that is able to add gluster SD
> What should I check now  ?

Rejy: as you reported this BZ: Can you please assist David with reproducing the scenario so this BZ can be VERIFIED (or reopened if not fixed)?
Many thanks in advance.

Comment 12 Rejy M Cyriac 2013-10-07 06:43:05 UTC
(In reply to Einav Cohen from comment #11)
> (In reply to David Botzer from comment #10)
> > 3.3/is16
> > I installed setup with host that is able to add gluster SD
> > What should I check now  ?
> 
> Rejy: as you reported this BZ: Can you please assist David with reproducing
> the scenario so this BZ can be VERIFIED (or reopened if not fixed)?
> Many thanks in advance.

Moving the 'needinfo' to Satheesaran, who is the current QA contact for RHEV+RHS . I will also provide assistance as and when required by Satheesaran.

Satheesaran,

Could you please assist David with verification of this BZ ?

- rejy (rmc)

Comment 13 SATHEESARAN 2013-10-07 07:39:18 UTC
(In reply to David Botzer from comment #10)
> 3.3/is16
> I installed setup with host that is able to add gluster SD
> What should I check now  ?

Hi David,

After installing RHEVM 3.3 IS16, you can do the following to test this,

1. Create a NFS Data-center of compatibility 3.2
2. Once Data-Center is created, you can see the "GuideMe" Dialog window, coming up, showing you help, to create "New Cluster".
3. Using that dialog window, create a new cluster of compatibility 3.2.
4. After creating a new cluster, again you will see a "GuideMe" dialog window coming up, showing "Configure Host".
5. Select "Configure Host" and add a RHS 2.1 Node to the cluster
6. Once the RHS Node is UP in the cluster, ssh in to RHS Node and check for newly added 'iptables' rules
(i.e) iptables -L

Expected : New rules for RHS Node should get added.

FYI : I tried the same with RHEVM 3.3 IS16 and I could not able to see any iptables rules added, when adding RHS 2.1 Node to the cluster.
But when I added RHS 2.1 Node, without using "GuideMe" dialog window,(using 'New" tab in the cluster) I could see the iptables rules are getting added.
Believe this will be then a FailedQA.

Comment 14 Haim 2013-10-27 16:52:55 UTC
> 
> FYI : I tried the same with RHEVM 3.3 IS16 and I could not able to see any
> iptables rules added, when adding RHS 2.1 Node to the cluster.
> But when I added RHS 2.1 Node, without using "GuideMe" dialog window,(using
> 'New" tab in the cluster) I could see the iptables rules are getting added.
> Believe this will be then a FailedQA.

based on comment 13, moving bug back to assigned.

Comment 17 Einav Cohen 2013-11-25 21:49:14 UTC
this is working for the BZ assignee, no one has contacted us with a reproducible setup, closing.

do not re-open unless you have a setup:
- in which this BZ is reproducible. 
- that is available for the BZ assignee to investigate on. 

thanks.

Comment 18 SATHEESARAN 2013-12-24 06:38:11 UTC
(In reply to Einav Cohen from comment #17)
> this is working for the BZ assignee, no one has contacted us with a
> reproducible setup, closing.
> 
> do not re-open unless you have a setup:
> - in which this BZ is reproducible. 
> - that is available for the BZ assignee to investigate on. 
> 
> thanks.

The issue still seen, but this missed the NEEDINFO raised previously.

The issue very clear, 
1. when I the datacenter is created, it automatically pops up "Guide Me" Dialog window, where I select "New Cluster"

2. I have created a new gluster enabled cluster, post the creation of the new cluster, it again pops up the "Guide Me" dialog box, which asks for adding new host to the cluster

3. Added RHSS Node to the gluster enabled cluster, with the help of "Guide Me"

Observation: There is no iptables rules added.
[root@rhss1 ~]# iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination  

4. I added a new RHSS Node to the cluster without using "Guide Me" window
I see that iptables rules are added to RHSS Nodes now
[root@rhss2 ~]# iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     all  --  anywhere             anywhere            state RELATED,ESTABLISHED 
ACCEPT     all  --  anywhere             anywhere            
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:54321 
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:ssh 
ACCEPT     udp  --  anywhere             anywhere            udp dpt:snmp 
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:24007 
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:webcache 
ACCEPT     udp  --  anywhere             anywhere            udp dpt:sunrpc 
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:38465 
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:38466 
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:sunrpc 
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:38467 
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:nfs 
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:38469 
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:39543 
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:55863 
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:38468 
ACCEPT     udp  --  anywhere             anywhere            udp dpt:963 
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:965 
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:ctdb 
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:netbios-ssn 
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:microsoft-ds 
ACCEPT     tcp  --  anywhere             anywhere            tcp dpts:24009:24108 
ACCEPT     tcp  --  anywhere             anywhere            tcp dpts:49152:49251 
REJECT     all  --  anywhere             anywhere            reject-with icmp-host-prohibited 

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         
REJECT     all  --  anywhere             anywhere            PHYSDEV match ! --physdev-is-bridged reject-with icmp-host-prohibited 

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

Comment 19 SATHEESARAN 2013-12-24 06:52:11 UTC
The issue is seen with IS28. I have setup available with me, please let me know if you need the setup information.

This issue was seen long back and mailed the same steps to Alexander Weils and providing the same steps again,

On 10/08/2013 12:50 AM, Alexander Wels wrote:
> Hi,
>
> I am looking at your latest comment on this bug, and I have tried to reproduce 
> the issue, but I am unable to do so. Can you provide a little more detail on 
> what you did to create the issue?
>
> Thanks,
> Alexander
Hi Alexander,

The problem is, there are no iptables rules added to RHS 2.1 Nodes, while
adding the RHS 2.1 Node to gluster enabled cluster using "Help" window.

If I do the same seperately, after creating the cluster I could see
IP tables rules added in RHS 2.1 nodes

To make it clear, I have attached the images. I have did the same tests and attached the screenshots
Test 1 - As you see in attachment, snapshot{80..85} - Describes the way to add the RHS Node to gluster enabled cluster
using "Guide Me" dialog window

Test 2 - Snapshot{86 .. 88} - shows the result when added the RHS Node seperately to gluster enabled cluster

Result of Test1
==========

[Tue Oct  8 14:59:05 UTC 2013 root.37.153:~ ] # iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination        

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination        

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination


Result of Test2
=========

[Tue Oct  8 14:59:08 UTC 2013 root.37.153:~ ] # iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination        
ACCEPT     all  --  anywhere             anywhere            state RELATED,ESTABLISHED
ACCEPT     all  --  anywhere             anywhere           
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:54321
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:ssh
ACCEPT     udp  --  anywhere             anywhere            udp dpt:snmp
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:24007
ACCEPT     udp  --  anywhere             anywhere            udp dpt:sunrpc
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:38465
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:38466
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:sunrpc
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:38467
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:nfs
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:39543
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:55863
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:38468
ACCEPT     udp  --  anywhere             anywhere            udp dpt:963
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:965
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:ctdb
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:netbios-ssn
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:microsoft-ds
ACCEPT     tcp  --  anywhere             anywhere            tcp dpts:24009:24108
ACCEPT     tcp  --  anywhere             anywhere            tcp dpts:49152:49251
REJECT     all  --  anywhere             anywhere            reject-with icmp-host-prohibited

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination        
REJECT     all  --  anywhere             anywhere            PHYSDEV match ! --physdev-is-bridged reject-with icmp-host-prohibited

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

Hope that helps
-- Satheesaran

Comment 20 SATHEESARAN 2013-12-24 06:54:52 UTC
Attaching the screenshots of adding RHSS Nodes using "Guide Me" dialog window.

Test 1 - As you see in attachment, snapshot{80..85} - Describes the way to add the RHS Node to gluster enabled cluster
using "Guide Me" dialog window

Test 2 - Snapshot{86 .. 88} - shows the result when added the RHS Node seperately to gluster enabled cluster - without making use of "Guide Me" option

Comment 21 SATHEESARAN 2013-12-24 06:58:29 UTC
Created attachment 841114 [details]
screenshots

Comment 23 Alexander Wels 2014-01-03 17:02:16 UTC
After getting the full steps to reproduce this. I was able to reproduce and fix the issue. I couldn't reproduce due to the fact I was testing the guide me in clusters section after I had already established a data center. Which the previous patch had fixed. But it turns out going from a fresh data center follows a different code path (different model) which was not yet fixed.

Comment 27 Arthur Berezin 2014-02-12 16:32:15 UTC
Per conversation with Einav this is an easy fix, Let's fix this for Z as well.

Comment 28 Einav Cohen 2014-02-20 12:32:36 UTC
QA: please test adding Host via the Add Host dialog from *all* of the following locations and make sure that the "override iptables" check-box is working properly:

- "Add Host" button/context-menu item within the Hosts main tab
- Cluster "Guide Me"
- Data Center "Guide Me"

Comment 30 Tareq Alayan 2014-02-25 15:04:13 UTC
iptables rules overridden:

Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     all  --  anywhere             anywhere            state RELATED,ESTABLISHED 
ACCEPT     all  --  anywhere             anywhere            
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:54321 
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:ssh 
ACCEPT     udp  --  anywhere             anywhere            udp dpt:snmp 
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:16514 
ACCEPT     tcp  --  anywhere             anywhere            multiport dports vnc-server:6923 
ACCEPT     tcp  --  anywhere             anywhere            multiport dports 49152:49216 
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:24007 
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:webcache 
ACCEPT     udp  --  anywhere             anywhere            udp dpt:sunrpc 
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:38465 
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:38466 
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:sunrpc 
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:38467 
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:nfs 
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:38469 
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:39543 
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:55863 
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:38468 
ACCEPT     udp  --  anywhere             anywhere            udp dpt:963 
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:965 
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:ctdb 
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:netbios-ssn 
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:microsoft-ds 
ACCEPT     tcp  --  anywhere             anywhere            tcp dpts:24009:24108 
ACCEPT     tcp  --  anywhere             anywhere            tcp dpts:49152:49251 
REJECT     all  --  anywhere             anywhere            reject-with icmp-host-prohibited 



verifed on ovirt-engine-3.4.0-0.11.beta3.el6.noarch

Comment 31 Itamar Heim 2014-06-12 14:04:44 UTC
Closing as part of 3.4.0