Bug 1411690 - Security groups firewall rules don't get applied on the compute nodes
Summary: Security groups firewall rules don't get applied on the compute nodes
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates
Version: 11.0 (Ocata)
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: Upstream M3
: 11.0 (Ocata)
Assignee: Brent Eagles
QA Contact: GenadiC
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-01-10 10:04 UTC by Marius Cornea
Modified: 2017-05-17 19:55 UTC (History)
10 users (show)

Fixed In Version: openstack-tripleo-heat-templates-6.0.0-0.20170127041112.ce54697.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-05-17 19:55:53 UTC


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2017:1245 normal SHIPPED_LIVE Red Hat OpenStack Platform 11.0 Bug Fix and Enhancement Advisory 2017-05-17 23:01:50 UTC
OpenStack gerrit 420947 None None None 2017-01-16 20:49:19 UTC
Launchpad 1656939 None None None 2017-01-16 20:41:58 UTC

Description Marius Cornea 2017-01-10 10:04:36 UTC
Description of problem:
The security groups firewall rules don't get applied on the compute nodes allowing all connections to overcloud instances.

Version-Release number of selected component (if applicable):
openstack-tripleo-heat-templates-6.0.0-0.20161220000655.58d711e.el7ost.noarch
openstack-neutron-common-10.0.0-0.20161220031059.85f748a.el7ost.noarch
openstack-neutron-ml2-10.0.0-0.20161220031059.85f748a.el7ost.noarch
openstack-neutron-openvswitch-10.0.0-0.20161220031059.85f748a.el7ost.noarch
openstack-neutron-10.0.0-0.20161220031059.85f748a.el7ost.noarch


How reproducible:
100%

Steps to Reproduce:
1. Deploy overcloud 
2. Create security group which allows only SSH connectivity:

neutron security-group-rule-list | grep server_security_group_noicmpssh

| 0003a27d-849a-4965-ad8f-893341b27125 | server_security_group_noicmpssh | egress    | IPv6      | any           | any             |
| 21e31c68-909c-406a-bef0-9e70385cede4 | server_security_group_noicmpssh | egress    | IPv4      | any           | any             |
| f2f91316-1ccb-442d-b1aa-958816273321 | server_security_group_noicmpssh | ingress   | IPv4      | 22/tcp        | any             |

3. Launch instance using this security group:

[stack@undercloud-0 ~]$ openstack server list
+--------------------------------------+-------------------------------------------------------+--------+-----------------------------------------------------------------------------------------------------+------------+
| ID                                   | Name                                                  | Status | Networks                                                                                            | Image Name |
+--------------------------------------+-------------------------------------------------------+--------+-----------------------------------------------------------------------------------------------------+------------+
| 29cc4b4b-5a71-419c-96fd-a22580e68bf7 | st--2e-instance-y33pgbra3ulz-my_instance-svroxlkg5e3m | ACTIVE | stack-2e-tenant_net_ext_tagged-wsm2ykihbt7j-private_network-y4trupq2kux7=10.10.10.11, 172.16.18.141 | Fedora     |
| 5f99a94f-9fa8-4eca-85e6-9004a0ac3a1d | st-provinstance-f7hxmxiiix4t-my_instance-m4xvs3525o62 | ACTIVE | provider01=172.16.19.18                                                                             | Fedora     |
+--------------------------------------+-------------------------------------------------------+--------+-----------------------------------------------------------------------------------------------------+------------+

[stack@undercloud-0 ~]$ openstack server show st-provinstance-f7hxmxiiix4t-my_instance-m4xvs3525o62 | grep security_groups
| security_groups                      | [{u'name': u'server_security_group_noicmpssh'}]          |

3. Ping instance:

[stack@undercloud-0 ~]$ ping -c1 172.16.19.18
PING 172.16.19.18 (172.16.19.18) 56(84) bytes of data.
64 bytes from 172.16.19.18: icmp_seq=1 ttl=63 time=0.937 ms


Actual results:
ICMP connectivity works.

Expected results:
ICMP connectivity shouldn't be allowed. 

Additional info:

The iptables rules on the compute node where the instance was launched:

[root@overcloud-compute-1 heat-admin]# iptables -nL
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0            /* 000 accept related established rules */ state RELATED,ESTABLISHED
ACCEPT     icmp --  0.0.0.0/0            0.0.0.0/0            /* 001 accept all icmp */ state NEW
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0            /* 002 accept all to lo interface */ state NEW
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            multiport dports 22 /* 003 accept ssh */ state NEW
ACCEPT     udp  --  0.0.0.0/0            0.0.0.0/0            multiport dports 123 /* 105 ntp */ state NEW
ACCEPT     udp  --  0.0.0.0/0            0.0.0.0/0            multiport dports 4789 /* 118 neutron vxlan networks */ state NEW
ACCEPT     udp  --  0.0.0.0/0            0.0.0.0/0            multiport dports 161 /* 127 snmp */ state NEW
ACCEPT     47   --  0.0.0.0/0            0.0.0.0/0            /* 136 neutron gre networks */
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            multiport dports 16509,16514,49152:49215,5900:5999 /* 200 nova_libvirt */ state NEW
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0            state RELATED,ESTABLISHED
ACCEPT     icmp --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0           
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0            state NEW tcp dpt:22
REJECT     all  --  0.0.0.0/0            0.0.0.0/0            reject-with icmp-host-prohibited
LOG        all  --  0.0.0.0/0            0.0.0.0/0            /* 998 log all */ LOG flags 0 level 4
DROP       all  --  0.0.0.0/0            0.0.0.0/0            /* 999 drop all */ state NEW

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         
REJECT     all  --  0.0.0.0/0            0.0.0.0/0            reject-with icmp-host-prohibited

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         


In /var/log/neutron/openvswitch-agent.log on the compute node I can see the following line:

2017-01-09 23:39:09.267 17916 INFO neutron.agent.securitygroups_rpc [req-9852164d-9de7-4d34-9160-5add8566310f - - - - -] Skipping method prepare_devices_filter as firewall is disabled or configured as NoopFirewallDriver.

It appears that the firewall_driver is not set in openvswitch_agent.ini:

[root@overcloud-compute-1 ~]# grep -Ri firewall_driver /etc/neutron/plugins/ml2/openvswitch_agent.ini
#firewall_driver = <None>
firewall_driver =

Comment 2 Assaf Muller 2017-01-16 19:01:11 UTC
Assigned to Jakub for root cause analysis.

Comment 3 Jakub Libosvar 2017-01-16 19:43:23 UTC
(In reply to Assaf Muller from comment #2)
> Assigned to Jakub for root cause analysis.

The RCA has been already done fine by the reporter :) The issue appears to be on the installer side as it doesn't set firewall_driver properly.

Comment 4 Brent Eagles 2017-01-16 20:30:20 UTC
In newton, setting an empty string in the heat templates as the default was resulting in the puppet default taking affect (ie. behaving as if unset). Something has changed in ocata that make this not the case. There are a few changes that might have caused this, but it's not clear at the moment precisely which one was the culprit. I'll submit a bug u/s as well as "harden" the neutron ovs agent template by adding a conditional on empty string.

Comment 5 Brent Eagles 2017-01-18 17:25:19 UTC
Patch merged u/s.

Comment 9 GenadiC 2017-03-20 15:28:34 UTC
Verified in openstack-tripleo-heat-templates-6.0.0-0.20170307170102.3134785.0rc2.el7ost.noarch

Created security group with only ssh enabled only and checked that no ICMP is passed to VM configured with this security group (Did it with basic test in neutron git scenario section)

Comment 11 errata-xmlrpc 2017-05-17 19:55:53 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1245


Note You need to log in before you can comment on or make changes to this bug.