Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1437576

Summary: Instance connectivity issues disabling security groups and port security
Product: Red Hat OpenStack Reporter: Punit Kundal <pkundal>
Component: openstack-neutronAssignee: Daniel Alvarez Sanchez <dalvarez>
Status: CLOSED ERRATA QA Contact: Toni Freger <tfreger>
Severity: medium Docs Contact:
Priority: medium    
Version: 10.0 (Newton)CC: amuller, chrisw, dalvarez, mburns, nshetty, nyechiel, oblaut, pkundal, rhel-osp-director-maint, sbaker, shardy, srevivo, zbitter
Target Milestone: z4Keywords: Triaged, ZStream
Target Release: 10.0 (Newton)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: openstack-neutron-9.1.1-7.el7ost Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-09-06 17:17:18 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
neutron, heat and openvswitch logs from the controller nodes none

Description Punit Kundal 2017-03-30 15:06:23 UTC
Created attachment 1267592 [details]
neutron, heat and openvswitch logs from the controller nodes

Description of problem:

When specifying the security_groups property as null and port_security_enabled property as false for a OS::Neutron::Port resource type, instance is inaccessible, when completely omitting these parameters from the resource definition in the heat template, instance can be accessed normally.

I have tested these properties in different scenarios i.e. instance with one nic and instance with multiple(2) nics. 

Version-Release number of selected component (if applicable):


How reproducible:
All the tested scenarios are always reproducible.

I created 2 tenant networks and connected them to a single neutron router as below:

[stack@instack ~]$ neutron net-list
+--------------------------------------+------+-------------------------------------------------------+
| id                                   | name | subnets                                               |
+--------------------------------------+------+-------------------------------------------------------+
| 00b4a812-4e9c-40e9-aa84-c1bc2ffb970b | net2 | f0c8075d-c080-4ae3-8f79-433ef2dc6fc1 192.168.200.0/24 |
| 84602360-2569-45a7-9ef0-2e855560f118 | net1 | 7547d68d-a9cb-4da8-90ca-1d8c66da3052 172.16.3.0/24    |
| eb98f2fb-92cb-4de3-8522-f5fc035c644a | net3 | 6077f955-e7b6-48f5-8f95-a64c96592ccf 192.168.202.0/24 |
+--------------------------------------+------+-------------------------------------------------------+

[stack@instack ~]$ neutron router-port-list router2
+--------------------------------------+------+-------------------+-----------------------------------------------------+
| id                                   | name | mac_address       | fixed_ips                                           |
+--------------------------------------+------+-------------------+-----------------------------------------------------+
| 980417c8-3e32-46d3-bb92-7f896976c639 |      | fa:16:3e:81:83:46 | {"subnet_id":                                       |
|                                      |      |                   | "f0c8075d-c080-4ae3-8f79-433ef2dc6fc1",             |
|                                      |      |                   | "ip_address": "192.168.200.1"}                      |
| ffe5d552-5eb5-459b-be12-10de528b0655 |      | fa:16:3e:bf:71:f1 | {"subnet_id":                                       |
|                                      |      |                   | "6077f955-e7b6-48f5-8f95-a64c96592ccf",             |
|                                      |      |                   | "ip_address": "192.168.202.1"}                      |
+--------------------------------------+------+-------------------+-----------------------------------------------------+

Scenario 1: Instance with a single nic and having security_groups property set to null and port_security_enabled property set to false inside the heat template.

Heat template:

+++
[stack@instack ~]$ cat test1.yaml 
heat_template_version: 2016-04-08

description: testing ports

resources:
  port1:
    type: OS::Neutron::Port
    properties:
      network: net2
      security_groups: null
      port_security_enabled: false

  inst1:
    type: OS::Nova::Server
    properties:
      flavor: m1.micro
      image: cirros2
      networks:
        - port: { get_resource: port1 }
+++

Stack created from this heat template:

[stack@instack ~]$ heat stack-create -f test1.yaml stack1

[stack@instack ~]$ heat stack-list
WARNING (shell) "heat stack-list" is deprecated, please use "openstack stack list" instead
+--------------------------------------+------------+-----------------+----------------------+--------------+
| id                                   | stack_name | stack_status    | creation_time        | updated_time |
+--------------------------------------+------------+-----------------+----------------------+--------------+
| 6ca32c64-0444-4070-9744-259ec5aa6732 | stack1     | CREATE_COMPLETE | 2017-03-30T07:16:20Z | None         |
+--------------------------------------+------------+-----------------+----------------------+--------------+

[stack@instack ~]$ nova list
+--------------------------------------+---------------------------+--------+------------+-------------+--------------------+
| ID                                   | Name                      | Status | Task State | Power State | Networks           |
+--------------------------------------+---------------------------+--------+------------+-------------+--------------------+
| 4711104b-7263-47c9-a000-1807978969ca | stack1-inst1-5k7gpj57na7r | ACTIVE | -          | Running     | net2=192.168.200.9 |
+--------------------------------------+---------------------------+--------+------------+-------------+--------------------+

Ping testing on the controller node through the qrouter namespace:

[root@overcloud-controller-0 ~]# ip netns exec qrouter-7c2ead35-24ed-4423-829d-d218b6728c76 ping -c4 -I qr-980417c8-3e 192.168.200.9
PING 192.168.200.9 (192.168.200.9) from 192.168.200.1 qr-980417c8-3e: 56(84) bytes of data.
^C
--- 192.168.200.9 ping statistics ---
4 packets transmitted, 0 received, 100% packet loss, time 3000ms

For scenario 1, ping testing failed.


Scenario 2: Instance with a single nic without security_groups and port_security_enabled properties set for the port:

Heat template:

+++
[stack@instack ~]$ cat test2.yaml 
heat_template_version: 2016-04-08

description: testing ports

resources:
  port1:
    type: OS::Neutron::Port
    properties:
      network: net2

  inst1:
    type: OS::Nova::Server
    properties:
      flavor: m1.micro
      image: cirros2
      networks:
        - port: { get_resource: port1 }
+++

Stack created from this template:

[stack@instack ~]$ heat stack-create -f test2.yaml stack2

[stack@instack ~]$ heat stack-list
WARNING (shell) "heat stack-list" is deprecated, please use "openstack stack list" instead
+--------------------------------------+------------+-----------------+----------------------+--------------+
| id                                   | stack_name | stack_status    | creation_time        | updated_time |
+--------------------------------------+------------+-----------------+----------------------+--------------+
| 6ca32c64-0444-4070-9744-259ec5aa6732 | stack1     | CREATE_COMPLETE | 2017-03-30T07:16:20Z | None         |
| ef6333a5-ca1f-423c-a813-64b7a2351dc4 | stack2     | CREATE_COMPLETE | 2017-03-30T07:22:47Z | None         |
+--------------------------------------+------------+-----------------+----------------------+--------------+

[stack@instack ~]$ nova list
+--------------------------------------+---------------------------+--------+------------+-------------+--------------------+
| ID                                   | Name                      | Status | Task State | Power State | Networks           |
+--------------------------------------+---------------------------+--------+------------+-------------+--------------------+
| 4711104b-7263-47c9-a000-1807978969ca | stack1-inst1-5k7gpj57na7r | ACTIVE | -          | Running     | net2=192.168.200.9 |
| 4a5cb0c6-46fc-4a96-a2cf-3497dad01198 | stack2-inst1-p6cvmn2qlhwc | ACTIVE | -          | Running     | net2=192.168.200.6 |
+--------------------------------------+---------------------------+--------+------------+-------------+--------------------+

Ping testing on the controller node through the qrouter namespace:

[root@overcloud-controller-0 ~]# ip netns exec qrouter-7c2ead35-24ed-4423-829d-d218b6728c76 ping -c4 -I qr-980417c8-3e 192.168.200.6
PING 192.168.200.6 (192.168.200.6) from 192.168.200.1 qr-980417c8-3e: 56(84) bytes of data.
64 bytes from 192.168.200.6: icmp_seq=1 ttl=64 time=1.75 ms
64 bytes from 192.168.200.6: icmp_seq=2 ttl=64 time=1.44 ms
64 bytes from 192.168.200.6: icmp_seq=3 ttl=64 time=0.756 ms
64 bytes from 192.168.200.6: icmp_seq=4 ttl=64 time=0.824 ms

Ping testing succeeded for scenario 2.

Scenario 3: Instance with 2 nics, with security_groups and port_security_enabled specified for port1 and omitted for port2.

Heat template:

+++
[stack@instack ~]$ cat test3.yaml 
heat_template_version: 2016-04-08

description: testing ports

resources:
  port1:
    type: OS::Neutron::Port
    properties:
      network: net2
      security_groups: null
      port_security_enabled: false
 
  port2:
    type: OS::Neutron::Port
    properties:
      network: net3
    

  inst1:
    type: OS::Nova::Server
    properties:
      flavor: m1.micro
      image: cirros2
      networks:
        - port: { get_resource: port1 }
        - port: { get_resource: port2 }
+++

Stack created from this template:

[stack@instack ~]$ heat stack-create -f test3.yaml stack3

[stack@instack ~]$ heat stack-list
WARNING (shell) "heat stack-list" is deprecated, please use "openstack stack list" instead
+--------------------------------------+------------+-----------------+----------------------+--------------+
| id                                   | stack_name | stack_status    | creation_time        | updated_time |
+--------------------------------------+------------+-----------------+----------------------+--------------+
| 6ca32c64-0444-4070-9744-259ec5aa6732 | stack1     | CREATE_COMPLETE | 2017-03-30T07:16:20Z | None         |
| ef6333a5-ca1f-423c-a813-64b7a2351dc4 | stack2     | CREATE_COMPLETE | 2017-03-30T07:22:47Z | None         |
| 8bd78418-1b1c-49e5-8f00-df245b54239b | stack3     | CREATE_COMPLETE | 2017-03-30T07:37:02Z | None         |
+--------------------------------------+------------+-----------------+----------------------+--------------+

[stack@instack ~]$ nova list
+--------------------------------------+---------------------------+--------+------------+-------------+----------------------------------------+
| ID                                   | Name                      | Status | Task State | Power State | Networks                               |
+--------------------------------------+---------------------------+--------+------------+-------------+----------------------------------------+
| 4711104b-7263-47c9-a000-1807978969ca | stack1-inst1-5k7gpj57na7r | ACTIVE | -          | Running     | net2=192.168.200.9                     |
| 4a5cb0c6-46fc-4a96-a2cf-3497dad01198 | stack2-inst1-p6cvmn2qlhwc | ACTIVE | -          | Running     | net2=192.168.200.6                     |
| f33f1dcc-9440-4708-bb4f-14d657b4796d | stack3-inst1-om2evmmeuhjk | ACTIVE | -          | Running     | net3=192.168.202.8; net2=192.168.200.5 |
+--------------------------------------+---------------------------+--------+------------+-------------+----------------------------------------+

Ping testing on the controller node through the qrouter namespace:

[root@overcloud-controller-0 ~]# ip netns exec qrouter-7c2ead35-24ed-4423-829d-d218b6728c76 ping -c4 -I qr-980417c8-3e 192.168.200.5
PING 192.168.200.5 (192.168.200.5) from 192.168.200.1 qr-980417c8-3e: 56(84) bytes of data.
^C
--- 192.168.200.5 ping statistics ---
4 packets transmitted, 0 received, 100% packet loss, time 3008ms

[root@overcloud-controller-0 ~]# ip netns exec qrouter-7c2ead35-24ed-4423-829d-d218b6728c76 ping -c4 -I qr-ffe5d552-5e 192.168.202.8
PING 192.168.202.8 (192.168.202.8) from 192.168.202.1 qr-ffe5d552-5e: 56(84) bytes of data.
^C
--- 192.168.202.8 ping statistics ---
4 packets transmitted, 0 received, 100% packet loss, time 3000ms

For scenario 3, ping tests failed for both the networks, when they should atleast work for 192.168.202.8 ip since security_groups and port_security_enabled were not specified for this port.

Scenario 4: Instance with 2 nics with security_groups and port_security_groups omitted for port1 and specified for port2

Heat template:

+++
[stack@instack ~]$ cat test4.yaml 
heat_template_version: 2016-04-08

description: testing ports

resources:
  port1:
    type: OS::Neutron::Port
    properties:
      network: net2
  
  port2:
    type: OS::Neutron::Port
    properties:
      network: net3
      security_groups: null
      port_security_enabled: false
 
  inst1:
    type: OS::Nova::Server
    properties:
      flavor: m1.micro
      image: cirros2
      networks:
        - port: { get_resource: port1 }
        - port: { get_resource: port2 }
+++

Stack created from this template:

[stack@instack ~]$ heat stack-create -f test4.yaml stack4

[stack@instack ~]$ heat stack-list
WARNING (shell) "heat stack-list" is deprecated, please use "openstack stack list" instead
+--------------------------------------+------------+-----------------+----------------------+--------------+
| id                                   | stack_name | stack_status    | creation_time        | updated_time |
+--------------------------------------+------------+-----------------+----------------------+--------------+
| 6ca32c64-0444-4070-9744-259ec5aa6732 | stack1     | CREATE_COMPLETE | 2017-03-30T07:16:20Z | None         |
| ef6333a5-ca1f-423c-a813-64b7a2351dc4 | stack2     | CREATE_COMPLETE | 2017-03-30T07:22:47Z | None         |
| 8bd78418-1b1c-49e5-8f00-df245b54239b | stack3     | CREATE_COMPLETE | 2017-03-30T07:37:02Z | None         |
| 757c8651-9bcc-4ce0-8608-c50b255c5964 | stack4     | CREATE_COMPLETE | 2017-03-30T08:36:24Z | None         |
+--------------------------------------+------------+-----------------+----------------------+--------------+

[stack@instack ~]$ nova list
+--------------------------------------+---------------------------+---------+------------+-------------+------------------------------------------+
| ID                                   | Name                      | Status  | Task State | Power State | Networks                                 |
+--------------------------------------+---------------------------+---------+------------+-------------+------------------------------------------+
| 4711104b-7263-47c9-a000-1807978969ca | stack1-inst1-5k7gpj57na7r | SHUTOFF | -          | Shutdown    | net2=192.168.200.9                       |
| 4a5cb0c6-46fc-4a96-a2cf-3497dad01198 | stack2-inst1-p6cvmn2qlhwc | SHUTOFF | -          | Shutdown    | net2=192.168.200.6                       |
| f33f1dcc-9440-4708-bb4f-14d657b4796d | stack3-inst1-om2evmmeuhjk | SHUTOFF | -          | Shutdown    | net3=192.168.202.8; net2=192.168.200.5   |
| cd41e124-f623-48ad-840b-c7567e0af198 | stack4-inst1-iwn5y2wzkdkw | ACTIVE  | -          | Running     | net3=192.168.202.12; net2=192.168.200.14 |
+--------------------------------------+---------------------------+---------+------------+-------------+------------------------------------------+

Ping testing on the controller node from the qrouter namespace:

[root@overcloud-controller-0 ~]# ip netns exec qrouter-7c2ead35-24ed-4423-829d-d218b6728c76 ping -c 4 -I qr-980417c8-3e 192.168.200.14
PING 192.168.200.14 (192.168.200.14) from 192.168.200.1 qr-980417c8-3e: 56(84) bytes of data.
64 bytes from 192.168.200.14: icmp_seq=1 ttl=64 time=1.98 ms
64 bytes from 192.168.200.14: icmp_seq=2 ttl=64 time=0.878 ms
64 bytes from 192.168.200.14: icmp_seq=3 ttl=64 time=0.678 ms
64 bytes from 192.168.200.14: icmp_seq=4 ttl=64 time=0.741 ms

--- 192.168.200.14 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3002ms
rtt min/avg/max/mdev = 0.678/1.071/1.987/0.533 ms
[root@overcloud-controller-0 ~]# ip netns exec qrouter-7c2ead35-24ed-4423-829d-d218b6728c76 ping -c 4 -I qr-ffe5d552-5e 192.168.202.12
PING 192.168.202.12 (192.168.202.12) from 192.168.202.1 qr-ffe5d552-5e: 56(84) bytes of data.
^C
--- 192.168.202.12 ping statistics ---
4 packets transmitted, 0 received, 100% packet loss, time 3000ms


Now in this scenario, pings were working for port1 and not for port2.

So it seems that order of ports for which the 2 properties are specified is also affecting the behaviour.


Actual results:
Instance connectivity issues when specifying security_groups and port_security_enabled parameters in a heat template.


Please let me know if any other information is required.

Thanks and Regards,
Punit Kundal

Comment 1 Zane Bitter 2017-03-30 22:23:53 UTC
Heat does very little beyond passing the data you give it straight to Neutron. Can you retest using the Neutron client directly? Unless that is shown to work then I think this bug report should be directed at Neutron.

In particular, the line:

      security_groups: null

has no effect on what Heat passes to Neutron.

> So it seems that order of ports for which the 2 properties are specified is also affecting the behaviour.

That's possible, but the required order can't be inferred from this test. The order the ports are created in is not deterministic in the absence of a dependency relationship, so there's no reason to think that port2 will always be created before port3, or vice-versa.

Comment 2 Punit Kundal 2017-04-03 14:54:25 UTC
Hello Zane,

Sorry for the delay.

Here are the steps that I tried:

1) Created a neutron network and a subnet:

[stack@instack ~]$ neutron net-create  net1
Created a new network:
+---------------------------+--------------------------------------+
| Field                     | Value                                |
+---------------------------+--------------------------------------+
| admin_state_up            | True                                 |
| availability_zone_hints   |                                      |
| availability_zones        |                                      |
| created_at                | 2017-04-03T08:52:48Z                 |
| description               |                                      |
| id                        | 8d7fa3b1-6f05-4778-8ff3-f757e46c73e7 |
| ipv4_address_scope        |                                      |
| ipv6_address_scope        |                                      |
| mtu                       | 1446                                 |
| name                      | net1                                 |
| port_security_enabled     | True                                 |
| project_id                | ff39a6e012c34afb8321448b43a0d9d9     |
| provider:network_type     | vxlan                                |
| provider:physical_network |                                      |
| provider:segmentation_id  | 75                                   |
| qos_policy_id             |                                      |
| revision_number           | 3                                    |
| router:external           | False                                |
| shared                    | False                                |
| status                    | ACTIVE                               |
| subnets                   |                                      |
| tags                      |                                      |
| tenant_id                 | ff39a6e012c34afb8321448b43a0d9d9     |
| updated_at                | 2017-04-03T08:52:49Z                 |
+---------------------------+--------------------------------------+

[stack@instack ~]$ neutron subnet-create --name subnet1 net1 172.0.3.0/24
Created a new subnet:
+-------------------+----------------------------------------------+
| Field             | Value                                        |
+-------------------+----------------------------------------------+
| allocation_pools  | {"start": "172.0.3.2", "end": "172.0.3.254"} |
| cidr              | 172.0.3.0/24                                 |
| created_at        | 2017-04-03T08:53:15Z                         |
| description       |                                              |
| dns_nameservers   |                                              |
| enable_dhcp       | True                                         |
| gateway_ip        | 172.0.3.1                                    |
| host_routes       |                                              |
| id                | a04fe0b0-a4d7-4734-a96d-1bb8b9cdaf2f         |
| ip_version        | 4                                            |
| ipv6_address_mode |                                              |
| ipv6_ra_mode      |                                              |
| name              | subnet1                                      |
| network_id        | 8d7fa3b1-6f05-4778-8ff3-f757e46c73e7         |
| project_id        | ff39a6e012c34afb8321448b43a0d9d9             |
| revision_number   | 2                                            |
| service_types     |                                              |
| subnetpool_id     |                                              |
| tenant_id         | ff39a6e012c34afb8321448b43a0d9d9             |
| updated_at        | 2017-04-03T08:53:15Z                         |
+-------------------+----------------------------------------------+

2) Created a neutron port with no security groups and disabled port security for the same

[stack@instack ~]$ neutron port-create --name port1 --no-security-groups --port-security-enabled=False net1
Created a new port:
+-----------------------+----------------------------------------------------------------------------------+
| Field                 | Value                                                                            |
+-----------------------+----------------------------------------------------------------------------------+
| admin_state_up        | True                                                                             |
| allowed_address_pairs |                                                                                  |
| binding:host_id       |                                                                                  |
| binding:profile       | {}                                                                               |
| binding:vif_details   | {}                                                                               |
| binding:vif_type      | unbound                                                                          |
| binding:vnic_type     | normal                                                                           |
| created_at            | 2017-04-03T08:55:32Z                                                             |
| description           |                                                                                  |
| device_id             |                                                                                  |
| device_owner          |                                                                                  |
| extra_dhcp_opts       |                                                                                  |
| fixed_ips             | {"subnet_id": "a04fe0b0-a4d7-4734-a96d-1bb8b9cdaf2f", "ip_address": "172.0.3.4"} |
| id                    | f4a1f767-022f-4a75-929f-e3cc930b8b7f                                             |
| mac_address           | fa:16:3e:85:7c:fd                                                                |
| name                  | port1                                                                            |
| network_id            | 8d7fa3b1-6f05-4778-8ff3-f757e46c73e7                                             |
| port_security_enabled | False                                                                            |
| project_id            | ff39a6e012c34afb8321448b43a0d9d9                                                 |
| qos_policy_id         |                                                                                  |
| revision_number       | 4                                                                                |
| security_groups       |                                                                                  |
| status                | DOWN                                                                             |
| tenant_id             | ff39a6e012c34afb8321448b43a0d9d9                                                 |
| updated_at            | 2017-04-03T08:55:33Z                                                             |
+-----------------------+----------------------------------------------------------------------------------+

3) launched an instance with this port:
[stack@instack ~]$ nova boot --flavor m1.tiny --image cirros --nic port-id=f4a1f767-022f-4a75-929f-e3cc930b8b7f test1 

[stack@instack ~]$ nova list
+--------------------------------------+-------+--------+------------+-------------+----------------+
| ID                                   | Name  | Status | Task State | Power State | Networks       |
+--------------------------------------+-------+--------+------------+-------------+----------------+
| 97a68193-8506-4de0-9346-9ef408a69ce3 | test1 | ACTIVE | -          | Running     | net1=172.0.3.4 |
+--------------------------------------+-------+--------+------------+-------------+----------------+

4) Once the instance was running properly, from the controller node tried pinging the port ip address:

[root@overcloud-controller-0 ~]# ip netns exec qdhcp-8d7fa3b1-6f05-4778-8ff3-f757e46c73e7 ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
8: tap2a396c7b-2b: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1446 qdisc noqueue state UNKNOWN qlen 1000
    link/ether fa:16:3e:b9:67:02 brd ff:ff:ff:ff:ff:ff
    inet 172.0.3.2/24 brd 172.0.3.255 scope global tap2a396c7b-2b
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:feb9:6702/64 scope link 
       valid_lft forever preferred_lft forever

[root@overcloud-controller-0 ~]# ip netns exec qdhcp-8d7fa3b1-6f05-4778-8ff3-f757e46c73e7 ping -c4 172.0.3.4
PING 172.0.3.4 (172.0.3.4) 56(84) bytes of data.
From 172.0.3.2 icmp_seq=1 Destination Host Unreachable
From 172.0.3.2 icmp_seq=2 Destination Host Unreachable
From 172.0.3.2 icmp_seq=3 Destination Host Unreachable
From 172.0.3.2 icmp_seq=4 Destination Host Unreachable

the pings failed indeed. 

Just to make sure that there were no issues with connectivity, I also tried pinging the dhcp port ip address:

[root@overcloud-controller-0 ~]# ip netns exec qdhcp-8d7fa3b1-6f05-4778-8ff3-f757e46c73e7 ping -c4 172.0.3.2
PING 172.0.3.2 (172.0.3.2) 56(84) bytes of data.
64 bytes from 172.0.3.2: icmp_seq=1 ttl=64 time=0.045 ms
64 bytes from 172.0.3.2: icmp_seq=2 ttl=64 time=0.530 ms
64 bytes from 172.0.3.2: icmp_seq=3 ttl=64 time=0.032 ms
64 bytes from 172.0.3.2: icmp_seq=4 ttl=64 time=0.027 ms

this worked as expected.

Could you please suggest if these are the right steps as you asked before ?

Can you please let me know how can we proceed further with this ?

Thanks and Regards,
Punit

Comment 3 Zane Bitter 2017-04-03 19:32:13 UTC
I'm not quite clear on what you're saying. Are you saying that the behaviour was the same when you used Neutron directly? In that case we should assign the issue to the openstack-neutron component.

Or are you saying that when Heat was not involved, things worked as you wanted them to?

I'm not sure if it's relevant, but the configuration in the template:

      security_groups: null

is *not* equivalent to the CLI option --no-security-groups. Heat's equivalent to that option would be:

      security_groups: []

Comment 4 Punit Kundal 2017-04-05 13:33:51 UTC
Hello Zane,

Sorry for the confusion.

What I meant to say was that the results were the same when I tried the same thing with the neutron client as well. 

To be more precise, when I created a neutron port with --no-security-groups and --port-security-enabled=False, I was expecting that the pings to the instance would be successful, which is not the case.

And I also tested with "security_groups: []"

+++
[stack@instack ~]$ cat test1.yaml 
heat_template_version: 2016-04-08

description: testing ports

resources:
  port1:
    type: OS::Neutron::Port
    properties:
      network: net1
      security_groups: []
      port_security_enabled: false

  inst1:
    type: OS::Nova::Server
    properties:
      flavor: m1.tiny
      image: cirros
      networks:
        - port: { get_resource: port1 }
+++

In this scenario also the results were the same, pings to the instance won't work:

[stack@instack ~]$ heat stack-create -f test1.yaml stack1

[stack@instack ~]$ nova list
+--------------------------------------+---------------------------+---------+------------+-------------+------------------+
| ID                                   | Name                      | Status  | Task State | Power State | Networks         |
+--------------------------------------+---------------------------+---------+------------+-------------+------------------+
| ad96bd81-1c5b-48a3-846e-c419d863c5db | stack1-inst1-73g6roxxuqae | ACTIVE  | -          | Running     | net1=172.0.3.8   |
| e5d1d794-8677-4374-9761-d3847a1cd093 | testinst                  | SHUTOFF | -          | Shutdown    | net1=172.0.3.3   |
| 08b58566-2659-4f2d-8b3f-b2624e06ea9d | testinst1                 | SHUTOFF | -          | Shutdown    | net2=198.168.1.9 |
+--------------------------------------+---------------------------+---------+------------+-------------+------------------+

[stack@instack ~]$ neutron port-list
+--------------------------------------+---------------------------+-------------------+-------------------------------------------+
| id                                   | name                      | mac_address       | fixed_ips                                 |
+--------------------------------------+---------------------------+-------------------+-------------------------------------------+
| 1fe32cd4-4994-4906-bd1b-03ef1930eca5 | stack1-port1-joyjjt46ijgb | fa:16:3e:df:82:7c | {"subnet_id": "a04fe0b0-a4d7-4734-a96d-   |
|                                      |                           |                   | 1bb8b9cdaf2f", "ip_address": "172.0.3.8"} |


[stack@instack ~]$ neutron port-show stack1-port1-joyjjt46ijgb
+-----------------------+----------------------------------------------------------------------------------+
| Field                 | Value                                                                            |
+-----------------------+----------------------------------------------------------------------------------+
| admin_state_up        | True                                                                             |
| allowed_address_pairs |                                                                                  |
| binding:host_id       | overcloud-compute-0.localdomain                                                  |
| binding:profile       | {}                                                                               |
| binding:vif_details   | {"port_filter": true, "ovs_hybrid_plug": true}                                   |
| binding:vif_type      | ovs                                                                              |
| binding:vnic_type     | normal                                                                           |
| created_at            | 2017-04-05T07:26:26Z                                                             |
| description           |                                                                                  |
| device_id             | ad96bd81-1c5b-48a3-846e-c419d863c5db                                             |
| device_owner          | compute:None                                                                     |
| extra_dhcp_opts       |                                                                                  |
| fixed_ips             | {"subnet_id": "a04fe0b0-a4d7-4734-a96d-1bb8b9cdaf2f", "ip_address": "172.0.3.8"} |
| id                    | 1fe32cd4-4994-4906-bd1b-03ef1930eca5                                             |
| mac_address           | fa:16:3e:df:82:7c                                                                |
| name                  | stack1-port1-joyjjt46ijgb                                                        |
| network_id            | 8d7fa3b1-6f05-4778-8ff3-f757e46c73e7                                             |
| port_security_enabled | False                                                                            |
| project_id            | ff39a6e012c34afb8321448b43a0d9d9                                                 |
| qos_policy_id         |                                                                                  |
| revision_number       | 8                                                                                |
| security_groups       |                                                                                  |
| status                | ACTIVE                                                                           |
| tenant_id             | ff39a6e012c34afb8321448b43a0d9d9                                                 |
| updated_at            | 2017-04-05T07:26:35Z                                                             |
+-----------------------+----------------------------------------------------------------------------------+

[root@overcloud-controller-0 ~]# ip netns exec qrouter-7045444f-d7e8-4a67-bbb2-773345f4431a ping -c4 172.0.3.8
PING 172.0.3.8 (172.0.3.8) 56(84) bytes of data.
From 172.0.3.1 icmp_seq=1 Destination Host Unreachable
From 172.0.3.1 icmp_seq=2 Destination Host Unreachable
From 172.0.3.1 icmp_seq=3 Destination Host Unreachable
From 172.0.3.1 icmp_seq=4 Destination Host Unreachable

So do we need to report this bug for neutron ?

Many thanks for the help so far.

Regards,
Punit Kundal

Comment 5 Zane Bitter 2017-04-05 18:45:44 UTC
Thanks for clearing that up :) I'm reassigning this bug to openstack-neutron.

Comment 7 Assaf Muller 2017-04-27 22:35:04 UTC
Assigning to Daniel for triage who dealt with similar port security issues in the past.

Comment 8 Daniel Alvarez Sanchez 2017-05-16 14:05:50 UTC
Hi, could this be a duplicate of [1]?

Looks pretty much the same and it was fixed in openstack-neutron-9.1.1-7.el7ost
Worth giving a try I think.
Thanks,
Daniel

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1406263

Comment 9 Daniel Alvarez Sanchez 2017-05-16 16:20:27 UTC
Maybe you could also paste the iptables output so that I can verify if it's the same issue.

Comment 20 Punit Kundal 2017-06-01 04:29:55 UTC
Hello Daniel,

Here are the neutron package versions:

from controllers:

openstack-neutron-lbaas-9.1.0-1.el7ost.noarch
openstack-neutron-ml2-9.1.0-8.el7ost.noarch
python-neutronclient-6.0.0-2.el7ost.noarch
python-neutron-lib-0.4.0-1.el7ost.noarch
openstack-neutron-bigswitch-lldp-9.40.0-1.1.el7ost.noarch
openstack-neutron-metering-agent-9.1.0-8.el7ost.noarch
puppet-neutron-9.4.2-1.el7ost.noarch
python-neutron-9.1.0-8.el7ost.noarch
python-neutron-tests-9.1.0-8.el7ost.noarch
python-neutron-lbaas-9.1.0-1.el7ost.noarch
openstack-neutron-openvswitch-9.1.0-8.el7ost.noarch
openstack-neutron-bigswitch-agent-9.40.0-1.1.el7ost.noarch
openstack-neutron-sriov-nic-agent-9.1.0-8.el7ost.noarch
openstack-neutron-common-9.1.0-8.el7ost.noarch
openstack-neutron-9.1.0-8.el7ost.noarch

from computes:

openstack-neutron-lbaas-9.1.0-1.el7ost.noarch
openstack-neutron-ml2-9.1.0-8.el7ost.noarch
python-neutronclient-6.0.0-2.el7ost.noarch
python-neutron-lib-0.4.0-1.el7ost.noarch
openstack-neutron-bigswitch-lldp-9.40.0-1.1.el7ost.noarch
openstack-neutron-metering-agent-9.1.0-8.el7ost.noarch
puppet-neutron-9.4.2-1.el7ost.noarch
python-neutron-9.1.0-8.el7ost.noarch
python-neutron-tests-9.1.0-8.el7ost.noarch
python-neutron-lbaas-9.1.0-1.el7ost.noarch
openstack-neutron-openvswitch-9.1.0-8.el7ost.noarch
openstack-neutron-bigswitch-agent-9.40.0-1.1.el7ost.noarch
openstack-neutron-sriov-nic-agent-9.1.0-8.el7ost.noarch
openstack-neutron-common-9.1.0-8.el7ost.noarch
openstack-neutron-9.1.0-8.el7ost.noarch

Additionally, the issue is also seen on compute node 4.

Please let me know if any more information is required.

Comment 21 Daniel Alvarez Sanchez 2017-06-01 16:10:45 UTC
Hi Punit,

Thanks for the info.
The issue I suspect was fixed in openstack-neutron-9.1.1-7.el7ost so the version the customer is running has the bug.
Is it possible for you to give it a try?

Thanks,
Daniel

Comment 27 errata-xmlrpc 2017-09-06 17:17:18 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:2663