Bug 1394814 - droping metadata traffic rule in iptable of namespace
Summary: droping metadata traffic rule in iptable of namespace
Keywords:
Status: CLOSED WORKSFORME
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-neutron
Version: 10.0 (Newton)
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: ---
Assignee: Assaf Muller
QA Contact: Toni Freger
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-11-14 13:45 UTC by Eran Kuris
Modified: 2016-11-17 11:20 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-11-17 11:20:40 UTC
Target Upstream Version:


Attachments (Terms of Use)

Description Eran Kuris 2016-11-14 13:45:45 UTC
Description of problem:
After deployment of OSP10 puddle 11.11.16 with ospd  I noticed that in namespace (qrouter) there is a rule that drop metadata traffic and as a results vm cant boot with IP address. 

log sosreport : 

https://drive.google.com/a/redhat.com/file/d/0B_izhJVSkOTDWXZpX1VCRTEtTGc/view?usp=sharing
Version-Release number of selected component (if applicable):
osp10 puddle 11-11-16 

How reproducible:
you can see in log from 13/11/16 around noon.

Steps to Reproduce:
1.after deployment of osp10 with ospd try to launch vm 
2.
3.

Actual results:
no ip address 

Expected results:

boot vm 
Additional info:

Comment 1 Jakub Libosvar 2016-11-15 09:08:18 UTC
(In reply to Eran Kuris from comment #0)
> 
> Actual results:
> no ip address 
> 

Does it mean the instance won't get IP address from DHCP server at all or you just can't SSH there because it won't get a key injected?

Note that DHCP works on a totally different port than metadata and DHCP Offers don't even go through qrouter namespace.

Comment 2 Jakub Libosvar 2016-11-15 10:02:31 UTC
I looked at provided sosreport and iptables rules seem fine:

PREROUTING:
-A neutron-l3-agent-PREROUTING -d 169.254.169.254/32 -i qr-+ -p tcp -m tcp --dport 80 -j MARK --set-xmark 0x1/0xffff

^^ this is where metadata requests get a mark

FITLER:
-A neutron-l3-agent-INPUT -m mark --mark 0x1/0xffff -j ACCEPT
-A neutron-l3-agent-INPUT -p tcp -m tcp --dport 9697 -j DROP

^^ The first rule accepts all marked traffic from PREROUTING table. All other non-marked requests are dropped.


It would be helpful to do some troubleshooting and get number of packets that hit the DROP rule and PREROUTING rules. Also to describe to which private networks are instances connected.

Comment 3 Eran Kuris 2016-11-15 11:22:44 UTC
it mean the instance won't get IP address from DHCP server at all.
About troubleshooting its little problematic because I continue my tests.
It would be great if you can have setup to reproduce it.

Comment 4 Eran Kuris 2016-11-17 09:41:35 UTC
Just to give more details the setup is SRIOV BM.

I changed this rule to accept and after that VM got IP address :
-A neutron-l3-agent-INPUT -p tcp -m tcp --dport 9697 -j ACCEPT

Details of the network & OVS bridge : 
neutron net-show 73dbb335-a2ef-416a-81e8-012a80981060
+---------------------------+--------------------------------------+
| Field                     | Value                                |
+---------------------------+--------------------------------------+
| admin_state_up            | True                                 |
| availability_zone_hints   |                                      |
| availability_zones        | nova                                 |
| created_at                | 2016-11-13T14:08:04Z                 |
| description               |                                      |
| id                        | 73dbb335-a2ef-416a-81e8-012a80981060 |
| ipv4_address_scope        |                                      |
| ipv6_address_scope        |                                      |
| mtu                       | 1496                                 |
| name                      | internal_network                     |
| port_security_enabled     | True                                 |
| project_id                | 7321ca08aa164d4490af6e5b59c114e3     |
| provider:network_type     | vlan                                 |
| provider:physical_network | tenant                               |
| provider:segmentation_id  | 225                                  |
| qos_policy_id             |                                      |
| revision_number           | 5                                    |
| router:external           | False                                |
| shared                    | False                                |
| status                    | ACTIVE                               |
| subnets                   | a61a12d9-92a1-42e3-86c3-cde007bf098b |
| tags                      |                                      |
| tenant_id                 | 7321ca08aa164d4490af6e5b59c114e3     |
| updated_at                | 2016-11-13T14:08:39Z                 |
+---------------------------+--------------------------------------+

[root@controller-0 ~]# ovs-vsctl  show 
  0f661470-f45e-4ecb-8bea-5bbe8d367f80
    Manager "ptcp:6640:127.0.0.1"
        is_connected: true
    Bridge br-ex
        Controller "tcp:127.0.0.1:6633"
            is_connected: true
        fail_mode: secure
        Port "vlan221"
            tag: 221
            Interface "vlan221"
                type: internal
        Port phy-br-ex
            Interface phy-br-ex
                type: patch
                options: {peer=int-br-ex}
        Port "vlan222"
            tag: 222
            Interface "vlan222"
                type: internal
        Port "vlan223"
            tag: 223
            Interface "vlan223"
                type: internal
        Port "vlan181"
            tag: 181
            Interface "vlan181"
                type: internal
        Port "vlan220"
            tag: 220
            Interface "vlan220"
                type: internal
        Port br-ex
            Interface br-ex
                type: internal
        Port "p1p2"
            Interface "p1p2"
    Bridge br-int
        Controller "tcp:127.0.0.1:6633"
            is_connected: true
        fail_mode: secure
        Port "qg-e105672a-49"
            tag: 2
            Interface "qg-e105672a-49"
                type: internal
        Port "tapdb0e4c0d-86"
            tag: 1
            Interface "tapdb0e4c0d-86"
                type: internal
        Port "tap161b6bae-16"
            tag: 3
            Interface "tap161b6bae-16"
                type: internal
        Port "qg-4eed9afd-5b"
            tag: 2
            Interface "qg-4eed9afd-5b"
                type: internal
        Port "qg-bb9f8ad2-d2"
            tag: 2
            Interface "qg-bb9f8ad2-d2"
                type: internal
        Port br-int
            Interface br-int
                type: internal
        Port "qr-ff23c319-91"
            tag: 3
            Interface "qr-ff23c319-91"
                type: internal
        Port "qr-e4a5f43d-96"
            tag: 1
            Interface "qr-e4a5f43d-96"
                type: internal
        Port "tapfbf1474a-13"
            tag: 2
            Interface "tapfbf1474a-13"
                type: internal
        Port int-br-ex
            Interface int-br-ex
                type: patch
                options: {peer=phy-br-ex}
    ovs_version: "2.5.0"





IPtable hit count :
[root@controller-0 ~]# ip net exec qrouter-0b07318a-b43e-4ec8-aaf7-9dbec040e7b6 bash 
[root@controller-0 ~]# iptables -vL
Chain INPUT (policy ACCEPT 16 packets, 656 bytes)
 pkts bytes target     prot opt in     out     source               destination         
   16   656 neutron-l3-agent-INPUT  all  --  any    any     anywhere             anywhere            

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 neutron-filter-top  all  --  any    any     anywhere             anywhere            
    0     0 neutron-l3-agent-FORWARD  all  --  any    any     anywhere             anywhere            

Chain OUTPUT (policy ACCEPT 15 packets, 988 bytes)
 pkts bytes target     prot opt in     out     source               destination         
   15   988 neutron-filter-top  all  --  any    any     anywhere             anywhere            
   15   988 neutron-l3-agent-OUTPUT  all  --  any    any     anywhere             anywhere            

Chain neutron-filter-top (2 references)
 pkts bytes target     prot opt in     out     source               destination         
   15   988 neutron-l3-agent-local  all  --  any    any     anywhere             anywhere            

Chain neutron-l3-agent-FORWARD (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 neutron-l3-agent-scope  all  --  any    any     anywhere             anywhere            

Chain neutron-l3-agent-INPUT (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ACCEPT     all  --  any    any     anywhere             anywhere             mark match 0x1/0xffff
    0     0 DROP       tcp  --  any    any     anywhere             anywhere             tcp dpt:9697

Chain neutron-l3-agent-OUTPUT (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain neutron-l3-agent-local (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain neutron-l3-agent-scope (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DROP       all  --  any    qr-e4a5f43d-96  anywhere             anywhere             mark match ! 0x4000000/0xffff0000





You can connect my setup to see ENV I see the issue now again .

Comment 5 Eran Kuris 2016-11-17 11:20:40 UTC
sorry my mistake it does not reproduce. 
I will close the bug till it reproduce .


Note You need to log in before you can comment on or make changes to this bug.