Hide Forgot
Description of problem: After deployment of OSP10 puddle 11.11.16 with ospd I noticed that in namespace (qrouter) there is a rule that drop metadata traffic and as a results vm cant boot with IP address. log sosreport : https://drive.google.com/a/redhat.com/file/d/0B_izhJVSkOTDWXZpX1VCRTEtTGc/view?usp=sharing Version-Release number of selected component (if applicable): osp10 puddle 11-11-16 How reproducible: you can see in log from 13/11/16 around noon. Steps to Reproduce: 1.after deployment of osp10 with ospd try to launch vm 2. 3. Actual results: no ip address Expected results: boot vm Additional info:
(In reply to Eran Kuris from comment #0) > > Actual results: > no ip address > Does it mean the instance won't get IP address from DHCP server at all or you just can't SSH there because it won't get a key injected? Note that DHCP works on a totally different port than metadata and DHCP Offers don't even go through qrouter namespace.
I looked at provided sosreport and iptables rules seem fine: PREROUTING: -A neutron-l3-agent-PREROUTING -d 169.254.169.254/32 -i qr-+ -p tcp -m tcp --dport 80 -j MARK --set-xmark 0x1/0xffff ^^ this is where metadata requests get a mark FITLER: -A neutron-l3-agent-INPUT -m mark --mark 0x1/0xffff -j ACCEPT -A neutron-l3-agent-INPUT -p tcp -m tcp --dport 9697 -j DROP ^^ The first rule accepts all marked traffic from PREROUTING table. All other non-marked requests are dropped. It would be helpful to do some troubleshooting and get number of packets that hit the DROP rule and PREROUTING rules. Also to describe to which private networks are instances connected.
it mean the instance won't get IP address from DHCP server at all. About troubleshooting its little problematic because I continue my tests. It would be great if you can have setup to reproduce it.
Just to give more details the setup is SRIOV BM. I changed this rule to accept and after that VM got IP address : -A neutron-l3-agent-INPUT -p tcp -m tcp --dport 9697 -j ACCEPT Details of the network & OVS bridge : neutron net-show 73dbb335-a2ef-416a-81e8-012a80981060 +---------------------------+--------------------------------------+ | Field | Value | +---------------------------+--------------------------------------+ | admin_state_up | True | | availability_zone_hints | | | availability_zones | nova | | created_at | 2016-11-13T14:08:04Z | | description | | | id | 73dbb335-a2ef-416a-81e8-012a80981060 | | ipv4_address_scope | | | ipv6_address_scope | | | mtu | 1496 | | name | internal_network | | port_security_enabled | True | | project_id | 7321ca08aa164d4490af6e5b59c114e3 | | provider:network_type | vlan | | provider:physical_network | tenant | | provider:segmentation_id | 225 | | qos_policy_id | | | revision_number | 5 | | router:external | False | | shared | False | | status | ACTIVE | | subnets | a61a12d9-92a1-42e3-86c3-cde007bf098b | | tags | | | tenant_id | 7321ca08aa164d4490af6e5b59c114e3 | | updated_at | 2016-11-13T14:08:39Z | +---------------------------+--------------------------------------+ [root@controller-0 ~]# ovs-vsctl show 0f661470-f45e-4ecb-8bea-5bbe8d367f80 Manager "ptcp:6640:127.0.0.1" is_connected: true Bridge br-ex Controller "tcp:127.0.0.1:6633" is_connected: true fail_mode: secure Port "vlan221" tag: 221 Interface "vlan221" type: internal Port phy-br-ex Interface phy-br-ex type: patch options: {peer=int-br-ex} Port "vlan222" tag: 222 Interface "vlan222" type: internal Port "vlan223" tag: 223 Interface "vlan223" type: internal Port "vlan181" tag: 181 Interface "vlan181" type: internal Port "vlan220" tag: 220 Interface "vlan220" type: internal Port br-ex Interface br-ex type: internal Port "p1p2" Interface "p1p2" Bridge br-int Controller "tcp:127.0.0.1:6633" is_connected: true fail_mode: secure Port "qg-e105672a-49" tag: 2 Interface "qg-e105672a-49" type: internal Port "tapdb0e4c0d-86" tag: 1 Interface "tapdb0e4c0d-86" type: internal Port "tap161b6bae-16" tag: 3 Interface "tap161b6bae-16" type: internal Port "qg-4eed9afd-5b" tag: 2 Interface "qg-4eed9afd-5b" type: internal Port "qg-bb9f8ad2-d2" tag: 2 Interface "qg-bb9f8ad2-d2" type: internal Port br-int Interface br-int type: internal Port "qr-ff23c319-91" tag: 3 Interface "qr-ff23c319-91" type: internal Port "qr-e4a5f43d-96" tag: 1 Interface "qr-e4a5f43d-96" type: internal Port "tapfbf1474a-13" tag: 2 Interface "tapfbf1474a-13" type: internal Port int-br-ex Interface int-br-ex type: patch options: {peer=phy-br-ex} ovs_version: "2.5.0" IPtable hit count : [root@controller-0 ~]# ip net exec qrouter-0b07318a-b43e-4ec8-aaf7-9dbec040e7b6 bash [root@controller-0 ~]# iptables -vL Chain INPUT (policy ACCEPT 16 packets, 656 bytes) pkts bytes target prot opt in out source destination 16 656 neutron-l3-agent-INPUT all -- any any anywhere anywhere Chain FORWARD (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination 0 0 neutron-filter-top all -- any any anywhere anywhere 0 0 neutron-l3-agent-FORWARD all -- any any anywhere anywhere Chain OUTPUT (policy ACCEPT 15 packets, 988 bytes) pkts bytes target prot opt in out source destination 15 988 neutron-filter-top all -- any any anywhere anywhere 15 988 neutron-l3-agent-OUTPUT all -- any any anywhere anywhere Chain neutron-filter-top (2 references) pkts bytes target prot opt in out source destination 15 988 neutron-l3-agent-local all -- any any anywhere anywhere Chain neutron-l3-agent-FORWARD (1 references) pkts bytes target prot opt in out source destination 0 0 neutron-l3-agent-scope all -- any any anywhere anywhere Chain neutron-l3-agent-INPUT (1 references) pkts bytes target prot opt in out source destination 0 0 ACCEPT all -- any any anywhere anywhere mark match 0x1/0xffff 0 0 DROP tcp -- any any anywhere anywhere tcp dpt:9697 Chain neutron-l3-agent-OUTPUT (1 references) pkts bytes target prot opt in out source destination Chain neutron-l3-agent-local (1 references) pkts bytes target prot opt in out source destination Chain neutron-l3-agent-scope (1 references) pkts bytes target prot opt in out source destination 0 0 DROP all -- any qr-e4a5f43d-96 anywhere anywhere mark match ! 0x4000000/0xffff0000 You can connect my setup to see ENV I see the issue now again .
sorry my mistake it does not reproduce. I will close the bug till it reproduce .