Bug 1720166

Summary: [DVR] No namespaces created on compute node when fip is attached to an instance
Product: Red Hat OpenStack Reporter: Jakub Libosvar <jlibosva>
Component: openstack-neutronAssignee: Slawek Kaplonski <skaplons>
Status: CLOSED CURRENTRELEASE QA Contact: Candido Campos <ccamposr>
Severity: high Docs Contact:
Priority: unspecified    
Version: 14.0 (Rocky)CC: amuller, bhaley, ccamposr, chrisw, dalvarez, njohnston, scohen, skaplons
Target Milestone: ---Keywords: ZStream
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-05-21 14:50:59 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Neutron logs none

Description Jakub Libosvar 2019-06-13 10:00:05 UTC
Description of problem:
When using DVR, fip is unreachable because compute node doesn't have qrouter and fip namespaces created.

There are server logs from controllers and l3 and ovs agent logs from the compute node. They should contain following resources:

(overcloud) [stack@f12-h17-b01-5039ms ~]$ openstack router list --name dvrrouter
+--------------------------------------+-----------+--------+-------+-------------+-------+----------------------------------+
| ID                                   | Name      | Status | State | Distributed | HA    | Project                          |
+--------------------------------------+-----------+--------+-------+-------------+-------+----------------------------------+
| b1622b17-bb7c-41d9-ba21-754874cdee1e | dvrrouter | ACTIVE | UP    | True        | False | 766f7590521c44cb9b5fb71e37346bab |
+--------------------------------------+-----------+--------+-------+-------------+-------+----------------------------------+
(overcloud) [stack@f12-h17-b01-5039ms ~]$ openstack network list --name dvrtest
+--------------------------------------+---------+--------------------------------------+
| ID                                   | Name    | Subnets                              |
+--------------------------------------+---------+--------------------------------------+
| 297c4367-3b59-46a9-9a93-738d13c9ce79 | dvrtest | 05801741-eb1b-48c1-b282-d3f66e8aaa2f |
+--------------------------------------+---------+--------------------------------------+
(overcloud) [stack@f12-h17-b01-5039ms ~]$ openstack subnet list --name dvrtest_subnet
+--------------------------------------+----------------+--------------------------------------+-----------------+
| ID                                   | Name           | Network                              | Subnet          |
+--------------------------------------+----------------+--------------------------------------+-----------------+
| 05801741-eb1b-48c1-b282-d3f66e8aaa2f | dvrtest_subnet | 297c4367-3b59-46a9-9a93-738d13c9ce79 | 192.168.42.0/24 |
+--------------------------------------+----------------+--------------------------------------+-----------------+
(overcloud) [stack@f12-h17-b01-5039ms ~]$ openstack port list --network dvrtest
+--------------------------------------+------+-------------------+------------------------------------------------------------------------------+--------+
| ID                                   | Name | MAC Address       | Fixed IP Addresses                                                           | Status |
+--------------------------------------+------+-------------------+------------------------------------------------------------------------------+--------+
| 78a276ea-f194-4114-8327-73d57d687ba6 |      | fa:16:3e:9c:bf:bb | ip_address='192.168.42.23', subnet_id='05801741-eb1b-48c1-b282-d3f66e8aaa2f' | ACTIVE |
| b244ae83-9aec-453c-ac4b-851c685d8875 |      | fa:16:3e:51:95:64 | ip_address='192.168.42.2', subnet_id='05801741-eb1b-48c1-b282-d3f66e8aaa2f'  | ACTIVE |
| c6377ace-66b0-45f2-a4aa-907682f671dd |      | fa:16:3e:8b:90:16 | ip_address='192.168.42.1', subnet_id='05801741-eb1b-48c1-b282-d3f66e8aaa2f'  | ACTIVE |
| c8f1b0ca-4aab-4440-b680-152f688fbfef |      | fa:16:3e:e7:a4:18 | ip_address='192.168.42.10', subnet_id='05801741-eb1b-48c1-b282-d3f66e8aaa2f' | ACTIVE |
| d42e1089-7c50-4618-a50e-9a59834d783e |      | fa:16:3e:d2:ac:a3 | ip_address='192.168.42.3', subnet_id='05801741-eb1b-48c1-b282-d3f66e8aaa2f'  | ACTIVE |
| df1e3d60-746d-4b97-bdb0-27bdd3c3fc9c |      | fa:16:3e:43:69:20 | ip_address='192.168.42.4', subnet_id='05801741-eb1b-48c1-b282-d3f66e8aaa2f'  | ACTIVE |
+--------------------------------------+------+-------------------+------------------------------------------------------------------------------+--------+
(overcloud) [stack@f12-h17-b01-5039ms ~]$ openstack port list --router dvrrouter
+--------------------------------------+------+-------------------+------------------------------------------------------------------------------+--------+
| ID                                   | Name | MAC Address       | Fixed IP Addresses                                                           | Status |
+--------------------------------------+------+-------------------+------------------------------------------------------------------------------+--------+
| 78a276ea-f194-4114-8327-73d57d687ba6 |      | fa:16:3e:9c:bf:bb | ip_address='192.168.42.23', subnet_id='05801741-eb1b-48c1-b282-d3f66e8aaa2f' | ACTIVE |
| c6377ace-66b0-45f2-a4aa-907682f671dd |      | fa:16:3e:8b:90:16 | ip_address='192.168.42.1', subnet_id='05801741-eb1b-48c1-b282-d3f66e8aaa2f'  | ACTIVE |
| e35b9b1d-35c2-4493-8dab-c7f44d16e3ee |      | fa:16:3e:29:8c:ed | ip_address='172.21.2.27', subnet_id='9ebc48aa-26e0-46af-a65f-27a31676d752'   | ACTIVE |
+--------------------------------------+------+-------------------+------------------------------------------------------------------------------+--------+

On compute node, there are no namespaces
[root@overcloud-compute-2 ~]# ip net
[root@overcloud-compute-2 ~]# 

The flows for br-int and br-tun are also attached. The traffic gets dropped by table 1

[root@overcloud-compute-2 ~]# ovs-ofctl dump-flows br-int in_port=2
 cookie=0xa74d484a786516f8, duration=3689.805s, table=0, n_packets=214, n_bytes=8988, priority=2,in_port="patch-tun",dl_src=fa:16:3f:00:0c:97 actions=resubmit(,1)
 cookie=0xa74d484a786516f8, duration=3689.800s, table=0, n_packets=77, n_bytes=3234, priority=2,in_port="patch-tun",dl_src=fa:16:3f:4b:ed:9e actions=resubmit(,1)
 cookie=0xa74d484a786516f8, duration=3689.796s, table=0, n_packets=63, n_bytes=3078, priority=2,in_port="patch-tun",dl_src=fa:16:3f:6c:66:e1 actions=resubmit(,1)
 cookie=0xa74d484a786516f8, duration=3689.791s, table=0, n_packets=485, n_bytes=20370, priority=2,in_port="patch-tun",dl_src=fa:16:3f:a5:46:32 actions=resubmit(,1)
 cookie=0xa74d484a786516f8, duration=3689.786s, table=0, n_packets=1941, n_bytes=179018, priority=2,in_port="patch-tun",dl_src=fa:16:3f:db:04:b0 actions=resubmit(,1)
 cookie=0xa74d484a786516f8, duration=3689.781s, table=0, n_packets=59, n_bytes=3158, priority=2,in_port="patch-tun",dl_src=fa:16:3f:e5:65:5e actions=resubmit(,1)
[root@overcloud-compute-2 ~]# ovs-ofctl dump-flows br-int table=1
 cookie=0xa74d484a786516f8, duration=3696.303s, table=1, n_packets=2831, n_bytes=217350, priority=1 actions=drop

Version-Release number of selected component (if applicable):
openstack-neutron-13.0.3-0.20190119134915.886782c.el7ost.noarch

How reproducible:
always

Steps to Reproduce:
openstack network create dvrtest
openstack subnet create --network dvrtest dvrtest_subnet --subnet-range 192.168.42.0/24
openstack router create dvrrouter
openstack router set --external-gateway public dvrrouter
openstack router add subnet  dvrrouter dvrtest_subnet
openstack server create --image cirros --flavor m1.tiny --nic net-id=dvrtest --security-group test  --availability-zone nova:overcloud-compute-2.localdomain dvrvm
# the test security group allows ssh and ping
openstack floating ip create public
openstack floating ip set --port c8f1b0ca-4aab-4440-b680-152f688fbfef 172.21.2.38
# c8f1b0ca-4aab-4440-b680-152f688fbfef is the port of the VM
# 172.21.2.38 is the floating ip
ping 172.21.2.38

Actual results:
floating ip is not reachable

Expected results:


Additional info:

Comment 1 Daniel Alvarez Sanchez 2019-06-13 10:04:27 UTC
Created attachment 1580203 [details]
Neutron logs

Comment 2 Daniel Alvarez Sanchez 2019-06-13 10:05:07 UTC
Attached neutron servers logs and L3/OVS agents from compute2