Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1613384

Summary: SRIOV instances on VLAN cannot use the distributed virtual routers.
Product: Red Hat OpenStack Reporter: Miguel Angel Ajo <majopela>
Component: openvswitchAssignee: Numan Siddique <nusiddiq>
Status: CLOSED CURRENTRELEASE QA Contact: Eran Kuris <ekuris>
Severity: high Docs Contact:
Priority: high    
Version: 13.0 (Queens)CC: amuller, apevec, atragler, chrisw, dalvarez, ekuris, haili, jschluet, lhh, lmartins, majopela, mmichels, nusiddiq, rhos-maint, slinaber, srevivo, tredaelli
Target Milestone: z6Keywords: TestOnly, Triaged, ZStream
Target Release: 13.0 (Queens)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: openvswitch-2.9.0-97.el7fdp Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1610711 Environment:
Last Closed: 2019-04-16 10:39:50 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1561880, 1673027    
Bug Blocks:    

Description Miguel Angel Ajo 2018-08-07 13:17:52 UTC 
With SRIOV instances, the VM has a virtual network card VF, that goes straight physically to the ToR switch on an specific VLAN.

This will not work with OVN because the gateway nodes need to be reached on the L3 router MAC address, and the MAC address flaps on current implementation.
Comment 1 Miguel Angel Ajo 2018-08-07 13:20:07 UTC 
We have Anil's patches that partially address VLAN support, but SRIOV+VLAN will be not covered because of the detail explained on previous comment
Comment 2 Miguel Angel Ajo 2018-08-07 13:22:22 UTC 
https://patchwork.ozlabs.org/bundle/mangelajo/l3-vlan/
Comment 8 Lon Hohberger 2019-03-21 10:35:57 UTC 
According to our records, this should be resolved by openvswitch-2.9.0-97.el7fdp.  This build is available now.
Comment 10 Eran Kuris 2019-03-28 07:00:38 UTC 
The target release is osp13z6 moving to modify until we get a puddle.
Comment 11 Eran Kuris 2019-04-16 10:32:39 UTC 
Fix verified: OpenStack/13.0-RHEL-7/2019-04-10.1/
[heat-admin@controller-1 ~]$ sudo -i 
[root@controller-1 ~]# rpm -qa | grep -i openvswitch 
openvswitch-selinux-extra-policy-1.0-9.el7fdp.noarch
openvswitch-ovn-host-2.9.0-103.el7fdp.x86_64
python-openvswitch-2.9.0-103.el7fdp.x86_64
openstack-neutron-openvswitch-12.0.5-11.el7ost.noarch
openvswitch-2.9.0-103.el7fdp.x86_64
openvswitch-ovn-central-2.9.0-103.el7fdp.x86_64
openvswitch-ovn-common-2.9.0-103.el7fdp.x86_64

Created setup 
                                 R
                                /  \
                    IPv4_subnet1     IPv4_subnet2
                       /                  \
        net-64-1_vf-sriov port         net-64-2_normal- normal port 


(overcloud) [stack@undercloud-0 ~]$ openstack server list --all 
+--------------------------------------+-----------------+--------+-------------------------------------------------------------+--------+-------------+
| ID                                   | Name            | Status | Networks                                                    | Image  | Flavor      |
+--------------------------------------+-----------------+--------+-------------------------------------------------------------+--------+-------------+
| 244eca44-e52b-48f7-a556-28b999b92912 | net-64-2_normal | ACTIVE | net-64-2=10.0.2.12, 2002::f816:3eff:fe58:ee1d, 10.46.21.214 | rhel75 | rhel_flavor |
| 778f0be1-f446-413a-a2df-af8f2fb7dda5 | net-64-1_normal | ACTIVE | net-64-1=10.0.1.15, 2001::f816:3eff:fe98:b79e, 10.46.21.216 | rhel75 | rhel_flavor |
| 249a6cfd-54c7-45b2-bd2b-6ce9e521b2bc | net-64-1_pf     | ACTIVE | net-64-1=10.0.1.12, 2001::faf2:1eff:fe15:a886, 10.46.21.218 | rhel75 | rhel_flavor |
| e89af562-2f09-4674-a700-46dd8dccc628 | net-64-1_vf     | ACTIVE | net-64-1=10.0.1.9, 2001::f816:3eff:fe7c:45a9, 10.46.21.219  | rhel75 | rhel_flavor |
+--------------------------------------+-----------------+--------+-------------------------------------------------------------+--------+-------------+
(overcloud) [stack@undercloud-0 ~]$ ssh root.21.219
Warning: Permanently added '10.46.21.219' (ECDSA) to the list of known hosts.
root.21.219's password: 
[root@host-10-0-1-9 ~]# ifconfig 
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.0.1.9  netmask 255.255.255.0  broadcast 10.0.1.255
        inet6 2001::f816:3eff:fe7c:45a9  prefixlen 64  scopeid 0x0<global>
        inet6 fe80::f816:3eff:fe7c:45a9  prefixlen 64  scopeid 0x20<link>
        ether fa:16:3e:7c:45:a9  txqueuelen 1000  (Ethernet)
        RX packets 705  bytes 70014 (68.3 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 760  bytes 62607 (61.1 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 20  bytes 1730 (1.6 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 20  bytes 1730 (1.6 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

[root@host-10-0-1-9 ~]# ping 10.46.21.214
PING 10.46.21.214 (10.46.21.214) 56(84) bytes of data.
64 bytes from 10.46.21.214: icmp_seq=1 ttl=62 time=1.94 ms
64 bytes from 10.46.21.214: icmp_seq=2 ttl=62 time=0.670 ms
64 bytes from 10.46.21.214: icmp_seq=3 ttl=62 time=0.360 ms
64 bytes from 10.46.21.214: icmp_seq=4 ttl=62 time=0.368 ms
64 bytes from 10.46.21.214: icmp_seq=5 ttl=62 time=0.397 ms
64 bytes from 10.46.21.214: icmp_seq=6 ttl=62 time=0.402 ms
^C
--- 10.46.21.214 ping statistics ---
6 packets transmitted, 6 received, 0% packet loss, time 5002ms
rtt min/avg/max/mdev = 0.360/0.690/1.948/0.573 ms


[root@controller-1 ~]# ovn-nbctl --db=tcp:10.20.1.62:6641 show
switch 19efe169-0b37-4df4-a264-7e1cd564d4fa (neutron-080a0fb8-a9b2-4709-9c3f-4beb95ff7705) (aka nova)
    port 8e4e049f-06af-4a7b-803f-d7af80552d7d
        type: router
        router-port: lrp-8e4e049f-06af-4a7b-803f-d7af80552d7d
    port c7e206bc-1b37-44f0-8af9-374c37a6626f
        type: localport
        addresses: ["fa:16:3e:00:86:04", "unknown"]
    port provnet-080a0fb8-a9b2-4709-9c3f-4beb95ff7705
        type: localnet
        tag: 319
        addresses: ["unknown"]
switch d175118d-9975-43ba-b378-c7a25451fdb1 (neutron-ed3a9153-c5c0-440c-a73e-5bf9cc7885c8) (aka net-64-1)
    port a3ddddd9-87f6-4419-b012-11edc9693e71
        type: router
        addresses: ["fa:16:3e:7d:c6:f9"]
        router-port: lrp-a3ddddd9-87f6-4419-b012-11edc9693e71
    port f35cd671-621b-48b6-b4b4-c0ce562a4a42
        addresses: ["fa:16:3e:39:f3:a1 10.0.1.3 2001::f816:3eff:fe39:f3a1", "unknown"]
    port ecb205cf-102a-4b76-9d13-d82f9468d3c4 (aka normal)
        addresses: ["fa:16:3e:98:b7:9e 10.0.1.15 2001::f816:3eff:fe98:b79e"]
    port 0844edf5-dbb8-4a9e-bec5-0a5aafe7cbfe (aka PF_sriov)
        addresses: ["f8:f2:1e:15:a8:86 10.0.1.12 2001::faf2:1eff:fe15:a886"]
    port provnet-ed3a9153-c5c0-440c-a73e-5bf9cc7885c8
        type: localnet
        tag: 323
        addresses: ["unknown"]
    port 6cda3e16-e686-418d-9e4d-8f85e139ff36
        type: router
        router-port: lrp-6cda3e16-e686-418d-9e4d-8f85e139ff36
    port ca4519b0-437b-4ee6-a828-5433e920fdec
        addresses: ["fa:16:3e:1d:c8:f5 10.0.1.5 2001::f816:3eff:fe1d:c8f5", "unknown"]
    port 360b3662-df64-4a00-ac3d-d7187752819a
        type: localport
        addresses: ["fa:16:3e:be:20:15 10.0.1.2 2001::f816:3eff:febe:2015", "unknown"]
    port 0fde6bcd-659d-4e5e-bc36-2e053575b91f (aka vf_sriov)
        addresses: ["fa:16:3e:7c:45:a9 10.0.1.9 2001::f816:3eff:fe7c:45a9"]
    port 8779df05-9221-4230-9e51-f9bdbb54487f
        addresses: ["fa:16:3e:62:40:d4 10.0.1.4 2001::f816:3eff:fe62:40d4", "unknown"]
switch 9aa96d63-fa87-4c28-bb66-3af577e11278 (neutron-c1bb9c11-a9ec-4945-bef6-ed1f02a95aa0) (aka net-64-2)
    port 5d92f56c-5a39-4f76-94c1-7b912e31240d
        type: localport
        addresses: ["fa:16:3e:29:89:9c 10.0.2.2 2002::f816:3eff:fe29:899c", "unknown"]
    port e8d98aae-ef6c-4ec7-b5c6-ed5235bc832b
        addresses: ["fa:16:3e:19:81:4c 10.0.2.4 2002::f816:3eff:fe19:814c", "unknown"]
    port 3acba78d-940a-4947-8298-901559e28daf
        addresses: ["fa:16:3e:54:f2:5e 10.0.2.3 2002::f816:3eff:fe54:f25e", "unknown"]
    port 4ef5a156-6dee-4612-b8a1-9af5a5be77ef (aka normal)
        addresses: ["fa:16:3e:58:ee:1d 10.0.2.12 2002::f816:3eff:fe58:ee1d"]
    port e8d77fa0-bc9f-4068-8fe4-ef64c7e5bad7
        type: router
        addresses: ["fa:16:3e:13:9c:2a"]
        router-port: lrp-e8d77fa0-bc9f-4068-8fe4-ef64c7e5bad7
    port provnet-c1bb9c11-a9ec-4945-bef6-ed1f02a95aa0
        type: localnet
        tag: 320
        addresses: ["unknown"]
    port a4de7be2-0319-49c7-9631-9f82d83d68c3
        type: router
        router-port: lrp-a4de7be2-0319-49c7-9631-9f82d83d68c3
    port cb7e5b87-ecbf-48e8-b3c4-28d90983780f
        addresses: ["fa:16:3e:4a:5a:14 10.0.2.5 2002::f816:3eff:fe4a:5a14", "unknown"]
router f6b64354-da16-4cf6-addc-0df5eba28ecf (neutron-f7d53331-b7d5-4e85-9396-b74a80a72dc1) (aka Router_eNet)
    port lrp-8e4e049f-06af-4a7b-803f-d7af80552d7d
        mac: "fa:16:3e:d0:31:89"
        networks: ["10.46.21.213/26"]
        gateway chassis: [904f6c76-32f5-4a84-8cd7-c7ef1fb308d1 a1fb4424-7d7b-4473-a947-dcdff3b1cfb6 858c5ba8-1099-462b-a50e-cd5d67d0cfb2]
    port lrp-a4de7be2-0319-49c7-9631-9f82d83d68c3
        mac: "fa:16:3e:0c:ef:d4"
        networks: ["10.0.2.1/24"]
    port lrp-6cda3e16-e686-418d-9e4d-8f85e139ff36
        mac: "fa:16:3e:5d:e1:b1"
        networks: ["10.0.1.1/24"]
    port lrp-a3ddddd9-87f6-4419-b012-11edc9693e71
        mac: "fa:16:3e:7d:c6:f9"
        networks: ["2001::1/64"]
    port lrp-e8d77fa0-bc9f-4068-8fe4-ef64c7e5bad7
        mac: "fa:16:3e:13:9c:2a"
        networks: ["2002::1/64"]
    nat 63539118-2cde-475c-8ab8-a6128bbca591
        external ip: "10.46.21.214"
        logical ip: "10.0.2.12"
        type: "dnat_and_snat"
    nat 6b213a81-65c8-4fe5-87f1-11676dd11dea
        external ip: "10.46.21.219"
        logical ip: "10.0.1.9"
        type: "dnat_and_snat"
    nat 750fc7c3-a4bf-4cc0-b47a-930d781a3bf2
        external ip: "10.46.21.218"
        logical ip: "10.0.1.12"
        type: "dnat_and_snat"
    nat c2c59e92-b63f-4d87-b714-031112698ccb
        external ip: "10.46.21.213"
        logical ip: "10.0.1.0/24"
        type: "snat"
    nat c4f71eb9-93a3-4863-bf0f-fe0351c20986
        external ip: "10.46.21.216"
        logical ip: "10.0.1.15"
        type: "dnat_and_snat"
    nat c644f52e-0f62-4617-8294-fab2bb32d26a
        external ip: "10.46.21.213"
        logical ip: "10.0.2.0/24"
        type: "snat"