Bug 1643905
| Summary: | Instances with floating IPs residing on two different subnet with their own routers cannot access each other using FIP on a RHOSP with OVN | ||
|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Daniel Alvarez Sanchez <dalvarez> |
| Component: | openvswitch | Assignee: | Daniel Alvarez Sanchez <dalvarez> |
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Roman Safronov <rsafrono> |
| Severity: | high | Docs Contact: | |
| Priority: | high | ||
| Version: | 13.0 (Queens) | CC: | apevec, chrisw, dalvarez, ekuris, lhh, lmartins, ltomasbo, majopela, mputhenp, nusiddiq, ovs-team, ragiman, rhos-maint, tredaelli |
| Target Milestone: | --- | Keywords: | TestOnly, Triaged, ZStream |
| Target Release: | --- | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | openvswitch-2.9.0-97.el7fdp | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | 1643902 | Environment: | |
| Last Closed: | 2019-03-25 10:35:54 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | 1642830, 1643900, 1643902 | ||
| Bug Blocks: | |||
|
Description
Daniel Alvarez Sanchez
2018-10-29 12:19:52 UTC
Patch is merged upstream in master branch: https://github.com/openvswitch/ovs/commit/81e928526b8a9393b90785fb0a9c82d79570ef84 Is it possible to get a backport to 2.9? I guess this would be ds only If not possible, we will just apply the workaround on networking-ovn side (In reply to Daniel Alvarez Sanchez from comment #4) > Patch is merged upstream in master branch: > https://github.com/openvswitch/ovs/commit/ > 81e928526b8a9393b90785fb0a9c82d79570ef84 > Is it possible to get a backport to 2.9? I guess this would be ds only Done. It's available in 2.9.0-89.el7fdn According to our records, this should be resolved by openvswitch-2.9.0-83.el7fdp.1. This build is available now. Tested on puddle: OpenStack/13.0-RHEL-7/2019-01-22.1
rpm: openvswitch-2.9.0-83.el7fdp.1.x86_64
The issue still occurs.
Steps that I did before the issue re-occurred:
Started from clean system.
1. Created internal network internal_A, subnet_A (192.168.3.0/24), router_A. Connected router_A to the external network. Connected internal network internal_A to the router. Created a VM and connected to the internal_A network. Attached an floating IP to the VM.
2. Created internal network internal_B, subnet_B (192.168.4.0/24), router_B. Connected router_B to the external network. Connected internal network internal_B to the router. Created a VM and connected to the internal_B network. Attached an floating IP to the VM.
3. On this stage I was able to ping VMs with floating IPs from external network and from other VM with floating IP (OK)
4. I deleted one of floating IPs. Then I re-created it again (same ip address, used command 'openstack floating ip create --floating-ip-address 10.0.0.220 nova' , 'nova' is external network name)
5. Attached the FIP to the other VM on other internal network, connected to other router. New mac_binding entry in north db was created (OK). Ping and ssh to this VM via FIP worked, however I noticed that arp table on hypevisor was not updated with new mac for 10.0.0.220.
6. Created internal network internal_C, subnet_C (192.168.5.0/24), router_C. Connected router_C to the external network. Connected internal network internal_C to the router. Created a VM and connected to the internal_C network.
7. Deleted FIP 10.0.0.20 and recreated it again (see step 4). Then attached it to the VM connected to internal_C network and router_C.
8. Connected to VM on internal_B network (via internal interface in namespace on compute node, the VM has FIP 10.0.0.212), tried to ping 10.0.0.220 - no replies (BUG)
This time MAC address in arp table was updated properly (OK)
[root@titan09 ~]# arp -n | grep 220
10.0.0.220 ether fa:16:3e:1a:b9:41 C external
However MAC_Binding table does not have entry for new mac (BAD)
_uuid : 983f17a8-39eb-42ef-81f7-014af443a257
datapath : e430caa6-45d6-4c89-b4ab-5f6fdd64421a
ip : "10.0.0.220"
logical_port : "lrp-1ad704fd-fe3d-45dc-8da0-de2c13bace3e"
mac : "fa:16:3e:47:cc:79"
_uuid : ed5dd9c1-81c1-4a8f-838f-329935403508
datapath : 3835e5d6-3cca-4568-a1d6-559d0de53095
ip : "10.0.0.212"
logical_port : "lrp-140de39c-b91c-4bce-9f80-42c0f9bdb646"
mac : "fa:16:3e:7a:cc:c2"
_uuid : 80cef92d-7f6d-4787-90ef-43c03765b605
datapath : e430caa6-45d6-4c89-b4ab-5f6fdd64421a
ip : "10.0.0.1"
logical_port : "lrp-1ad704fd-fe3d-45dc-8da0-de2c13bace3e"
mac : "52:54:00:90:eb:a9"
_uuid : 44f147b2-9033-4fa8-bf80-7df7b8c57e60
datapath : d59102ca-a542-48bc-9d6a-d815865e9c48
ip : "10.0.0.1"
logical_port : "lrp-95a230dc-355e-4a0f-a480-4aec7876df5a"
mac : "52:54:00:90:eb:a9"
_uuid : 23336992-adf1-4461-864f-b44589e78615
datapath : 3835e5d6-3cca-4568-a1d6-559d0de53095
ip : "10.0.0.220"
logical_port : "lrp-140de39c-b91c-4bce-9f80-42c0f9bdb646"
mac : "fa:16:3e:7a:cc:c2"
_uuid : 0a3c46b6-1e46-42d0-abe8-5d12796015da
datapath : e430caa6-45d6-4c89-b4ab-5f6fdd64421a
ip : "10.0.0.211"
logical_port : "lrp-1ad704fd-fe3d-45dc-8da0-de2c13bace3e"
mac : "fa:16:3e:47:cc:79"
_uuid : 34f08edb-7c1e-4783-88c4-93f8aa6f2205
datapath : 3835e5d6-3cca-4568-a1d6-559d0de53095
ip : "10.0.0.1"
logical_port : "lrp-140de39c-b91c-4bce-9f80-42c0f9bdb646"
mac : "52:54:00:90:eb:a9"
'ovn-nbctl show' output
switch 695bcff4-7363-414f-b220-26da7f8cc65a (neutron-41728a0a-ee96-4924-b186-2026f6de66fe) (aka nova)
port 140de39c-b91c-4bce-9f80-42c0f9bdb646
type: router
router-port: lrp-140de39c-b91c-4bce-9f80-42c0f9bdb646
port 1ad704fd-fe3d-45dc-8da0-de2c13bace3e
type: router
router-port: lrp-1ad704fd-fe3d-45dc-8da0-de2c13bace3e
port 95a230dc-355e-4a0f-a480-4aec7876df5a
type: router
router-port: lrp-95a230dc-355e-4a0f-a480-4aec7876df5a
port bb87562e-72b3-433b-9d35-5510753058ec
type: localport
addresses: ["fa:16:3e:98:3e:66"]
port provnet-41728a0a-ee96-4924-b186-2026f6de66fe
type: localnet
addresses: ["unknown"]
switch 0ead1979-c739-443e-b218-7f8f1ad8439c (neutron-08dad4fc-6343-48be-8d36-638eb2fb429e) (aka internal_B)
port d61578bd-b68c-4c60-99f3-de715680bbf3
addresses: ["fa:16:3e:49:8d:19 192.168.4.17"]
port 6023cc1b-1b73-4154-8990-4861eadf080c
addresses: ["fa:16:3e:13:c3:b0 192.168.4.8"]
port 7116fe78-eb78-41dc-933b-cef29df90594
type: localport
addresses: ["fa:16:3e:73:4b:40 192.168.4.2"]
port 28bf9b5d-35a7-471c-a41b-20455b384091
type: router
router-port: lrp-28bf9b5d-35a7-471c-a41b-20455b384091
port 82f37936-50cf-429b-aac2-77cb1f5da84a
addresses: ["fa:16:3e:a5:e8:a2 192.168.4.28"]
switch 44b34927-080d-4611-861a-d78ddf3fed6f (neutron-3512997c-85f4-4abb-aa2a-72f477f6f210) (aka internal_C)
port b1d736ab-ff6d-4420-b29c-b8f2902e4d3b
type: router
router-port: lrp-b1d736ab-ff6d-4420-b29c-b8f2902e4d3b
port 613aa235-b953-4c00-9c28-d9f937c10db4
addresses: ["fa:16:3e:b8:44:03 192.168.5.8"]
port 7dac22b5-fb59-4250-81bb-a7de9abe9572
type: localport
addresses: ["fa:16:3e:3c:42:4c 192.168.5.2"]
switch 9fc44ed7-90a4-4315-bcba-ac59a848d80f (neutron-aea40033-582b-4d0b-b144-05e678d6e9cf) (aka internal_A)
port 8f7b4921-17d9-4aaf-8ab6-2ca789abaed1
addresses: ["fa:16:3e:03:08:a8 192.168.3.5"]
port c65a2216-2a95-46a8-9a39-c9e935912f00
addresses: ["fa:16:3e:e5:01:04 192.168.3.27"]
port 12fb654b-8a0d-4821-8de7-3214b19ba04f
type: router
router-port: lrp-12fb654b-8a0d-4821-8de7-3214b19ba04f
port de34f213-677e-4804-a6a1-4f1ae97a5722
type: localport
addresses: ["fa:16:3e:5e:2f:34 192.168.3.2"]
router 8e2e6395-3d9a-4b34-a66d-75a3d2814702 (neutron-d1b6a1cc-5470-41a0-b3a8-970e6cffa128) (aka routerC)
port lrp-95a230dc-355e-4a0f-a480-4aec7876df5a
mac: "fa:16:3e:1a:b9:41"
networks: ["10.0.0.217/24"]
gateway chassis: [5ddb95b8-4303-45c5-8cf9-cbacffeafc08 3c0ba0c0-b214-40a5-8823-268fbc3aa80e 5d86147f-c008-4029-a7cb-0aa3c1cf5793]
port lrp-b1d736ab-ff6d-4420-b29c-b8f2902e4d3b
mac: "fa:16:3e:f8:d1:f8"
networks: ["192.168.5.1/24"]
nat 84d15a9d-250c-429d-918c-8aa2b06ce12d
external ip: "10.0.0.220"
logical ip: "192.168.5.8"
type: "dnat_and_snat"
nat c010110e-a10a-4815-9a20-3f961a523538
external ip: "10.0.0.217"
logical ip: "192.168.5.0/24"
type: "snat"
router 4dd82062-886b-4aa0-84b8-4b3b8ea4744c (neutron-4face940-f57c-4bb7-b3fe-ea96040cd31b) (aka routerB)
port lrp-140de39c-b91c-4bce-9f80-42c0f9bdb646
mac: "fa:16:3e:47:cc:79"
networks: ["10.0.0.223/24"]
gateway chassis: [5d86147f-c008-4029-a7cb-0aa3c1cf5793 5ddb95b8-4303-45c5-8cf9-cbacffeafc08 3c0ba0c0-b214-40a5-8823-268fbc3aa80e]
port lrp-28bf9b5d-35a7-471c-a41b-20455b384091
mac: "fa:16:3e:3a:7e:fe"
networks: ["192.168.4.1/24"]
nat a68563bd-93c5-485b-90e8-c1e258cb7a82
external ip: "10.0.0.211"
logical ip: "192.168.4.8"
type: "dnat_and_snat"
nat d09e2f63-9d3d-4bea-9cdd-82a1e8fc4af3
external ip: "10.0.0.223"
logical ip: "192.168.4.0/24"
type: "snat"
router 81db4a46-0024-4d39-abb1-2693d21975a4 (neutron-679cead8-e326-4dc1-9c9d-ecfde38b3274) (aka routerA)
port lrp-1ad704fd-fe3d-45dc-8da0-de2c13bace3e
mac: "fa:16:3e:7a:cc:c2"
networks: ["10.0.0.224/24"]
gateway chassis: [3c0ba0c0-b214-40a5-8823-268fbc3aa80e 5ddb95b8-4303-45c5-8cf9-cbacffeafc08 5d86147f-c008-4029-a7cb-0aa3c1cf5793]
port lrp-12fb654b-8a0d-4821-8de7-3214b19ba04f
mac: "fa:16:3e:68:be:79"
networks: ["192.168.3.1/24"]
nat 1e984589-ca55-4c04-9485-98eae836bb5e
external ip: "10.0.0.212"
logical ip: "192.168.3.5"
type: "dnat_and_snat"
nat 365d2cdd-31c2-4efb-8652-3c785e551081
external ip: "10.0.0.224"
logical ip: "192.168.3.0/24"
type: "snat"
Verified on 13.0-RHEL-7/2019-03-01.1 with openvswitch-2.9.0-97.el7fdp.x86_64 and python-networking-ovn-4.0.3-3.el7ost.noarch Used the following scenario: 1. Created internal network internal_A, subnet_A (192.168.1.0/24), router_A. Connected router_A to the external network. Connected internal network internal_A to the router. 2. Created VM1 and connected to the internal_A network. Attached a floating IP (10.0.0.177) to the VM1. 3. Created VM2 and connected to the internal_A network. Attached a floating IP (10.0.0.173) to the VM2. 4. Verified that connectivity works in all directions between floating IPs and external network gateway. 5. Created internal network internal_B, subnet_B (192.168.2.0/24), router_B. Connected router_B to the external network. Connected internal network internal_B to the router. 6. Deleted one of the floating IP that was in use (10.0.0.177) and recreated it again with another uuid but same IP. Used command 'openstack floating ip create --floating-ip-address 10.0.0.177 external' 7. Created VM3 and connected to the internal_B network. Attached the floating IP 10.0.0.177 to it. Verified that arp table on the external network is updated with a new MAC for 10.0.0.177 8. Verified that VM3 is reachable using the assigned floating IP. Verified that connectivity between all servers works using floating IPs. 9. Repeated steps 6-9 with second floating IP and VM4 connected to internal_B network. All worked as expected. |