Bug 2099288
| Summary: | [ovn-dvr] dnat_snat traffic becomes centralized during VIP assigned for first time | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux Fast Datapath | Reporter: | Fiorella Yanac <fyanac> |
| Component: | ovn22.03 | Assignee: | Mohammad Heib <mheib> |
| Status: | CLOSED ERRATA | QA Contact: | Jianlin Shi <jishi> |
| Severity: | high | Docs Contact: | |
| Priority: | high | ||
| Version: | FDP 20.A | CC: | apevec, averdagu, bcafarel, ctrautma, dceara, jiji, lhh, majopela, mheib, mlavalle, mmichels, ralonsoh, scohen |
| Target Milestone: | --- | Keywords: | Triaged |
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2022-10-11 12:21:17 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
I had a look at the NB database ovsdb logs and I see:
record 6840: 2022-07-04 21:20:54.420
[...]
table Logical_Switch_Port insert row "1536b0bd-ccde-43b4-93ce-e86d5e797052" (b1d43df5):
name="1536b0bd-ccde-43b4-93ce-e86d5e797052"
addresses=["fa:16:3e:1a:ae:ea 192.168.0.145"]
port_security=["fa:16:3e:1a:ae:ea 192.168.0.145"]
options={mcast_flood_reports="true", requested-chassis=""}
external_ids={"neutron:cidrs"="192.168.0.145/24", "neutron:device_id"="", "neutron:device_owner"="", "neutron:network_name"=neutron-436a7267-25b6-4681-af04-1eb8dce38f0b, "neutron:port_name"="", "neutron:project_id"=b059c55a3d734ad2bf0ac691d0fa824a, "neutron:revision_number"="1", "neutron:security_group_ids"="cb8dee07-0fc9-41db-8d83-8ec3bb0ddf18"}
dhcpv4_options=17e8eadf-850e-4bd6-b37e-93c98ee8ac6b
enabled=true
# At this point the type for port "1536b0bd-ccde-43b4-93ce-e86d5e797052" is "", regular VIF.
# Later this gets turned into a virtual port:
record 6846: 2022-07-04 21:20:55.198
table Logical_Switch_Port row "1536b0bd-ccde-43b4-93ce-e86d5e797052" (b1d43df5) diff:
options={virtual-ip="192.168.0.145", virtual-parents="9b49310a-6a59-49e7-91b4-a1c836ad650a"}
type=virtual
table Logical_Switch_Port row "9b49310a-6a59-49e7-91b4-a1c836ad650a" (4775b19e) diff:
port_security=["fa:16:3e:8f:b0:21 192.168.0.72", "fa:16:3e:8f:b0:21 192.168.0.72 192.168.0.145"]
external_ids={"neutron:revision_number"="5"}
# And even later the virtual parents are set.
record 6850: 2022-07-04 21:20:56.216
[...]
table Logical_Switch_Port row "1536b0bd-ccde-43b4-93ce-e86d5e797052" (b1d43df5) diff:
options={virtual-parents="7bf4442b-9585-4296-8897-36e23307b474,9b49310a-6a59-49e7-91b4-a1c836ad650a"}
Changing from/to VIF/child port(container/virtual) ports is problematic. AFAICT we handled the case for container ports:
https://github.com/ovn-org/ovn/commit/cd3b685043fa9758df3665bf3e3fc972048698a6
But it seems that we need the same fix applied for virtual ports. @mheib what do you think?
Hi, after applying the two patches [1][2] that handle the port type update in northd and updated the 3 controllers in panther23.lab.eng.tlv2.redhat.com setup the test seems to pass now, i tried to re-run it several times and it seems to be okay, @ralonsoh can you please do some more test execution in setup panther23.lab.eng.tlv2.redhat.com and see if you still have the issue or not. [1] https://github.com/ovn-org/ovn/commit/cd3b685043fa9758df3665bf3e3fc972048698a6 [2] https://patchwork.ozlabs.org/project/ovn/patch/20220719133328.2037257-1-mheib@redhat.com/ thanks. ovn22.06 fast-datapath-rhel-8 clone created at https://bugzilla.redhat.com/show_bug.cgi?id=2115334 ovn22.06 fast-datapath-rhel-9 clone created at https://bugzilla.redhat.com/show_bug.cgi?id=2115335 using following script to test patch itself:
systemctl start openvswitch
systemctl start ovn-northd
ovn-nbctl set-connection ptcp:6641
ovn-sbctl set-connection ptcp:6642
ovs-vsctl set open . external_ids:system-id=hv1 external_ids:ovn-remote=tcp:1.1.40.25:6642 external_ids:ovn-encap-type=geneve external_ids:ovn-encap-ip=1.1.40.25
systemctl restart ovn-controller
ovn-nbctl ls-add ls1
ovn-nbctl lsp-add ls1 ls1p1
ovn-nbctl lsp-set-addresses ls1p1 "00:00:00:01:01:01 192.168.1.1 2001::1"
ovn-nbctl lsp-add ls1 ls1p2
ovn-nbctl lsp-set-addresses ls1p2 "00:00:00:01:01:02 192.168.1.2 2001::2"
ovn-nbctl lsp-add ls1 ls1-vir
ovn-nbctl lsp-set-type ls1-vir virtual
ovn-nbctl lsp-set-addresses ls1-vir "00:00:00:01:01:11 192.168.1.11 2001::11"
ovn-nbctl set logical_switch_port ls1-vir options:virtual-ip=192.168.1.11
ovn-nbctl set logical_switch_port ls1-vir options:virtual-parents="ls1p1,ls1p2"
ovn-nbctl lr-add lr1
ovn-nbctl lrp-add lr1 lr1-ls1 00:00:00:00:00:01 192.168.1.254/24 2001::a/64
ovn-nbctl lsp-add ls1 ls1-lr1
ovn-nbctl lsp-set-addresses ls1-lr1 "00:00:00:00:00:01 192.168.1.254 2001::a"
ovn-nbctl lsp-set-type ls1-lr1 router
ovn-nbctl lsp-set-options ls1-lr1 router-port=lr1-ls1
ovs-vsctl add-port br-int ls1p1 -- set interface ls1p1 type=internal external_ids:iface-id=ls1p1
ip netns add ls1p1
ip link set ls1p1 netns ls1p1
ip netns exec ls1p1 ip link set ls1p1 address 00:00:00:01:01:01
ip netns exec ls1p1 ip link set ls1p1 up
ip netns exec ls1p1 ip addr add 192.168.1.1/24 dev ls1p1
ip netns exec ls1p1 ip addr add 2001::1/64 dev ls1p1
ip netns exec ls1p1 ip route add default via 192.168.1.254 dev ls1p1
ip netns exec ls1p1 ip -6 route add default via 2001::a dev ls1p1
ovn-nbctl --wait=hv sync
ovn-sbctl list port_binding ls1-vir
ip netns exec ls1p1 ip addr add 192.168.1.11/24 dev ls1p1
ip netns exec ls1p1 arping -U -I ls1p1 -c 1 192.168.1.11
sleep 2
ovn-sbctl list port_binding ls1-vir
ovn-nbctl set logical_switch_port ls1-vir type=\"\"
ovn-nbctl list logical_switch_port ls1-vir
ovn-nbctl --wait=hv sync
ovn-sbctl list port_binding ls1-vir
reproduced on ovn22.06-27:
[root@dell-per740-69 bz2099288]# rpm -qa | grep -E "openvswitch2.15|ovn22.06"
openvswitch2.15-2.15.0-38.el8fdp.x86_64
ovn22.06-22.06.0-27.el8fdp.x86_64
ovn22.06-host-22.06.0-27.el8fdp.x86_64
ovn22.06-central-22.06.0-27.el8fdp.x86_64
+ ovn-sbctl list port_binding ls1-vir
_uuid : 097cc76e-505b-4b67-b40e-9fb4eebde89f
additional_chassis : []
additional_encap : []
chassis : []
datapath : 09c9e073-674a-457f-baae-b9d232fe2f4b
encap : []
external_ids : {}
gateway_chassis : []
ha_chassis_group : []
logical_port : ls1-vir
mac : ["00:00:00:01:01:11 192.168.1.11 2001::11"]
nat_addresses : []
options : {virtual-ip="192.168.1.11", virtual-parents="ls1p1,ls1p2"}
parent_port : []
port_security : []
requested_additional_chassis: []
requested_chassis : []
tag : []
tunnel_key : 3
type : virtual
up : false
virtual_parent : []
+ ip netns exec ls1p1 ip addr add 192.168.1.11/24 dev ls1p1
+ ip netns exec ls1p1 arping -U -I ls1p1 -c 1 192.168.1.11
ARPING 192.168.1.11 from 192.168.1.11 ls1p1
Sent 1 probes (1 broadcast(s))
Received 0 response(s)
+ sleep 2
+ ovn-sbctl list port_binding ls1-vir
_uuid : 097cc76e-505b-4b67-b40e-9fb4eebde89f
additional_chassis : []
additional_encap : []
chassis : b84f8aec-8ab6-427c-ac1b-05ef3dacd7a8
datapath : 09c9e073-674a-457f-baae-b9d232fe2f4b
encap : []
external_ids : {}
gateway_chassis : []
ha_chassis_group : []
logical_port : ls1-vir
mac : ["00:00:00:01:01:11 192.168.1.11 2001::11"]
nat_addresses : []
options : {virtual-ip="192.168.1.11", virtual-parents="ls1p1,ls1p2"}
parent_port : []
port_security : []
requested_additional_chassis: []
requested_chassis : []
tag : []
tunnel_key : 3
type : virtual
up : true
virtual_parent : ls1p1
+ ovn-nbctl set logical_switch_port ls1-vir 'type=""'
+ ovn-nbctl list logical_switch_port ls1-vir
_uuid : c1a3aa9b-97d3-414c-bd34-c00df8bf6b09
addresses : ["00:00:00:01:01:11 192.168.1.11 2001::11"]
dhcpv4_options : []
dhcpv6_options : []
dynamic_addresses : []
enabled : []
external_ids : {}
ha_chassis_group : []
name : ls1-vir
options : {virtual-ip="192.168.1.11", virtual-parents="ls1p1,ls1p2"}
parent_name : []
port_security : []
tag : []
tag_request : []
type : ""
up : false
+ ovn-nbctl --wait=hv sync
+ ovn-sbctl list port_binding ls1-vir
_uuid : 097cc76e-505b-4b67-b40e-9fb4eebde89f
additional_chassis : []
additional_encap : []
chassis : []
datapath : 09c9e073-674a-457f-baae-b9d232fe2f4b
encap : []
external_ids : {}
gateway_chassis : []
ha_chassis_group : []
logical_port : ls1-vir
mac : ["00:00:00:01:01:11 192.168.1.11 2001::11"]
nat_addresses : []
options : {virtual-ip="192.168.1.11", virtual-parents="ls1p1,ls1p2"}
parent_port : []
port_security : []
requested_additional_chassis: []
requested_chassis : []
tag : []
tunnel_key : 3
type : ""
up : true
<=== up is still true even change the type to "" for virtual port
virtual_parent : []
Verified on ovn22.06-57:
[root@dell-per740-69 bz2099288]# rpm -qa | grep -E "openvswitch2.15|ovn22.06"
openvswitch2.15-2.15.0-38.el8fdp.x86_64
ovn22.06-22.06.0-57.el8fdp.x86_64
ovn22.06-central-22.06.0-57.el8fdp.x86_64
ovn22.06-host-22.06.0-57.el8fdp.x86_64
+ ip netns exec ls1p1 arping -U -I ls1p1 -c 1 192.168.1.11
ARPING 192.168.1.11 from 192.168.1.11 ls1p1
Sent 1 probes (1 broadcast(s))
Received 0 response(s)
+ sleep 2
+ ovn-sbctl list port_binding ls1-vir
_uuid : f1c2bef7-84eb-4969-aad5-272474850d5f
additional_chassis : []
additional_encap : []
chassis : 1689782a-1e3f-47cc-9154-b69d5f2cfab5
datapath : 1189c03f-43f4-4711-b8d5-ee1d4ec44783
encap : []
external_ids : {}
gateway_chassis : []
ha_chassis_group : []
logical_port : ls1-vir
mac : ["00:00:00:01:01:11 192.168.1.11 2001::11"]
nat_addresses : []
options : {virtual-ip="192.168.1.11", virtual-parents="ls1p1,ls1p2"}
parent_port : []
port_security : []
requested_additional_chassis: []
requested_chassis : []
tag : []
tunnel_key : 3
type : virtual
up : true
virtual_parent : ls1p1
+ ovn-nbctl set logical_switch_port ls1-vir 'type=""'
+ ovn-nbctl list logical_switch_port ls1-vir
_uuid : 5632af16-2bb7-4e5a-9ee2-a7a60cf37400
addresses : ["00:00:00:01:01:11 192.168.1.11 2001::11"]
dhcpv4_options : []
dhcpv6_options : []
dynamic_addresses : []
enabled : []
external_ids : {}
ha_chassis_group : []
name : ls1-vir
options : {virtual-ip="192.168.1.11", virtual-parents="ls1p1,ls1p2"}
parent_name : []
port_security : []
tag : []
tag_request : []
type : ""
up : false
+ ovn-nbctl --wait=hv sync
+ ovn-sbctl list port_binding ls1-vir
_uuid : efefafbb-1ca4-4a99-9afe-d9125457be3f
additional_chassis : []
additional_encap : []
chassis : []
datapath : 1189c03f-43f4-4711-b8d5-ee1d4ec44783
encap : []
external_ids : {}
gateway_chassis : []
ha_chassis_group : []
logical_port : ls1-vir
mac : ["00:00:00:01:01:11 192.168.1.11 2001::11"]
nat_addresses : []
options : {virtual-ip="192.168.1.11", virtual-parents="ls1p1,ls1p2"}
parent_port : []
port_security : []
requested_additional_chassis: []
requested_chassis : []
tag : []
tunnel_key : 5
type : ""
up : false
virtual_parent : []
reproduced on ovn22.03-22.03.0-69: [root@dell-per740-70 bz2099288]# rpm -qa | grep -E "openvswitch2.15|ovn22.03" openvswitch2.15-2.15.0-38.el8fdp.x86_64 ovn22.03-22.03.0-69.el8fdp.x86_64 ovn22.03-host-22.03.0-69.el8fdp.x86_64 ovn22.03-central-22.03.0-69.el8fdp.x86_64 + ovn-sbctl list port_binding ls1-vir _uuid : c82fcc5c-90f9-41be-846a-b94dc1de0bab chassis : [] datapath : 3f38a9a2-35b6-4a56-8c9f-01f04d7ccf76 encap : [] external_ids : {} gateway_chassis : [] ha_chassis_group : [] logical_port : ls1-vir mac : ["00:00:00:01:01:11 192.168.1.11 2001::11"] nat_addresses : [] options : {virtual-ip="192.168.1.11", virtual-parents="ls1p1,ls1p2"} parent_port : [] requested_chassis : [] tag : [] tunnel_key : 3 type : virtual up : false virtual_parent : [] + ip netns exec ls1p1 ip addr add 192.168.1.11/24 dev ls1p1 + ip netns exec ls1p1 arping -U -I ls1p1 -c 1 192.168.1.11 ARPING 192.168.1.11 from 192.168.1.11 ls1p1 Sent 1 probes (1 broadcast(s)) Received 0 response(s) + sleep 2 + ovn-sbctl list port_binding ls1-vir _uuid : c82fcc5c-90f9-41be-846a-b94dc1de0bab chassis : b4fb110d-914f-40bc-a3fb-1ce27672f5cf datapath : 3f38a9a2-35b6-4a56-8c9f-01f04d7ccf76 encap : [] external_ids : {} gateway_chassis : [] ha_chassis_group : [] logical_port : ls1-vir mac : ["00:00:00:01:01:11 192.168.1.11 2001::11"] nat_addresses : [] options : {virtual-ip="192.168.1.11", virtual-parents="ls1p1,ls1p2"} parent_port : [] requested_chassis : [] tag : [] tunnel_key : 3 type : virtual up : true virtual_parent : ls1p1 + ovn-nbctl set logical_switch_port ls1-vir 'type=""' + ovn-nbctl list logical_switch_port ls1-vir _uuid : f7fa2113-d65e-4f46-a030-88d4d420a1cd addresses : ["00:00:00:01:01:11 192.168.1.11 2001::11"] dhcpv4_options : [] dhcpv6_options : [] dynamic_addresses : [] enabled : [] external_ids : {} ha_chassis_group : [] name : ls1-vir options : {virtual-ip="192.168.1.11", virtual-parents="ls1p1,ls1p2"} parent_name : [] port_security : [] tag : [] tag_request : [] type : "" up : false + ovn-nbctl --wait=hv sync + ovn-sbctl list port_binding ls1-vir _uuid : c82fcc5c-90f9-41be-846a-b94dc1de0bab chassis : [] datapath : 3f38a9a2-35b6-4a56-8c9f-01f04d7ccf76 encap : [] external_ids : {} gateway_chassis : [] ha_chassis_group : [] logical_port : ls1-vir mac : ["00:00:00:01:01:11 192.168.1.11 2001::11"] nat_addresses : [] options : {virtual-ip="192.168.1.11", virtual-parents="ls1p1,ls1p2"} parent_port : [] requested_chassis : [] tag : [] tunnel_key : 3 type : "" up : true <== up is still true virtual_parent : [] Verified on ovn22.03-22.03.0-95: [root@dell-per740-70 bz2099288]# rpm -qa | grep -E "openvswitch2.15|ovn22.03" openvswitch2.15-2.15.0-38.el8fdp.x86_64 ovn22.03-22.03.0-95.el8fdp.x86_64 ovn22.03-central-22.03.0-95.el8fdp.x86_64 ovn22.03-host-22.03.0-95.el8fdp.x86_64 + ovn-sbctl list port_binding ls1-vir _uuid : 7995b874-dc5a-496b-a228-a71d583e7143 chassis : [] datapath : 34978aaf-e205-4c15-847a-7a8a7aadc803 encap : [] external_ids : {} gateway_chassis : [] ha_chassis_group : [] logical_port : ls1-vir mac : ["00:00:00:01:01:11 192.168.1.11 2001::11"] nat_addresses : [] options : {virtual-ip="192.168.1.11", virtual-parents="ls1p1,ls1p2"} parent_port : [] requested_chassis : [] tag : [] tunnel_key : 3 type : virtual up : false virtual_parent : [] + ip netns exec ls1p1 ip addr add 192.168.1.11/24 dev ls1p1 + ip netns exec ls1p1 arping -U -I ls1p1 -c 1 192.168.1.11 ARPING 192.168.1.11 from 192.168.1.11 ls1p1 Sent 1 probes (1 broadcast(s)) Received 0 response(s) + sleep 2 + ovn-sbctl list port_binding ls1-vir _uuid : 7995b874-dc5a-496b-a228-a71d583e7143 chassis : c0e1efe3-befc-4926-ac20-2a2c2d23f010 datapath : 34978aaf-e205-4c15-847a-7a8a7aadc803 encap : [] external_ids : {} gateway_chassis : [] ha_chassis_group : [] logical_port : ls1-vir mac : ["00:00:00:01:01:11 192.168.1.11 2001::11"] nat_addresses : [] options : {virtual-ip="192.168.1.11", virtual-parents="ls1p1,ls1p2"} parent_port : [] requested_chassis : [] tag : [] tunnel_key : 3 type : virtual up : true virtual_parent : ls1p1 + ovn-nbctl set logical_switch_port ls1-vir 'type=""' + ovn-nbctl list logical_switch_port ls1-vir _uuid : 8f294abf-9c15-4106-993e-c533c01b9814 addresses : ["00:00:00:01:01:11 192.168.1.11 2001::11"] dhcpv4_options : [] dhcpv6_options : [] dynamic_addresses : [] enabled : [] external_ids : {} ha_chassis_group : [] name : ls1-vir options : {virtual-ip="192.168.1.11", virtual-parents="ls1p1,ls1p2"} parent_name : [] port_security : [] tag : [] tag_request : [] type : "" up : false + ovn-nbctl --wait=hv sync + ovn-sbctl list port_binding ls1-vir _uuid : 9c02d2fd-8573-4c24-996b-161c006262b0 chassis : [] datapath : 34978aaf-e205-4c15-847a-7a8a7aadc803 encap : [] external_ids : {} gateway_chassis : [] ha_chassis_group : [] logical_port : ls1-vir mac : ["00:00:00:01:01:11 192.168.1.11 2001::11"] nat_addresses : [] options : {virtual-ip="192.168.1.11", virtual-parents="ls1p1,ls1p2"} parent_port : [] requested_chassis : [] tag : [] tunnel_key : 5 type : "" up : false <=== up is false virtual_parent : [] set VERIFIED per comment 23 Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (ovn22.03 bug fix and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2022:6870 |
Description of problem: The test neutron_plugin.tests.scenario.test_dvr_ovn.OvnDvrTest.test_dvr_vip_failover [1] is failing in low frequency (4/600). When de VIP_FIP is assigned for the first time to a VM, the mac address is the router-gw MAC instead of the VIP-FIP MAC itself. The drop flow rule with priority 80 is not added in "ovn-sbctl lflow-list " [root@controller-1 ~]# ovn-sbctl lflow-list tempest-router-571443429 | grep lr_in_gw_redirect table=18(lr_in_gw_redirect ), priority=100 , match=(ip4.src == 192.168.0.192 && outport == "lrp-68d50fa4-990f-4e73-9bc5-89343a958912" && is_chassis_resident("eb6a1352-a7c2-4a75-b78a-db009640f15e")), action=(eth.src = fa:16:3e:d1:b9:45; reg1 = 10.0.0.185; next;) table=18(lr_in_gw_redirect ), priority=100 , match=(ip4.src == 192.168.0.226 && outport == "lrp-68d50fa4-990f-4e73-9bc5-89343a958912" && is_chassis_resident("4ee1cb83-56af-4726-8408-7db201324ebe")), action=(eth.src = fa:16:3e:ec:1c:bd; reg1 = 10.0.0.182; next;) table=18(lr_in_gw_redirect ), priority=100 , match=(ip4.src == 192.168.0.52 && outport == "lrp-68d50fa4-990f-4e73-9bc5-89343a958912" && is_chassis_resident("72767d62-1438-42b5-8c09-e1a4fb50ed03")), action=(eth.src = fa:16:3e:75:7c:9f; reg1 = 10.0.0.216; next;) table=18(lr_in_gw_redirect ), priority=80 , match=(ip4.src == 192.168.0.226 && outport == "lrp-68d50fa4-990f-4e73-9bc5-89343a958912"), action=(drop;) <-----------------------------------------------This rule is not shows when the test is fail table=18(lr_in_gw_redirect ), priority=50 , match=(outport == "lrp-68d50fa4-990f-4e73-9bc5-89343a958912"), action=(outport = "cr-lrp-68d50fa4-990f-4e73-9bc5-89343a958912"; next;) table=18(lr_in_gw_redirect ), priority=0 , match=(1), action=(next;) The traffic is not distributed, is centralized despite enabling DVR. However, after doing the VIP failover the problem is solved (also the drop flow rule is added) The problem is similar as this bug: https://bugzilla.redhat.com/show_bug.cgi?id=2083527. This create as a new bug because the fail is not always reproduce. Version: RHOS-16.2-RHEL-8-20220603.n.1 python3-networking-ovn-7.4.2-2.20220409154849.el8ost [heat-admin@controller-0 ~]$ sudo podman exec -it ovn_controller rpm -qa | grep ovn rhosp-ovn-host-2021-4.el8ost.1.noarch ovn-2021-21.12.0-46.el8fdp.x86_64 rhosp-ovn-2021-4.el8ost.1.noarch ovn-2021-host-21.12.0-46.el8fdp.x86_64 Reproduce: - Assigned VIP_FIP to VM. - Establish connection. - Capture the traffic on controller and computes. [1] https://code.engineering.redhat.com/gerrit/plugins/gitiles/rhos-qe-tests/tempest_neutron_plugin/+/dcefdfec23003790cb4a37ae56f07f7614dc98f0/neutron_plugin/tests/scenario/test_dvr_ovn.py#614