Red Hat Bugzilla – Bug 1515815
[Netvirt][NAT] SNAT flows are not removed after removing an external interface of a router
Last modified: 2018-10-18 03:22:34 EDT
Description of problem: After removing the external interface of a router no SNAT flows are removed. After removing of a router there are still flows on a compute node with the IP of the router. An example of flows: cookie=0x8000003, duration=6586.136s, table=21, n_packets=0, n_bytes=0, priority=42,ip,metadata=0x324b0/0xfffffe,nw_dst=10.0.0.214 actions=goto_table:25 cookie=0x122201d9, duration=205.588s, table=81, n_packets=0, n_bytes=0, priority=100,arp,metadata=0x4157e000000/0xfffffffff000000,arp_tpa=10.0.0.214,arp_op=1 actions=move:NXM_OF_ETH_SRC[]>NXM_OF_ETH_DST[],set_field:fa:16:3e:9a:c9:20>eth_src,load:0x2->NXM_OF_ARP_OP[],move:NXM_NX_ARP_SHA[]>NXM_NX_ARP_THA[],move:NXM_OF_ARP_SPA[]>NXM_OF_ARP_TPA[],load:0xfa163e9ac920->NXM_NX_ARP_SHA[],load:0xa0000d6->NXM_OF_ARP_SPA[],load:0->NXM_OF_IN_PORT[],load:0x400->NXM_NX_REG6[],write_metadata:0/0x1,goto_table:220 Version-Release number of selected component (if applicable): Carbon opendaylight-6.2.0-4.el7ost.noarch How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Expected results: There should be no flows after removing an external interface from the router Additional info: u/s bug - https://jira.opendaylight.org/browse/NETVIRT-1020
(In reply to Itzik Brown from comment #0) > Description of problem: > After removing the external interface of a router no SNAT flows are removed. > After removing of a router there are still flows on a compute node with the > IP of the router. > > An example of flows: > cookie=0x8000003, duration=6586.136s, table=21, n_packets=0, n_bytes=0, > priority=42,ip,metadata=0x324b0/0xfffffe,nw_dst=10.0.0.214 > actions=goto_table:25 > > cookie=0x122201d9, duration=205.588s, table=81, n_packets=0, n_bytes=0, > priority=100,arp,metadata=0x4157e000000/0xfffffffff000000,arp_tpa=10.0.0.214, > arp_op=1 > actions=move:NXM_OF_ETH_SRC[]>NXM_OF_ETH_DST[],set_field:fa:16:3e:9a:c9: > 20>eth_src,load:0x2->NXM_OF_ARP_OP[],move:NXM_NX_ARP_SHA[]>NXM_NX_ARP_THA[], > move:NXM_OF_ARP_SPA[]>NXM_OF_ARP_TPA[],load:0xfa163e9ac920->NXM_NX_ARP_SHA[], > load:0xa0000d6->NXM_OF_ARP_SPA[],load:0->NXM_OF_IN_PORT[],load:0x400- > >NXM_NX_REG6[],write_metadata:0/0x1,goto_table:220 > > Version-Release number of selected component (if applicable): > Carbon opendaylight-6.2.0-4.el7ost.noarch > > How reproducible: > > > Steps to Reproduce: > 1. > 2. > 3. > > Actual results: > > > Expected results: > There should be no flows after removing an external interface from the router > > Additional info: > u/s bug - https://jira.opendaylight.org/browse/NETVIRT-1020 The flow to "nw_dst=10.0.0.214 actions=goto_table:25" is flow from FIP. So seems to be an issue related to FIP. Do you have any steps to reproduce this ? Was this a result of tempest test? I tried creating a fip in vms across computes and tried multiple scenario and didn't observe this issue.
Currently after removing the External network these are the flows: # ovs-ofctl -O OpenFlow13 dump-flows br-int |grep 10.0.0 cookie=0x8000000, duration=85485.560s, table=0, n_packets=756488, n_bytes=84687076, priority=4,in_port=1,vlan_tci=0x0000/0x1fff actions=write_metadata:0x110000000001/0xffffff0000000001,goto_table:17 cookie=0x8000001, duration=776.462s, table=17, n_packets=633, n_bytes=70526, priority=10,metadata=0x110000000000/0xffffff0000000000 actions=load:0x1926a->NXM_NX_REG3[0..24],write_metadata:0x90001100000324d4/0xfffffffffffffffe,goto_table:19 cookie=0x8040000, duration=776.462s, table=17, n_packets=629, n_bytes=70078, priority=10,metadata=0x9000110000000000/0xffffff0000000000 actions=load:0x11->NXM_NX_REG1[0..19],load:0x1388->NXM_NX_REG7[0..15],write_metadata:0xa000111388000000/0xfffffffffffffffe,goto_table:43 cookie=0x1080000, duration=85470.515s, table=19, n_packets=759100, n_bytes=85337506, priority=0 actions=resubmit(,17) cookie=0x1030000, duration=85470.515s, table=20, n_packets=0, n_bytes=0, priority=0 actions=goto_table:80 cookie=0x8000003, duration=746.867s, table=21, n_packets=0, n_bytes=0, priority=42,ip,metadata=0x324da/0xfffffe,nw_dst=10.0.0.1 actions=set_field:52:54:00:67:51:ed->eth_dst,load:0x1100->NXM_NX_REG6[],resubmit(,220) cookie=0x8000003, duration=85487.369s, table=21, n_packets=0, n_bytes=0, priority=34,ip,metadata=0x33c22/0xfffffe,nw_dst=10.0.0.0/24 actions=write_metadata:0x1770033c22/0xfffffffffe,goto_table:22 cookie=0x8000003, duration=768.846s, table=21, n_packets=0, n_bytes=0, priority=34,ip,metadata=0x324da/0xfffffe,nw_dst=10.0.0.0/24 actions=write_metadata:0x13880324da/0xfffffffffe,goto_table:22 cookie=0x8000004, duration=85487.369s, table=22, n_packets=0, n_bytes=0, priority=42,ip,metadata=0x33c22/0xfffffe,nw_dst=10.0.0.255 actions=drop cookie=0x8000004, duration=768.846s, table=22, n_packets=0, n_bytes=0, priority=42,ip,metadata=0x324da/0xfffffe,nw_dst=10.0.0.255 actions=drop cookie=0x1080000, duration=85470.515s, table=23, n_packets=0, n_bytes=0, priority=0 actions=resubmit(,17) cookie=0x8800011, duration=776.451s, table=55, n_packets=0, n_bytes=0, priority=10,tun_id=0x11,metadata=0x110000000000/0xfffff0000000000 actions=drop cookie=0x1030000, duration=85470.513s, table=80, n_packets=0, n_bytes=0, priority=0 actions=resubmit(,17) After talking with Aswin it should be fixed in 6.2.0-5
Fixed in version 6.2.0-5
There are stale flows as reported in the u/s bug. Checked with opendaylight-8.0.0-2.el7ost.noarch
*** Bug 1558523 has been marked as a duplicate of this bug. ***
Aswin, any progress on this?
[1] solves some of them. But still there are occasional flows in some tables. Which I am working on. [1]https://git.opendaylight.org/gerrit/#/c/70375/
Aswin, Any update on this? I see the partial fix was merged a while ago, is there anything else still needed to fix this?
There is one more patch, which should solve this bug. https://git.opendaylight.org/gerrit/#/c/71495/
Based on discussion with Aswin, seems these stale flows aren't causing functional failures, so lowering the priority of the bug.
Verified with the following scenario (see scenario output attachments). # Create a cirros Image. # Open security group rules for ICMP and SSH. # Create an external network and a subnet. # Create a router and attach an interface. # Create a tenant network. # Attach the tenant network to the router. # Create a floating IP. # Launch an instance and associate a Floating IP to the instance. # Check connectivity. # Check OVS SNAT flows - with Network object already created (see snat output 1). # Delete Network, Subnet, Router, Ports and Floating IP. # Check OVS SNAT flows on each controller - After Network objects were removed (see snat output 2) # Re-create Network, Subnet, Router and Floating IP, and check connectivity. # Check OVS SNAT flows on each controller - After Network objects were re-created (see snat output 3).
Created attachment 1458382 [details] ODL Check SNAT flows scenario (output is record since the first snat check, after objects were already created).
Created attachment 1458383 [details] 1) check snat with pre-created network objects
Created attachment 1458384 [details] 2) check snat after removing network objects
Created attachment 1458385 [details] 3) check snat after network objects were created again
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2018:2215