Bug 1686029

Summary: [3.9] Network Policy Plugin does not clean up flows from deleted namespaces
Product: OpenShift Container Platform Reporter: Dan Winship <danw>
Component: NetworkingAssignee: Dan Winship <danw>
Status: CLOSED WONTFIX QA Contact: zhaozhanqi <zzhao>
Severity: high Docs Contact:
Priority: unspecified    
Version: 3.9.0CC: aos-bugs, zzhao
Target Milestone: ---   
Target Release: 3.9.z   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Cause: Multiple bugs Consequence: When deleting a Namespace that contained NetworkPolicies, some of the OVS rules pertaining to those NetworkPolicies did not get cleaned up. While these rules were harmless in themselves (they wouldn't cause any packets to be mistakenly accepted or rejected), it is possible that the build-up of junk rules might have eventually caused slowdowns. (But note that we are not aware of any customers actually experiencing problems because of this at this time.) Fix: All OVS flows associated with a Namespace should now be deleted properly when that Namespace is deleted. Result: No problems
Story Points: ---
Clone Of: 1682955 Environment:
Last Closed: 2019-03-14 15:54:56 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Comment 1 Dan Winship 2019-03-06 15:08:06 UTC
https://github.com/openshift/ose/pull/1498

Comment 2 zhaozhanqi 2019-03-12 08:50:56 UTC
QE did the following scenarios on version v3.9.72, this issue had been fixed.

1. Create a namespace. Create some pods and NetworkPolicies in that namespace. Confirm with "ovs-ofctl -O OpenFlow13 dump-flows br0 table=80" on a node that there are flows referencing that namespace (via "reg0" in the OVS flow referring to the NetID of the NetNamespace associated with the Namespace). Now delete the namespace and wait for it to disappear from "oc get namespaces". All of the OVS flows referencing that namespace should now be gone.

2. Create a namespace. Create some pods and NetworkPolicies in that namespace. Kill the SDN pods and wait for them to restart. Delete the namespace. Confirm that the OVS flows are deleted just like in the above case.

2. Create a namespace. Create two pods and a "default deny" NetworkPolicy in that namespace and confirm that the pods can't talk to each other. Kill the SDN pods and wait for them to restart. Add a NetworkPolicy to the namespace to allow communication between the two pods. (Don't create any new pods or re-label the existing pods after restarting the SDN; *Only* add a NetworkPolicy.) Confirm that new OVS flows have been created and the pods can talk.

Comment 4 Dan Winship 2019-03-14 15:23:07 UTC
Doh. Ignore previous comment. That was meant for bug 1679270

Comment 5 Dan Winship 2019-03-14 15:54:56 UTC
This is being reverted from 3.11 and earlier. Note that while stale OVS flows may build up at times, there is already a periodic job that eventually cleans them up.