Bug 1799401

Summary: Failure updating security group rules from network policy
Product: OpenShift Container Platform Reporter: Maysa Macedo <mdemaced>
Component: NetworkingAssignee: Maysa Macedo <mdemaced>
Networking sub component: kuryr QA Contact: Jon Uriarte <juriarte>
Status: CLOSED ERRATA Docs Contact:
Severity: medium    
Priority: unspecified CC: gcheresh, juriarte, ltomasbo
Version: 4.4   
Target Milestone: ---   
Target Release: 4.3.z   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1795631 Environment:
Last Closed: 2020-03-10 23:53:27 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1795631    
Bug Blocks:    

Description Maysa Macedo 2020-02-06 16:58:45 UTC 
+++ This bug was initially created as a clone of Bug #1795631 +++

Description of problem:

When running K8s NP tests in parallel (in different terminal), the Kuryr controller restarts due to the next error:

(
kubetest --provider=local  --check-version-skew=false --test --test_args="--ginkgo.focus=\[Feature:NetworkPolicy-0 --host=https://api.ostest.shiftstack.com:6443"
kubetest --provider=local  --check-version-skew=false --test --test_args="--ginkgo.focus=\[Feature:NetworkPolicy-1 --host=https://api.ostest.shiftstack.com:6443"
)

ERROR kuryr_kubernetes.handlers.logging [-] Failed to handle event {'type': 'ADDED', 'object': {'kind': 'NetworkPolicy', 'apiVersion': 'networking.k8s.io/v1', 'metadata': {'name': 'allow-from-client-b-pod-selector', 'namespace': 'network-policy-9108', 'selfLink': '/apis/networking.k8s.io/v1/namespaces/network-policy-9108/networkpolicies/allow-from-client-b-pod-selector', 'uid': '3f9a4598-4c48-44ed-8ff8-fb8ac56622b5', 'resourceVersion': '656984', 'generation': 1, 'creationTimestamp': '2020-01-28T10:54:23Z', 'annotations': {'kuryrnetpolicy_selfLink': '/apis/openstack.org/v1/namespaces/network-policy-9108/kuryrnetpolicies/np-allow-from-client-b-pod-selector'}}, 'spec': {'podSelector': {'matchLabels': {'pod-name': 'server'}}, 'ingress': [{'from': [{'podSelector': {'matchLabels': {'pod-name': 'client-b'}}}]}], 'policyTypes': ['Ingress']}}}: kuryr_kubernetes.exceptions.K8sClientException: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"KuryrNetPolicy.openstack.org \"np-allow-from-client-b-pod-selector\" is invalid: spec.ingressSgRules.security_group_rule.id: Required value","reason":"Invalid","details":{"name":"np-allow-from-client-b-pod-selector","group":"openstack.org","kind":"KuryrNetPolicy","causes":[{"reason":"FieldValueRequired","message":"Required value","field":"spec.ingressSgRules.security_group_rule.id"}]},"code":422}
ERROR kuryr_kubernetes.handlers.logging Traceback (most recent call last):
ERROR kuryr_kubernetes.handlers.logging   File "/usr/lib/python3.6/site-packages/kuryr_kubernetes/handlers/logging.py", line 37, in __call__
ERROR kuryr_kubernetes.handlers.logging     self._handler(event)
ERROR kuryr_kubernetes.handlers.logging   File "/usr/lib/python3.6/site-packages/kuryr_kubernetes/handlers/retry.py", line 78, in __call__
ERROR kuryr_kubernetes.handlers.logging     self._handler(event)
ERROR kuryr_kubernetes.handlers.logging   File "/usr/lib/python3.6/site-packages/kuryr_kubernetes/handlers/k8s_base.py", line 75, in __call__
ERROR kuryr_kubernetes.handlers.logging     self.on_present(obj)
ERROR kuryr_kubernetes.handlers.logging   File "/usr/lib/python3.6/site-packages/kuryr_kubernetes/controller/handlers/policy.py", line 70, in on_present
ERROR kuryr_kubernetes.handlers.logging     project_id)
ERROR kuryr_kubernetes.handlers.logging   File "/usr/lib/python3.6/site-packages/kuryr_kubernetes/controller/drivers/network_policy.py", line 59, in ensure_network_policy
ERROR kuryr_kubernetes.handlers.logging     self.update_security_group_rules_from_network_policy(policy))
ERROR kuryr_kubernetes.handlers.logging   File "/usr/lib/python3.6/site-packages/kuryr_kubernetes/controller/drivers/network_policy.py", line 123, in update_security_group_rules_from_network_policy
     np_spec=policy['spec'])
   File "/usr/lib/python3.6/site-packages/kuryr_kubernetes/controller/drivers/utils.py", line 245, in patch_kuryrnetworkpolicy_crd
     'networkpolicy_spec': np_spec})
   File "/usr/lib/python3.6/site-packages/kuryr_kubernetes/k8s_client.py", line 137, in patch_crd
     raise exc.K8sClientException(response.text)
 kuryr_kubernetes.exceptions.K8sClientException: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"KuryrNetPolicy.openstack.org \"np-allow-from-client-b-pod-selector\" is invalid: spec.ingressSgRules.security_group_rule.id: Required value","reason":"Invalid","details":{"name":"np-allow-from-client-b-pod-selector","group":"openstack.org","kind":"KuryrNetPolicy","causes":[{"reason":"FieldValueRequired","message":"Required value","field":"spec.ingressSgRules.security_group_rule.id"}]},"code":422}


Version-Release number of selected component (if applicable):

OCP: 4.4.0-0.nightly-2020-01-24-141203
OSP: RHOS_TRUNK-16.0-RHEL-8-20191224.n.0

Using Octavia amphora driver.

$ openstack loadbalancer provider list
+---------+-------------------------------------------------+
| name    | description                                     |
+---------+-------------------------------------------------+
| amphora | The Octavia Amphora driver.                     |
| octavia | Deprecated alias of the Octavia Amphora driver. |
+---------+-------------------------------------------------+
Comment 3 Jon Uriarte 2020-02-25 09:11:51 UTC 
Verified in 4.3.0-0.nightly-2020-02-20-235803 on top of OSP 16 RHOS_TRUNK-16.0-RHEL-8-20200220.n.0 compose.

After running K8s NP tests in parallel, the Kuryr controller doesn't show the next error:

ERROR kuryr_kubernetes.handlers.logging [-] Failed to handle event {'type': 'ADDED', 'object': {'kind': 'NetworkPolicy', 'apiVersion': 'networking.k8s.io/v1', 'metadata': {'name': 'allow-from-client-b-pod-selector', 'namespace': 'network-policy-9108', 'selfLink': '/apis/networking.k8s.io/v1/namespaces/network-policy-9108/networkpolicies/allow-from-client-b-pod-selector', 'uid': '3f9a4598-4c48-44ed-8ff8-fb8ac56622b5', 'resourceVersion': '656984', 'generation': 1, 'creationTimestamp': '2020-01-28T10:54:23Z', 'annotations': {'kuryrnetpolicy_selfLink': '/apis/openstack.org/v1/namespaces/network-policy-9108/kuryrnetpolicies/np-allow-from-client-b-pod-selector'}}, 'spec': {'podSelector': {'matchLabels': {'pod-name': 'server'}}, 'ingress': [{'from': [{'podSelector': {'matchLabels': {'pod-name': 'client-b'}}}]}], 'policyTypes': ['Ingress']}}}: kuryr_kubernetes.exceptions.K8sClientException: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"KuryrNetPolicy.openstack.org \"np-allow-from-client-b-pod-selector\" is invalid: spec.ingressSgRules.security_group_rule.id: Required value","reason":"Invalid","details":{"name":"np-allow-from-client-b-pod-selector","group":"openstack.org","kind":"KuryrNetPolicy","causes":[{"reason":"FieldValueRequired","message":"Required value","field":"spec.ingressSgRules.security_group_rule.id"}]},"code":422}
Comment 5 errata-xmlrpc 2020-03-10 23:53:27 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0676