Bug 1856830

Summary: NP CRD unable to be patched because of missing sg rule ID
Product: OpenShift Container Platform Reporter: Maysa Macedo <mdemaced>
Component: NetworkingAssignee: Maysa Macedo <mdemaced>
Networking sub component: kuryr QA Contact: GenadiC <gcheresh>
Status: CLOSED ERRATA Docs Contact:
Severity: medium    
Priority: unspecified CC: juriarte
Version: 4.6   
Target Milestone: ---   
Target Release: 4.6.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1893996 (view as bug list) Environment:
Last Closed: 2020-10-27 16:14:36 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1893996    

Description Maysa Macedo 2020-07-14 14:24:18 UTC
Description of problem:

During the Network Policy creation it's possible that the CRD is patched with repeated sg rules, which is not allowed, resulting in validation error as the repeated sg rules will not have the sg rule id.

2020-07-09 16:44:09.964 1 DEBUG kuryr_kubernetes.controller.drivers.utils [-] Patching KuryrNetPolicy CRD np-allow-to-server-a-pod-selector patch_kuryrnetworkpolicy_crd /usr/local/lib/python3.6/site-packages/kuryr_kubernetes/controller/drivers/utils.py:221[00m
2020-07-09 16:44:09.964 1 DEBUG kuryr_kubernetes.k8s_client [-] Patch /apis/openstack.org/v1/namespaces/network-policy-6479/kuryrnetpolicies/np-allow-to-server-a-pod-selector: [{'op': 'replace', 'path': '/spec/ingressSgRules', 'value': [{'security_group_rule': {'ethertype': 'IPv4', 'security_group_id': '718bef93-c6d9-4e52-a081-9d7a703ab3b4', 'direction': 'ingress', 'description': 'Kuryr-Kubernetes NetPolicy SG rule', 'id': '79c924de-0983-4b60-8f92-7beb1cac17cd'}}, {'security_group_rule': {'ethertype': 'IPv6', 'security_group_id': '718bef93-c6d9-4e52-a081-9d7a703ab3b4', 'direction': 'ingress', 'description': 'Kuryr-Kubernetes NetPolicy SG rule', 'id': '703b7159-d3f7-409e-b19d-c44eba3b1201'}}]}, {'op': 'replace', 'path': '/spec/egressSgRules', 'value': [{'security_group_rule': {'ethertype': 'IPv4', 'security_group_id': '718bef93-c6d9-4e52-a081-9d7a703ab3b4', 'description': 'Kuryr-Kubernetes NetPolicy SG rule', 'direction': 'egress', 'protocol': 'udp', 'port_range_min': 53, 'port_range_max': 53, 'id': '14eedd8d-3a67-4828-be1f-64b8d5220ea9'}}, {'security_group_rule': {'ethertype': 'IPv4', 'security_group_id': '718bef93-c6d9-4e52-a081-9d7a703ab3b4', 'description': 'Kuryr-Kubernetes NetPolicy SG rule', 'direction': 'egress', 'protocol': 'udp', 'port_range_min': 53, 'port_range_max': 53, 'remote_ip_prefix': '10.1.0.128/26'}}, {'security_group_rule': {'ethertype': 'IPv6', 'security_group_id': '718bef93-c6d9-4e52-a081-9d7a703ab3b4', 'description': 'Kuryr-Kubernetes NetPolicy SG rule', 'direction': 'egress', 'protocol': 'udp', 'port_range_min': 53, 'port_range_max': 53, 'id': '3616868c-c52d-4fc6-a596-9197ecea3cba'}}, {'security_group_rule': {'ethertype': 'IPv4', 'security_group_id': '718bef93-c6d9-4e52-a081-9d7a703ab3b4', 'description': 'Kuryr-Kubernetes NetPolicy SG rule', 'direction': 'egress', 'protocol': 'udp', 'port_range_min': 53, 'port_range_max': 53, 'remote_ip_prefix': '10.1.0.128/26'}}, {'security_group_rule': {'ethertype': 'IPv4', 'security_group_id': '718bef93-c6d9-4e52-a081-9d7a703ab3b4', 'description': 'Kuryr-Kubernetes NetPolicy SG rule', 'direction': 'egress', 'protocol': 'tcp', 'port_range_min': 1, 'port_range_max': 65535, 'remote_ip_prefix': '10.1.2.244', 'id': 'f553e38e-6240-46d9-b3b1-03e071d2340d'}, 'namespace': 'network-policy-6479'}, {'security_group_rule': {'ethertype': 'IPv4', 'security_group_id': '718bef93-c6d9-4e52-a081-9d7a703ab3b4', 'description': 'Kuryr-Kubernetes NetPolicy SG rule', 'direction': 'egress', 'protocol': 'tcp', 'port_range_min': 1, 'port_range_max': 65535, 'remote_ip_prefix': '10.1.0.150', 'id': 'c92a7d7e-0a96-4e5d-a572-74b9fbed3ec3'}}, {'security_group_rule': {'ethertype': 'IPv4', 'security_group_id': '718bef93-c6d9-4e52-a081-9d7a703ab3b4', 'description': 'Kuryr-Kubernetes NetPolicy SG rule', 'direction': 'egress', 'protocol': 'tcp', 'port_range_min': 1, 'port_range_max': 65535, 'remote_ip_prefix': '10.1.0.145', 'id': '6663def7-5c7e-4600-8aa2-e48ebed8cc38'}}]}, {'op': 'replace', 'path': '/spec/podSelector', 'value': {'matchLabels': {'pod-name': 'client-a'}}}, {'op': 'replace', 'path': '/spec/networkpolicy_spec', 'value': {'podSelector': {'matchLabels': {'pod-name': 'client-a'}}, 'egress': [{'ports': [{'protocol': 'UDP', 'port': 53}]}, {'to': [{'podSelector': {'matchLabels': {'pod-name': 'server'}}}]}], 'policyTypes': ['Egress']}}] patch_crd /usr/local/lib/python3.6/site-packages/kuryr_kubernetes/k8s_client.py:134[00m
2020-07-09 16:44:09.978 1 ERROR kuryr_kubernetes.controller.drivers.utils [-] Error updating kuryrnetpolicy CRD np-allow-to-server-a-pod-selector: kuryr_kubernetes.exceptions.K8sClientException: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"KuryrNetPolicy.openstack.org \"np-allow-to-server-a-pod-selector\" is invalid: spec.egressSgRules.security_group_rule.id: Required value","reason":"Invalid","details":{"name":"np-allow-to-server-a-pod-selector","group":"openstack.org","kind":"KuryrNetPolicy","causes":[{"reason":"FieldValueRequired","message":"Required value","field":"spec.egressSgRules.security_group_rule.id"}]},"code":422}
2020-07-09 16:44:09.978 1 ERROR kuryr_kubernetes.controller.drivers.utils Traceback (most recent call last):
2020-07-09 16:44:09.978 1 ERROR kuryr_kubernetes.controller.drivers.utils File "/usr/local/lib/python3.6/site-packages/kuryr_kubernetes/controller/drivers/utils.py", line 227, in patch_kuryrnetworkpolicy_crd
2020-07-09 16:44:09.978 1 ERROR kuryr_kubernetes.controller.drivers.utils 'networkpolicy_spec': np_spec})
2020-07-09 16:44:09.978 1 ERROR kuryr_kubernetes.controller.drivers.utils File "/usr/local/lib/python3.6/site-packages/kuryr_kubernetes/k8s_client.py", line 139, in patch_crd
2020-07-09 16:44:09.978 1 ERROR kuryr_kubernetes.controller.drivers.utils self._raise_from_response(response)
2020-07-09 16:44:09.978 1 ERROR kuryr_kubernetes.controller.drivers.utils File "/usr/local/lib/python3.6/site-packages/kuryr_kubernetes/k8s_client.py", line 83, in _raise_from_response
2020-07-09 16:44:09.978 1 ERROR kuryr_kubernetes.controller.drivers.utils raise exc.K8sClientException(response.text)
2020-07-09 16:44:09.978 1 ERROR kuryr_kubernetes.controller.drivers.utils kuryr_kubernetes.exceptions.K8sClientException: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"KuryrNetPolicy.openstack.org \"np-allow-to-server-a-pod-selector\" is invalid: spec.egressSgRules.security_group_rule.id: Required value","reason":"Invalid","details":{"name":"np-allow-to-server-a-pod-selector","group":"openstack.org","kind":"KuryrNetPolicy","causes":[{"reason":"FieldValueRequired","message":"Required value","field":"spec.egressSgRules.security_group_rule.id"}]},"code":422}
2020-07-09 16:44:09.978 1 ERROR kuryr_kubernetes.controller.drivers.utils
2020-07-09 16:44:09.978 1 ERROR kuryr_kubernetes.controller.drivers.utils [00m
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry [-] Report handler unhealthy NetworkPolicyHandler: kuryr_kubernetes.exceptions.K8sClientException: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"KuryrNetPolicy.openstack.org \"np-allow-to-server-a-pod-selector\" is invalid: spec.egressSgRules.security_group_rule.id: Required value","reason":"Invalid","details":{"name":"np-allow-to-server-a-pod-selector","group":"openstack.org","kind":"KuryrNetPolicy","causes":[{"reason":"FieldValueRequired","message":"Required value","field":"spec.egressSgRules.security_group_rule.id"}]},"code":422}
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry Traceback (most recent call last):
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry File "/usr/local/lib/python3.6/site-packages/kuryr_kubernetes/handlers/retry.py", line 78, in __call__
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry self._handler(event, *args, **kwargs)
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry File "/usr/local/lib/python3.6/site-packages/kuryr_kubernetes/handlers/k8s_base.py", line 90, in __call__
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry self.on_present(obj)
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry File "/usr/local/lib/python3.6/site-packages/kuryr_kubernetes/controller/handlers/policy.py", line 53, in on_present
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry project_id)
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry File "/usr/local/lib/python3.6/site-packages/kuryr_kubernetes/controller/drivers/network_policy.py", line 58, in ensure_network_policy
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry self.update_security_group_rules_from_network_policy(policy))
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry File "/usr/local/lib/python3.6/site-packages/kuryr_kubernetes/controller/drivers/network_policy.py", line 119, in update_security_group_rules_from_network_policy
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry np_spec=policy['spec'])
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry File "/usr/local/lib/python3.6/site-packages/kuryr_kubernetes/controller/drivers/utils.py", line 227, in patch_kuryrnetworkpolicy_crd
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry 'networkpolicy_spec': np_spec})
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry File "/usr/local/lib/python3.6/site-packages/kuryr_kubernetes/k8s_client.py", line 139, in patch_crd
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry self._raise_from_response(response)
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry File "/usr/local/lib/python3.6/site-packages/kuryr_kubernetes/k8s_client.py", line 83, in _raise_from_response
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry raise exc.K8sClientException(response.text)
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry kuryr_kubernetes.exceptions.K8sClientException: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"KuryrNetPolicy.openstack.org \"np-allow-to-server-a-pod-selector\" is invalid: spec.egressSgRules.security_group_rule.id: Required value","reason":"Invalid","details":{"name":"np-allow-to-server-a-pod-selector","group":"openstack.org","kind":"KuryrNetPolicy","causes":[{"reason":"FieldValueRequired","message":"Required value","field":"spec.egressSgRules.security_group_rule.id"}]},"code":422}

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1. Run Network Policy tests and verify the controller is not restarted due to failure on patching the CRD
2.
3.

Actual results:


Expected results:


Additional info:

Comment 3 Jon Uriarte 2020-07-22 11:24:26 UTC
Verified in 4.6.0-0.nightly-2020-07-21-004949 on top of OSP 13.0.12 (2020-07-09.1).

OCP installation succeeded with Kuryr and the 23 NP tests passed. Kuryr controller pod
was restarted two times but not due to the error described in this BZ.
Could not find the message "Error updating kuryrnetpolicy CRD" with "code":422

Comment 5 errata-xmlrpc 2020-10-27 16:14:36 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.6 GA Images), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:4196