Description of problem:

During the Network Policy creation it's possible that the CRD is patched with repeated sg rules, which is not allowed, resulting in validation error as the repeated sg rules will not have the sg rule id.

2020-07-09 16:44:09.964 1 DEBUG kuryr_kubernetes.controller.drivers.utils [-] Patching KuryrNetPolicy CRD np-allow-to-server-a-pod-selector patch_kuryrnetworkpolicy_crd /usr/local/lib/python3.6/site-packages/kuryr_kubernetes/controller/drivers/utils.py:221[00m
2020-07-09 16:44:09.964 1 DEBUG kuryr_kubernetes.k8s_client [-] Patch /apis/openstack.org/v1/namespaces/network-policy-6479/kuryrnetpolicies/np-allow-to-server-a-pod-selector: [{'op': 'replace', 'path': '/spec/ingressSgRules', 'value': [{'security_group_rule': {'ethertype': 'IPv4', 'security_group_id': '718bef93-c6d9-4e52-a081-9d7a703ab3b4', 'direction': 'ingress', 'description': 'Kuryr-Kubernetes NetPolicy SG rule', 'id': '79c924de-0983-4b60-8f92-7beb1cac17cd'}}, {'security_group_rule': {'ethertype': 'IPv6', 'security_group_id': '718bef93-c6d9-4e52-a081-9d7a703ab3b4', 'direction': 'ingress', 'description': 'Kuryr-Kubernetes NetPolicy SG rule', 'id': '703b7159-d3f7-409e-b19d-c44eba3b1201'}}]}, {'op': 'replace', 'path': '/spec/egressSgRules', 'value': [{'security_group_rule': {'ethertype': 'IPv4', 'security_group_id': '718bef93-c6d9-4e52-a081-9d7a703ab3b4', 'description': 'Kuryr-Kubernetes NetPolicy SG rule', 'direction': 'egress', 'protocol': 'udp', 'port_range_min': 53, 'port_range_max': 53, 'id': '14eedd8d-3a67-4828-be1f-64b8d5220ea9'}}, {'security_group_rule': {'ethertype': 'IPv4', 'security_group_id': '718bef93-c6d9-4e52-a081-9d7a703ab3b4', 'description': 'Kuryr-Kubernetes NetPolicy SG rule', 'direction': 'egress', 'protocol': 'udp', 'port_range_min': 53, 'port_range_max': 53, 'remote_ip_prefix': '10.1.0.128/26'}}, {'security_group_rule': {'ethertype': 'IPv6', 'security_group_id': '718bef93-c6d9-4e52-a081-9d7a703ab3b4', 'description': 'Kuryr-Kubernetes NetPolicy SG rule', 'direction': 'egress', 'protocol': 'udp', 'port_range_min': 53, 'port_range_max': 53, 'id': '3616868c-c52d-4fc6-a596-9197ecea3cba'}}, {'security_group_rule': {'ethertype': 'IPv4', 'security_group_id': '718bef93-c6d9-4e52-a081-9d7a703ab3b4', 'description': 'Kuryr-Kubernetes NetPolicy SG rule', 'direction': 'egress', 'protocol': 'udp', 'port_range_min': 53, 'port_range_max': 53, 'remote_ip_prefix': '10.1.0.128/26'}}, {'security_group_rule': {'ethertype': 'IPv4', 'security_group_id': '718bef93-c6d9-4e52-a081-9d7a703ab3b4', 'description': 'Kuryr-Kubernetes NetPolicy SG rule', 'direction': 'egress', 'protocol': 'tcp', 'port_range_min': 1, 'port_range_max': 65535, 'remote_ip_prefix': '10.1.2.244', 'id': 'f553e38e-6240-46d9-b3b1-03e071d2340d'}, 'namespace': 'network-policy-6479'}, {'security_group_rule': {'ethertype': 'IPv4', 'security_group_id': '718bef93-c6d9-4e52-a081-9d7a703ab3b4', 'description': 'Kuryr-Kubernetes NetPolicy SG rule', 'direction': 'egress', 'protocol': 'tcp', 'port_range_min': 1, 'port_range_max': 65535, 'remote_ip_prefix': '10.1.0.150', 'id': 'c92a7d7e-0a96-4e5d-a572-74b9fbed3ec3'}}, {'security_group_rule': {'ethertype': 'IPv4', 'security_group_id': '718bef93-c6d9-4e52-a081-9d7a703ab3b4', 'description': 'Kuryr-Kubernetes NetPolicy SG rule', 'direction': 'egress', 'protocol': 'tcp', 'port_range_min': 1, 'port_range_max': 65535, 'remote_ip_prefix': '10.1.0.145', 'id': '6663def7-5c7e-4600-8aa2-e48ebed8cc38'}}]}, {'op': 'replace', 'path': '/spec/podSelector', 'value': {'matchLabels': {'pod-name': 'client-a'}}}, {'op': 'replace', 'path': '/spec/networkpolicy_spec', 'value': {'podSelector': {'matchLabels': {'pod-name': 'client-a'}}, 'egress': [{'ports': [{'protocol': 'UDP', 'port': 53}]}, {'to': [{'podSelector': {'matchLabels': {'pod-name': 'server'}}}]}], 'policyTypes': ['Egress']}}] patch_crd /usr/local/lib/python3.6/site-packages/kuryr_kubernetes/k8s_client.py:134[00m
2020-07-09 16:44:09.978 1 ERROR kuryr_kubernetes.controller.drivers.utils [-] Error updating kuryrnetpolicy CRD np-allow-to-server-a-pod-selector: kuryr_kubernetes.exceptions.K8sClientException: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"KuryrNetPolicy.openstack.org \"np-allow-to-server-a-pod-selector\" is invalid: spec.egressSgRules.security_group_rule.id: Required value","reason":"Invalid","details":{"name":"np-allow-to-server-a-pod-selector","group":"openstack.org","kind":"KuryrNetPolicy","causes":[{"reason":"FieldValueRequired","message":"Required value","field":"spec.egressSgRules.security_group_rule.id"}]},"code":422}
2020-07-09 16:44:09.978 1 ERROR kuryr_kubernetes.controller.drivers.utils Traceback (most recent call last):
2020-07-09 16:44:09.978 1 ERROR kuryr_kubernetes.controller.drivers.utils File "/usr/local/lib/python3.6/site-packages/kuryr_kubernetes/controller/drivers/utils.py", line 227, in patch_kuryrnetworkpolicy_crd
2020-07-09 16:44:09.978 1 ERROR kuryr_kubernetes.controller.drivers.utils 'networkpolicy_spec': np_spec})
2020-07-09 16:44:09.978 1 ERROR kuryr_kubernetes.controller.drivers.utils File "/usr/local/lib/python3.6/site-packages/kuryr_kubernetes/k8s_client.py", line 139, in patch_crd
2020-07-09 16:44:09.978 1 ERROR kuryr_kubernetes.controller.drivers.utils self._raise_from_response(response)
2020-07-09 16:44:09.978 1 ERROR kuryr_kubernetes.controller.drivers.utils File "/usr/local/lib/python3.6/site-packages/kuryr_kubernetes/k8s_client.py", line 83, in _raise_from_response
2020-07-09 16:44:09.978 1 ERROR kuryr_kubernetes.controller.drivers.utils raise exc.K8sClientException(response.text)
2020-07-09 16:44:09.978 1 ERROR kuryr_kubernetes.controller.drivers.utils kuryr_kubernetes.exceptions.K8sClientException: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"KuryrNetPolicy.openstack.org \"np-allow-to-server-a-pod-selector\" is invalid: spec.egressSgRules.security_group_rule.id: Required value","reason":"Invalid","details":{"name":"np-allow-to-server-a-pod-selector","group":"openstack.org","kind":"KuryrNetPolicy","causes":[{"reason":"FieldValueRequired","message":"Required value","field":"spec.egressSgRules.security_group_rule.id"}]},"code":422}
2020-07-09 16:44:09.978 1 ERROR kuryr_kubernetes.controller.drivers.utils
2020-07-09 16:44:09.978 1 ERROR kuryr_kubernetes.controller.drivers.utils [00m
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry [-] Report handler unhealthy NetworkPolicyHandler: kuryr_kubernetes.exceptions.K8sClientException: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"KuryrNetPolicy.openstack.org \"np-allow-to-server-a-pod-selector\" is invalid: spec.egressSgRules.security_group_rule.id: Required value","reason":"Invalid","details":{"name":"np-allow-to-server-a-pod-selector","group":"openstack.org","kind":"KuryrNetPolicy","causes":[{"reason":"FieldValueRequired","message":"Required value","field":"spec.egressSgRules.security_group_rule.id"}]},"code":422}
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry Traceback (most recent call last):
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry File "/usr/local/lib/python3.6/site-packages/kuryr_kubernetes/handlers/retry.py", line 78, in __call__
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry self._handler(event, *args, **kwargs)
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry File "/usr/local/lib/python3.6/site-packages/kuryr_kubernetes/handlers/k8s_base.py", line 90, in __call__
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry self.on_present(obj)
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry File "/usr/local/lib/python3.6/site-packages/kuryr_kubernetes/controller/handlers/policy.py", line 53, in on_present
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry project_id)
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry File "/usr/local/lib/python3.6/site-packages/kuryr_kubernetes/controller/drivers/network_policy.py", line 58, in ensure_network_policy
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry self.update_security_group_rules_from_network_policy(policy))
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry File "/usr/local/lib/python3.6/site-packages/kuryr_kubernetes/controller/drivers/network_policy.py", line 119, in update_security_group_rules_from_network_policy
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry np_spec=policy['spec'])
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry File "/usr/local/lib/python3.6/site-packages/kuryr_kubernetes/controller/drivers/utils.py", line 227, in patch_kuryrnetworkpolicy_crd
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry 'networkpolicy_spec': np_spec})
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry File "/usr/local/lib/python3.6/site-packages/kuryr_kubernetes/k8s_client.py", line 139, in patch_crd
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry self._raise_from_response(response)
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry File "/usr/local/lib/python3.6/site-packages/kuryr_kubernetes/k8s_client.py", line 83, in _raise_from_response
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry raise exc.K8sClientException(response.text)
2020-07-09 16:44:09.979 1 ERROR kuryr_kubernetes.handlers.retry kuryr_kubernetes.exceptions.K8sClientException: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"KuryrNetPolicy.openstack.org \"np-allow-to-server-a-pod-selector\" is invalid: spec.egressSgRules.security_group_rule.id: Required value","reason":"Invalid","details":{"name":"np-allow-to-server-a-pod-selector","group":"openstack.org","kind":"KuryrNetPolicy","causes":[{"reason":"FieldValueRequired","message":"Required value","field":"spec.egressSgRules.security_group_rule.id"}]},"code":422}

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1. Run Network Policy tests and verify the controller is not restarted due to failure on patching the CRD
2.
3.

Actual results:


Expected results:


Additional info: