Bug 1883565
| Summary: | unhealthy KuryrLoadBalancerHandler: AttributeError: 'NoneType' object has no attribute 'id' | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | rlobillo | ||||||||
| Component: | Networking | Assignee: | Maysa Macedo <mdemaced> | ||||||||
| Networking sub component: | kuryr | QA Contact: | GenadiC <gcheresh> | ||||||||
| Status: | CLOSED ERRATA | Docs Contact: | |||||||||
| Severity: | high | ||||||||||
| Priority: | unspecified | CC: | bbennett | ||||||||
| Version: | 4.6 | ||||||||||
| Target Milestone: | --- | ||||||||||
| Target Release: | 4.6.0 | ||||||||||
| Hardware: | Unspecified | ||||||||||
| OS: | Unspecified | ||||||||||
| Whiteboard: | |||||||||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |||||||||
| Doc Text: | Story Points: | --- | |||||||||
| Clone Of: | Environment: | ||||||||||
| Last Closed: | 2020-10-27 16:46:19 UTC | Type: | Bug | ||||||||
| Regression: | --- | Mount Type: | --- | ||||||||
| Documentation: | --- | CRM: | |||||||||
| Verified Versions: | Category: | --- | |||||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||||
| Embargoed: | |||||||||||
| Bug Depends On: | 1883166 | ||||||||||
| Bug Blocks: | |||||||||||
| Attachments: |
|
||||||||||
Verified on OCP4.6.0-0.nightly-2020-10-02-001427 over OSP16.1 (RHOS-16.1-RHEL-8-20200917.n.3) with OVN-Octavia. NP run successfully without restarts: # Kuryr pods before running NP tests - ANSIBLE MANAGED BLOCK NAME READY STATUS RESTARTS AGE kuryr-cni-89pm9 1/1 Running 1 35m kuryr-cni-jfltp 1/1 Running 4 61m kuryr-cni-k4j95 1/1 Running 0 61m kuryr-cni-l87vw 1/1 Running 0 35m kuryr-cni-zhvzc 1/1 Running 0 35m kuryr-cni-zpmfv 1/1 Running 0 61m kuryr-controller-775ff4bb-bgpml 1/1 Running 1 61m # END ANSIBLE MANAGED BLOCK # Kuryr pods after running NP tests - ANSIBLE MANAGED BLOCK NAME READY STATUS RESTARTS AGE kuryr-cni-89pm9 1/1 Running 1 138m kuryr-cni-jfltp 1/1 Running 4 164m kuryr-cni-k4j95 1/1 Running 0 164m kuryr-cni-l87vw 1/1 Running 0 138m kuryr-cni-zhvzc 1/1 Running 0 138m kuryr-cni-zpmfv 1/1 Running 0 164m kuryr-controller-775ff4bb-bgpml 1/1 Running 1 164m # END ANSIBLE MANAGED BLOCK Kuryr controller logs and NP results attached. Created attachment 1718414 [details]
NP test results
Created attachment 1718415 [details]
kuryr-controller logs
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (OpenShift Container Platform 4.6 GA Images), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:4196 |
Created attachment 1717571 [details] kuryr-controller logs Description of problem: Running NP tests, below exception is observed on kuryr-controller, which leads to a restart: 2020-09-29 15:04:42.948 1 ERROR kuryr_kubernetes.handlers.retry [-] Report handler unhealthy KuryrLoadBalancerHandler: AttributeError: 'NoneType' object has no attribute 'id' 2020-09-29 15:04:42.948 1 ERROR kuryr_kubernetes.handlers.retry Traceback (most recent call last): 2020-09-29 15:04:42.948 1 ERROR kuryr_kubernetes.handlers.retry File "/usr/lib/python3.6/site-packages/kuryr_kubernetes/handlers/retry.py", line 81, in __call__ 2020-09-29 15:04:42.948 1 ERROR kuryr_kubernetes.handlers.retry self._handler(event, *args, **kwargs) 2020-09-29 15:04:42.948 1 ERROR kuryr_kubernetes.handlers.retry File "/usr/lib/python3.6/site-packages/kuryr_kubernetes/handlers/k8s_base.py", line 84, in __call__ 2020-09-29 15:04:42.948 1 ERROR kuryr_kubernetes.handlers.retry self.on_present(obj) 2020-09-29 15:04:42.948 1 ERROR kuryr_kubernetes.handlers.retry File "/usr/lib/python3.6/site-packages/kuryr_kubernetes/controller/handlers/loadbalancer.py", line 90, in on_present 2020-09-29 15:04:42.948 1 ERROR kuryr_kubernetes.handlers.retry if self._sync_lbaas_members(loadbalancer_crd): 2020-09-29 15:04:42.948 1 ERROR kuryr_kubernetes.handlers.retry File "/usr/lib/python3.6/site-packages/kuryr_kubernetes/controller/handlers/loadbalancer.py", line 206, in _sync_lbaas_members 2020-09-29 15:04:42.948 1 ERROR kuryr_kubernetes.handlers.retry if self._sync_lbaas_pools(loadbalancer_crd): 2020-09-29 15:04:42.948 1 ERROR kuryr_kubernetes.handlers.retry File "/usr/lib/python3.6/site-packages/kuryr_kubernetes/controller/handlers/loadbalancer.py", line 467, in _sync_lbaas_pools 2020-09-29 15:04:42.948 1 ERROR kuryr_kubernetes.handlers.retry if self._sync_lbaas_listeners(loadbalancer_crd): 2020-09-29 15:04:42.948 1 ERROR kuryr_kubernetes.handlers.retry File "/usr/lib/python3.6/site-packages/kuryr_kubernetes/controller/handlers/loadbalancer.py", line 565, in _sync_lbaas_listeners 2020-09-29 15:04:42.948 1 ERROR kuryr_kubernetes.handlers.retry if self._add_new_listeners(loadbalancer_crd): 2020-09-29 15:04:42.948 1 ERROR kuryr_kubernetes.handlers.retry File "/usr/lib/python3.6/site-packages/kuryr_kubernetes/controller/handlers/loadbalancer.py", line 604, in _add_new_listeners 2020-09-29 15:04:42.948 1 ERROR kuryr_kubernetes.handlers.retry service_type=loadbalancer_crd['spec'].get('type')) 2020-09-29 15:04:42.948 1 ERROR kuryr_kubernetes.handlers.retry File "/usr/lib/python3.6/site-packages/kuryr_kubernetes/controller/drivers/lbaasv2.py", line 362, in ensure_listener 2020-09-29 15:04:42.948 1 ERROR kuryr_kubernetes.handlers.retry os_net.update_port(vip_port.id, security_groups=[]) 2020-09-29 15:04:42.948 1 ERROR kuryr_kubernetes.handlers.retry AttributeError: 'NoneType' object has no attribute 'id' 2020-09-29 15:04:42.948 1 ERROR kuryr_kubernetes.handlers.retry ^[[00m Version-Release number of selected component (if applicable): (shiftstack) [stack@undercloud-0 ~]$ oc get clusterversion NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.6.0-0.nightly-2020-09-28-212756 True False 3h1m Cluster version is 4.6.0-0.nightly-2020-09-28-212756 (shiftstack) [stack@undercloud-0 ~]$ cat core_puddle_version RHOS-16.1-RHEL-8-20200917.n.3 How reproducible: Ramdon Steps to Reproduce: 1. Run NP tests and check restarts on kuryr-controllers. 2. 3. Actual results: kuryr-controller is getting restarted. Expected results: No kuryr-controller restarts. Additional info: