Bug 2123657
| Summary: | Error while updating the load balancer status: 'NoneType' object has no attribute 'project_id' | ||
|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Gregory Thiemonge <gthiemon> |
| Component: | openstack-octavia | Assignee: | Nate Johnston <nate.johnston> |
| Status: | CLOSED DUPLICATE | QA Contact: | Bruna Bonguardo <bbonguar> |
| Severity: | high | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 17.0 (Wallaby) | CC: | ihrachys, lpeer, majopela, scohen |
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2022-09-06 12:31:57 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Description of problem: When using a 3rd party provider, Octavia resource statuses might be updated multiple times for some reasons (concurrent requests, or successive requests that trigger errors) For instance the ovn-provider, a cascade deletion request triggered an error in OVN and the provider sent a status update to Octavia with some newly deleted resources: 2022-09-02 07:32:51.290 16 INFO octavia.api.v2.controllers.load_balancer [req-8965a055-f039-43bb-94c6-19c71b67c8e0 - 79f6bc108da84222b42dea61f0a97116 - default default] Sending delete Load Balancer 3fe6f3c3-bc9b-4a7a-aee3-617513ef1b8f to provider ovn 2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper [-] Exception occurred during deletion of loadbalancer: ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Switch with uuid=42019590-84d6-4aea-80d5-5805ee939f1c 2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper Traceback (most recent call last): 2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper File "/usr/lib/python3.9/site-packages/ovn_octavia_provider/helper.py", line 1093, in lb_delete 2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper status = self._lb_delete(loadbalancer, ovn_lb, status) 2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper File "/usr/lib/python3.9/site-packages/ovn_octavia_provider/helper.py", line 1168, in _lb_delete 2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper self._execute_commands(commands) 2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper File "/usr/lib/python3.9/site-packages/ovn_octavia_provider/helper.py", line 505, in _execute_commands 2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper txn.add(command) 2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper File "/usr/lib64/python3.9/contextlib.py", line 126, in __exit__ 2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper next(self.gen) 2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper File "/usr/lib/python3.9/site-packages/ovn_octavia_provider/ovsdb/impl_idl_ovn.py", line 165, in transaction 2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper yield t 2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper File "/usr/lib64/python3.9/contextlib.py", line 126, in __exit__ 2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper next(self.gen) 2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper File "/usr/lib/python3.9/site-packages/ovsdbapp/api.py", line 110, in transaction 2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper del self._nested_txns_map[cur_thread_id] 2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper File "/usr/lib/python3.9/site-packages/ovsdbapp/api.py", line 61, in __exit__ 2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper self.result = self.commit() 2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper File "/usr/lib/python3.9/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 65, in commit 2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper raise result.ex 2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper File "/usr/lib/python3.9/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 131, in run 2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper txn.results.put(txn.do_commit()) 2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper File "/usr/lib/python3.9/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 93, in do_commit 2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper command.run_idl(txn) 2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper File "/usr/lib/python3.9/site-packages/ovsdbapp/schema/ovn_northbound/commands.py", line 1220, in run_idl 2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper ls = self.api.lookup('Logical_Switch', self.switch) 2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper File "/usr/lib/python3.9/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 208, in lookup 2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper return self._lookup(table, record) 2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper File "/usr/lib/python3.9/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 240, in _lookup 2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper raise idlutils.RowNotFound(table=table, col='uuid', 2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Switch with uuid=42019590-84d6-4aea-80d5-5805ee939f1c 2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper 2022-09-02 07:32:51.366 16 DEBUG ovn_octavia_provider.helper [-] Updating status to octavia: {'loadbalancers': [{'id': '3fe6f3c3-bc9b-4a7a-aee3-617513ef1b8f', 'provisioning_status': 'ERROR', 'operating_status': 'ERROR'}], 'listeners': [{'id': '4e0f41a0-fc84-4f1a-b29c-1e2999ce67f4', 'provisioning_status': 'DELETED', 'operating_status': 'OFFLINE'}, {'id': 'd7cf0ad1-06f4-4e7f-9178-a61df93be0af', 'provisioning_status': 'DELETED', 'operating_status': 'OFFLINE'}], 'pools': [{'id': '15a8b3ce-ab48-46f4-9bb5-4399cdc00268', 'provisioning_status': 'DELETED'}, {'id': '4c8b7c7a-8187-4495-87c4-899492d1d03c', 'provisioning_status': 'DELETED'}], 'members': [{'id': 'b355ab65-c427-4439-aecb-4bc5db414ada', 'provisioning_status': 'DELETED'}, {'id': '0c4a6b5c-f874-4d26-9101-9f574a07994b', 'provisioning_status': 'DELETED'}]} _update_status_to_octavia /usr/lib/python3.9/site-packages/ovn_octavia_provider/helper.py:370 The LB was then in ERROR, another deletion request was sent, ovn-provider sent a new status update: 2022-09-02 07:37:58.442 17 INFO octavia.api.v2.controllers.load_balancer [req-58e71239-90c7-4250-a734-947bba62f46c - 79f6bc108da84222b42dea61f0a97116 - default default] Sending delete Load Balancer 3fe6f3c3-bc9b-4a7a-aee3-617513ef1b8f to provider ovn 2022-09-02 07:37:58.444 17 DEBUG ovn_octavia_provider.helper [-] Handling request lb_delete with info {'id': '3fe6f3c3-bc9b-4a7a-aee3-617513ef1b8f', 'cascade': True} request_handler /usr/lib/python3.9/site-packages/ovn_octavia_provider/helper.py:343 2022-09-02 07:37:59.846 17 DEBUG ovn_octavia_provider.helper [-] Updating status to octavia: {'loadbalancers': [{'id': '3fe6f3c3-bc9b-4a7a-aee3-617513ef1b8f', 'provisioning_status': 'DELETED', 'operating_status': 'OFFLINE'}], 'listeners': [{'id': '4e0f41a0-fc84-4f1a-b29c-1e2999ce67f4', 'provisioning_status': 'DELETED', 'operating_status': 'OFFLINE'}, {'id': 'd7cf0ad1-06f4-4e7f-9178-a61df93be0af', 'provisioning_status': 'DELETED', 'operating_status': 'OFFLINE'}], 'pools': [{'id': '15a8b3ce-ab48-46f4-9bb5-4399cdc00268', 'provisioning_status': 'DELETED'}, {'id': '4c8b7c7a-8187-4495-87c4-899492d1d03c', 'provisioning_status': 'DELETED'}], 'members': [{'id': 'b355ab65-c427-4439-aecb-4bc5db414ada', 'provisioning_status': 'DELETED'}, {'id': '0c4a6b5c-f874-4d26-9101-9f574a07994b', 'provisioning_status': 'DELETED'}]} _update_status_to_octavia /usr/lib/python3.9/site-packages/ovn_octavia_provider/helper.py:370 2022-09-02 07:48:55.134 17 DEBUG ovn_octavia_provider.helper [-] Updating status to octavia: {'loadbalancers': [{'id': '3fe6f3c3-bc9b-4a7a-aee3-617513ef1b8f', 'provisioning_status': 'DELETED', 'operating_status': 'OFFLINE'}], 'listeners': [{'id': '4e0f41a0-fc84-4f1a-b29c-1e2999ce67f4', 'provisioning_status': 'DELETED', 'operating_status': 'OFFLINE'}, {'id': 'd7cf0ad1-06f4-4e7f-9178-a61df93be0af', 'provisioning_status': 'DELETED', 'operating_status': 'OFFLINE'}], 'pools': [{'id': '15a8b3ce-ab48-46f4-9bb5-4399cdc00268', 'provisioning_status': 'DELETED'}, {'id': '4c8b7c7a-8187-4495-87c4-899492d1d03c', 'provisioning_status': 'DELETED'}], 'members': [{'id': 'b355ab65-c427-4439-aecb-4bc5db414ada', 'provisioning_status': 'DELETED'}, {'id': '0c4a6b5c-f874-4d26-9101-9f574a07994b', 'provisioning_status': 'DELETED'}]} _update_status_to_octavia /usr/lib/python3.9/site-packages/ovn_octavia_provider/helper.py:370 2022-09-02 07:48:55.330 17 ERROR ovn_octavia_provider.helper [-] Error while updating the load balancer status: 'NoneType' object has no attribute 'project_id': octavia_lib.api.drivers.exceptions.UpdateStatusError: 'NoneType' object has no attribute 'project_id' The exception "UpdateStatusError: 'NoneType' object has no attribute 'project_id'" was raised, it appears that NoneType was returned when getting the listener from DB (the listener was deleted from the DB in the previous call) Version-Release number of selected component (if applicable): 17.0 How reproducible: randomly