2123657 – Error while updating the load balancer status: 'NoneType' object has no attribute 'project_id'

Bug 2123657 - Error while updating the load balancer status: 'NoneType' object has no attribute 'project_id'

Summary: Error while updating the load balancer status: 'NoneType' object has no attri...

Keywords:
Status:	CLOSED DUPLICATE of bug 2123658
Alias:	None
Product:	Red Hat OpenStack
Classification:	Red Hat
Component:	openstack-octavia
Sub Component:
Version:	17.0 (Wallaby)
Hardware:	Unspecified
OS:	Unspecified
Priority:	unspecified
Severity:	high
Target Milestone:	---
Target Release:	---
Assignee:	Nate Johnston
QA Contact:	Bruna Bonguardo
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+	depends on / blocked

Reported:	2022-09-02 10:03 UTC by Gregory Thiemonge
Modified:	2022-09-06 12:31 UTC (History)
CC List:	4 users (show)
Fixed In Version:
Doc Type:	If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:	2022-09-06 12:31:57 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)

Links
System	ID	Private	Priority	Status	Summary	Last Updated
Red Hat Issue Tracker	OSP-18523	0	None	None	None	2022-09-02 10:09:52 UTC

Description Gregory Thiemonge 2022-09-02 10:03:48 UTC

Description of problem:
When using a 3rd party provider, Octavia resource statuses might be updated multiple times for some reasons (concurrent requests, or successive requests that trigger errors)

For instance the ovn-provider, a cascade deletion request triggered an error in OVN and the provider sent a status update to Octavia with some newly deleted resources:

2022-09-02 07:32:51.290 16 INFO octavia.api.v2.controllers.load_balancer [req-8965a055-f039-43bb-94c6-19c71b67c8e0 - 79f6bc108da84222b42dea61f0a97116 - default default] Sending delete Load Balancer 3fe6f3c3-bc9b-4a7a-aee3-617513ef1b8f to provider ovn
2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper [-] Exception occurred during deletion of loadbalancer: ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Switch with uuid=42019590-84d6-4aea-80d5-5805ee939f1c
2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper Traceback (most recent call last):
2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper   File "/usr/lib/python3.9/site-packages/ovn_octavia_provider/helper.py", line 1093, in lb_delete
2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper     status = self._lb_delete(loadbalancer, ovn_lb, status)
2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper   File "/usr/lib/python3.9/site-packages/ovn_octavia_provider/helper.py", line 1168, in _lb_delete
2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper     self._execute_commands(commands)
2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper   File "/usr/lib/python3.9/site-packages/ovn_octavia_provider/helper.py", line 505, in _execute_commands
2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper     txn.add(command)
2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper   File "/usr/lib64/python3.9/contextlib.py", line 126, in __exit__
2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper     next(self.gen)
2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper   File "/usr/lib/python3.9/site-packages/ovn_octavia_provider/ovsdb/impl_idl_ovn.py", line 165, in transaction
2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper     yield t
2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper   File "/usr/lib64/python3.9/contextlib.py", line 126, in __exit__
2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper     next(self.gen)
2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper   File "/usr/lib/python3.9/site-packages/ovsdbapp/api.py", line 110, in transaction
2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper     del self._nested_txns_map[cur_thread_id]
2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper   File "/usr/lib/python3.9/site-packages/ovsdbapp/api.py", line 61, in __exit__
2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper     self.result = self.commit()
2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper   File "/usr/lib/python3.9/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 65, in commit
2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper     raise result.ex
2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper   File "/usr/lib/python3.9/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 131, in run
2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper     txn.results.put(txn.do_commit())
2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper   File "/usr/lib/python3.9/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 93, in do_commit
2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper     command.run_idl(txn)
2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper   File "/usr/lib/python3.9/site-packages/ovsdbapp/schema/ovn_northbound/commands.py", line 1220, in run_idl
2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper     ls = self.api.lookup('Logical_Switch', self.switch)
2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper   File "/usr/lib/python3.9/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 208, in lookup
2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper     return self._lookup(table, record)
2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper   File "/usr/lib/python3.9/site-packages/ovsdbapp/backend/ovs_idl/__init__.py", line 240, in _lookup
2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper     raise idlutils.RowNotFound(table=table, col='uuid',
2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper ovsdbapp.backend.ovs_idl.idlutils.RowNotFound: Cannot find Logical_Switch with uuid=42019590-84d6-4aea-80d5-5805ee939f1c
2022-09-02 07:32:51.364 16 ERROR ovn_octavia_provider.helper
2022-09-02 07:32:51.366 16 DEBUG ovn_octavia_provider.helper [-] Updating status to octavia: {'loadbalancers': [{'id': '3fe6f3c3-bc9b-4a7a-aee3-617513ef1b8f', 'provisioning_status': 'ERROR', 'operating_status': 'ERROR'}], 'listeners': [{'id': '4e0f41a0-fc84-4f1a-b29c-1e2999ce67f4', 'provisioning_status': 'DELETED', 'operating_status': 'OFFLINE'}, {'id': 'd7cf0ad1-06f4-4e7f-9178-a61df93be0af', 'provisioning_status': 'DELETED', 'operating_status': 'OFFLINE'}], 'pools': [{'id': '15a8b3ce-ab48-46f4-9bb5-4399cdc00268', 'provisioning_status': 'DELETED'}, {'id': '4c8b7c7a-8187-4495-87c4-899492d1d03c', 'provisioning_status': 'DELETED'}], 'members': [{'id': 'b355ab65-c427-4439-aecb-4bc5db414ada', 'provisioning_status': 'DELETED'}, {'id': '0c4a6b5c-f874-4d26-9101-9f574a07994b', 'provisioning_status': 'DELETED'}]} _update_status_to_octavia /usr/lib/python3.9/site-packages/ovn_octavia_provider/helper.py:370

The LB was then in ERROR, another deletion request was sent, ovn-provider sent a new status update:

2022-09-02 07:37:58.442 17 INFO octavia.api.v2.controllers.load_balancer [req-58e71239-90c7-4250-a734-947bba62f46c - 79f6bc108da84222b42dea61f0a97116 - default default] Sending delete Load Balancer 3fe6f3c3-bc9b-4a7a-aee3-617513ef1b8f to provider ovn
2022-09-02 07:37:58.444 17 DEBUG ovn_octavia_provider.helper [-] Handling request lb_delete with info {'id': '3fe6f3c3-bc9b-4a7a-aee3-617513ef1b8f', 'cascade': True} request_handler /usr/lib/python3.9/site-packages/ovn_octavia_provider/helper.py:343
2022-09-02 07:37:59.846 17 DEBUG ovn_octavia_provider.helper [-] Updating status to octavia: {'loadbalancers': [{'id': '3fe6f3c3-bc9b-4a7a-aee3-617513ef1b8f', 'provisioning_status': 'DELETED', 'operating_status': 'OFFLINE'}], 'listeners': [{'id': '4e0f41a0-fc84-4f1a-b29c-1e2999ce67f4', 'provisioning_status': 'DELETED', 'operating_status': 'OFFLINE'}, {'id': 'd7cf0ad1-06f4-4e7f-9178-a61df93be0af', 'provisioning_status': 'DELETED', 'operating_status': 'OFFLINE'}], 'pools': [{'id': '15a8b3ce-ab48-46f4-9bb5-4399cdc00268', 'provisioning_status': 'DELETED'}, {'id': '4c8b7c7a-8187-4495-87c4-899492d1d03c', 'provisioning_status': 'DELETED'}], 'members': [{'id': 'b355ab65-c427-4439-aecb-4bc5db414ada', 'provisioning_status': 'DELETED'}, {'id': '0c4a6b5c-f874-4d26-9101-9f574a07994b', 'provisioning_status': 'DELETED'}]} _update_status_to_octavia /usr/lib/python3.9/site-packages/ovn_octavia_provider/helper.py:370
2022-09-02 07:48:55.134 17 DEBUG ovn_octavia_provider.helper [-] Updating status to octavia: {'loadbalancers': [{'id': '3fe6f3c3-bc9b-4a7a-aee3-617513ef1b8f', 'provisioning_status': 'DELETED', 'operating_status': 'OFFLINE'}], 'listeners': [{'id': '4e0f41a0-fc84-4f1a-b29c-1e2999ce67f4', 'provisioning_status': 'DELETED', 'operating_status': 'OFFLINE'}, {'id': 'd7cf0ad1-06f4-4e7f-9178-a61df93be0af', 'provisioning_status': 'DELETED', 'operating_status': 'OFFLINE'}], 'pools': [{'id': '15a8b3ce-ab48-46f4-9bb5-4399cdc00268', 'provisioning_status': 'DELETED'}, {'id': '4c8b7c7a-8187-4495-87c4-899492d1d03c', 'provisioning_status': 'DELETED'}], 'members': [{'id': 'b355ab65-c427-4439-aecb-4bc5db414ada', 'provisioning_status': 'DELETED'}, {'id': '0c4a6b5c-f874-4d26-9101-9f574a07994b', 'provisioning_status': 'DELETED'}]} _update_status_to_octavia /usr/lib/python3.9/site-packages/ovn_octavia_provider/helper.py:370
2022-09-02 07:48:55.330 17 ERROR ovn_octavia_provider.helper [-] Error while updating the load balancer status: 'NoneType' object has no attribute 'project_id': octavia_lib.api.drivers.exceptions.UpdateStatusError: 'NoneType' object has no attribute 'project_id'


The exception "UpdateStatusError: 'NoneType' object has no attribute 'project_id'" was raised, it appears that NoneType was returned when getting the listener from DB (the listener was deleted from the DB in the previous call)


Version-Release number of selected component (if applicable):
17.0

How reproducible:
randomly

Note You need to log in before you can comment on or make changes to this bug.