Bug 1975264
| Summary: | "neutron agent-list" command fails because of AttributeError: Row instance has no attribute 'hostname' | ||
|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Alex Stupnikov <astupnik> |
| Component: | python-networking-ovn | Assignee: | PURANDHAR SAIRAM MANNIDI <pmannidi> |
| Status: | CLOSED DUPLICATE | QA Contact: | Eran Kuris <ekuris> |
| Severity: | high | Docs Contact: | |
| Priority: | high | ||
| Version: | 16.1 (Train) | CC: | apevec, lhh, lmartins, majopela, pambre, ralonsoh, scohen |
| Target Milestone: | --- | Keywords: | Triaged |
| Target Release: | --- | ||
| Hardware: | All | ||
| OS: | All | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2021-07-23 01:25:11 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
This looks more like https://bugzilla.redhat.com/show_bug.cgi?id=1849166 you need to restart neutron_api containers or systemd service to make sure ml2/ovn syncs with OVN DB after OVN SB DB records in Chassis and Chassis_Private are deleted. *** This bug has been marked as a duplicate of bug 1849166 *** |
Description of problem: "neutron agent-list" is blocked, we see traces [1] in Neutron Server logs. It looks like some OVN table contain incorrect entry, but it is hard to understand which one. I am setting high severity because we need a workaround for production RHOSP deployment. It could be reduced after workaround is applied. Version-Release number of selected component (if applicable): 16.1.5 GA How reproducible: It is unclear which steps caused this state of OVN DB. At this point "neutron agent-list" is blocked. [1] 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource [req-24b87380-c9bc-4b8c-b30f-0d9dd8f74c23 del del - default default] index failed: No details.: AttributeError: Row instance has no attribute 'hostname' 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource Traceback (most recent call last): 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource File "/usr/lib64/python3.6/site-packages/ovs/db/idl.py", line 1018, in __getattr__ 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource column = self._table.columns[column_name] 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource KeyError: 'hostname' 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource During handling of the above exception, another exception occurred: 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource Traceback (most recent call last): 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron/api/v2/resource.py", line 98, in resource 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource result = method(request=request, **args) 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 139, in wrapped 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource setattr(e, '_RETRY_EXCEEDED', True) 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource self.force_reraise() 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb) 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/six.py", line 675, in reraise 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource raise value 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 135, in wrapped 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource return f(*args, **kwargs) 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_db/api.py", line 154, in wrapper 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource ectxt.value = e.inner_exc 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource self.force_reraise() 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb) 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/six.py", line 675, in reraise 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource raise value 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_db/api.py", line 142, in wrapper 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource return f(*args, **kwargs) 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 183, in wrapped 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource LOG.debug("Retry wrapper got retriable exception: %s", e) 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource self.force_reraise() 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource six.reraise(self.type_, self.value, self.tb) 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/six.py", line 675, in reraise 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource raise value 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron_lib/db/api.py", line 179, in wrapped 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource return f(*dup_args, **dup_kwargs) 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron/api/v2/base.py", line 369, in index 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource return self._items(request, True, parent_id) 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/neutron/api/v2/base.py", line 304, in _items 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource obj_list = obj_getter(request.context, **kwargs) 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/networking_ovn/ml2/mech_driver.py", line 1060, in fn 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource return op(results, new_method(*args, _driver=self, **kwargs)) 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/networking_ovn/ml2/mech_driver.py", line 1132, in get_agents 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource for agent in _driver.agents_from_chassis(ch, update_db).values(): 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/networking_ovn/ml2/mech_driver.py", line 1050, in agents_from_chassis 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource agent_dict[agent.agent_id] = agent.as_dict(alive) 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource File "/usr/lib/python3.6/site-packages/networking_ovn/agent/neutron_agent.py", line 51, in as_dict 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource 'host': self.chassis.hostname, 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource File "/usr/lib64/python3.6/site-packages/ovs/db/idl.py", line 1021, in __getattr__ 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource (self.__class__.__name__, column_name)) 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource AttributeError: Row instance has no attribute 'hostname' 2021-06-23 12:37:16.373 35 ERROR neutron.api.v2.resource 2021-06-23 12:37:16.375 35 INFO neutron.wsgi [req-24b87380-c9bc-4b8c-b30f-0d9dd8f74c23 del del - default default] 192.168.1.1 "GET /v2.0/agents HTTP/1.1" status: 500 len: 344 time: 0.3377833