Bug 1828889

Summary: [ovn]: CRUD agent operations don't work for agents stored in Neutron database
Product: Red Hat OpenStack Reporter: Eduardo Olivares <eolivare>
Component: python-networking-ovnAssignee: Terry Wilson <twilson>
Status: CLOSED ERRATA QA Contact: Eran Kuris <ekuris>
Severity: low Docs Contact:
Priority: high    
Version: 16.1 (Train)CC: amcleod, apevec, ebarrera, ekuris, gregraka, igallagh, jlibosva, jmelvin, lhh, lmartins, majopela, rdiwakar, rsafrono, scohen, svigan, twilson, yocha
Target Milestone: z4Keywords: Triaged
Target Release: 16.1 (Train on RHEL 8.2)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: python-networking-ovn-7.3.1-1.20201114024050.el8ost Doc Type: Bug Fix
Doc Text:
Before this update, the OVN mechanism driver did not correctly merge its agent list with those stored in the Networking (neutron) service database. With this update, the results from the OVN and Networking service database are merged before the API returns the result.
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-03-17 15:30:39 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1666684, 1823334    

Description Eduardo Olivares 2020-04-28 14:23:07 UTC
Description of problem:
 (overcloud) [stack@undercloud-0 tempest-dir]$ openstack network agent list 
 +--------------------------------------+----------------------+----------------------------+-------------------+-------+-------+-------------------------------+
 | ID                                   | Agent Type           | Host                       | Availability Zone | Alive | State | Binary                        |
 +--------------------------------------+----------------------+----------------------------+-------------------+-------+-------+-------------------------------+
 | 2fe52280-b3c2-4bf8-82ec-82a1a15ec12f | NIC Switch agent     | computesriov-0.localdomain | None              | :-)   | UP    | neutron-sriov-nic-agent       |
 | 8a4c0e90-5fa5-4ae8-b74f-fae0142b22b3 | NIC Switch agent     | computesriov-1.localdomain | None              | :-)   | UP    | neutron-sriov-nic-agent       |
 | 13969db2-fde1-42d0-8e6b-e709c9d07981 | OVN Controller agent | controller-2.localdomain   | n/a               | :-)   | UP    | ovn-controller                |
 | f01508ff-7ce7-4de8-8bdf-ee46a77d0674 | OVN Metadata agent   | controller-2.localdomain   | n/a               | :-)   | UP    | networking-ovn-metadata-agent |
 | 479e0c95-8438-4a22-aa6f-68673be395bf | OVN Controller agent | controller-0.localdomain   | n/a               | :-)   | UP    | ovn-controller                |
 | 53b06fdb-baad-4bda-beb8-0fdecff98887 | OVN Metadata agent   | controller-0.localdomain   | n/a               | :-)   | UP    | networking-ovn-metadata-agent |
 | 23f36188-5536-434c-ac1d-0c3dd92db54d | OVN Controller agent | controller-1.localdomain   | n/a               | :-)   | UP    | ovn-controller                |
 | 0d9d6405-1e90-4ff3-b0b8-11a6284742f4 | OVN Metadata agent   | controller-1.localdomain   | n/a               | :-)   | UP    | networking-ovn-metadata-agent |
 | 369294de-5897-49c6-94b8-710cd8ae68dd | OVN Controller agent | computesriov-0.localdomain | n/a               | :-)   | UP    | ovn-controller                |
 | ab54d69c-6496-4c14-a442-dc6cf7e5590b | OVN Metadata agent   | computesriov-0.localdomain | n/a               | :-)   | UP    | networking-ovn-metadata-agent |
 | fc062427-7332-43f7-b61b-eb00f9f5ba98 | OVN Controller agent | computesriov-1.localdomain | n/a               | :-)   | UP    | ovn-controller                |
 | 2903a683-83f7-4b9c-9759-d0cf41867aea | OVN Metadata agent   | computesriov-1.localdomain | n/a               | :-)   | UP    | networking-ovn-metadata-agent |
 +--------------------------------------+----------------------+----------------------------+-------------------+-------+-------+-------------------------------+
 (overcloud) [stack@undercloud-0 tempest-dir]$ openstack network agent show 2fe52280-b3c2-4bf8-82ec-82a1a15ec12f
 No Agent found for 2fe52280-b3c2-4bf8-82ec-82a1a15ec12f: Client Error for url: http://10.46.21.194:9696/v2.0/agents/2fe52280-b3c2-4bf8-82ec-82a1a15ec12f, Agent 2fe52280-b3c2-4bf8-82ec-82a1a15ec12f could not be found.


Version-Release number of selected component (if applicable):
RHOS-16.1-RHEL-8-20200424.n.0
python3-networking-ovn-7.1.1-0.20200422213423.1d7e326.el8ost.noarch


How reproducible:
100%

Steps to Reproduce:
1. run command 'openstack network agent show <neutron-sriov-nic-agent-id>'
2.
3.

Actual results:
Error

Expected results:
Show agent information

Additional info:

Comment 3 Jakub Libosvar 2020-08-07 14:33:40 UTC
*** Bug 1862890 has been marked as a duplicate of this bug. ***

Comment 4 Jakub Libosvar 2020-08-31 07:49:59 UTC
*** Bug 1695073 has been marked as a duplicate of this bug. ***

Comment 5 Roman Safronov 2020-09-08 14:01:11 UTC
*** Bug 1651523 has been marked as a duplicate of this bug. ***

Comment 6 Terry Wilson 2020-09-09 19:21:00 UTC
The issue with not being able to show/update/delete non-OVN agents was due to a bug where we would catch KeyError to signal that we should pass the agent request back up to ML2 since the agent wasn't ours, but at some point the exception was changed to an AgentNotFound. This meant that we were only handling OVN agents and never handing control back to ML2 for any others.

Comment 8 Jakub Libosvar 2020-09-18 09:11:22 UTC
*** Bug 1838989 has been marked as a duplicate of this bug. ***

Comment 9 Jakub Libosvar 2020-09-18 09:11:39 UTC
*** Bug 1738554 has been marked as a duplicate of this bug. ***

Comment 20 Terry Wilson 2021-01-15 22:55:12 UTC
*** Bug 1695073 has been marked as a duplicate of this bug. ***

Comment 21 Eran Kuris 2021-01-24 12:27:40 UTC
Fix verified:
[root@controller-2 ~]# podman  exec  -it neutron_api /bin/bash 
()[neutron@controller-2 /]$ rpm -qa | grep ovn
python3-networking-ovn-7.3.1-1.20201114024050.el8ost.noarch



(overcloud) [stack@undercloud-0 ~]$ openstack network agent list
+--------------------------------------+------------------------------+----------------------------+-------------------+-------+-------+-------------------------------+
| ID                                   | Agent Type                   | Host                       | Availability Zone | Alive | State | Binary                        |
+--------------------------------------+------------------------------+----------------------------+-------------------+-------+-------+-------------------------------+
| 14141e98-6822-4797-aecb-8d94b7ad9618 | NIC Switch agent             | computesriov-1.localdomain | None              | :-)   | UP    | neutron-sriov-nic-agent       |
| 4eba124c-52fd-4c22-ac30-e734dfa16159 | NIC Switch agent             | computesriov-0.localdomain | None              | :-)   | UP    | neutron-sriov-nic-agent       |
| 3a3f9720-89f1-45fb-8a80-d604710e1cfd | OVN Controller agent         | computesriov-0.localdomain |                   | :-)   | UP    | ovn-controller                |
| 28a4d143-deb8-4467-a49e-ec87b10b5b5c | OVN Metadata agent           | computesriov-0.localdomain |                   | :-)   | UP    | networking-ovn-metadata-agent |
| 4c9d522e-459d-41fe-a5f1-dec123b28c44 | OVN Controller agent         | computesriov-1.localdomain |                   | :-)   | UP    | ovn-controller                |
| cb243789-1c0c-4754-8a74-b04c835eafa7 | OVN Metadata agent           | computesriov-1.localdomain |                   | :-)   | UP    | networking-ovn-metadata-agent |
| b1aaec30-8539-4045-9ff4-8aa98e56420c | OVN Controller Gateway agent | controller-1.localdomain   |                   | :-)   | UP    | ovn-controller                |
| fe97f43c-aa79-4b1b-8188-c195670ed568 | OVN Metadata agent           | controller-1.localdomain   |                   | :-)   | UP    | networking-ovn-metadata-agent |
| a8e2618c-bd71-4665-bd59-103d4192b341 | OVN Controller Gateway agent | controller-0.localdomain   |                   | :-)   | UP    | ovn-controller                |
| ec73385a-c961-4634-95a2-b99c159a22fd | OVN Metadata agent           | controller-0.localdomain   |                   | :-)   | UP    | networking-ovn-metadata-agent |
| 75409ed7-5ad5-4775-abf8-f9ba65eae92f | OVN Controller Gateway agent | controller-2.localdomain   |                   | :-)   | UP    | ovn-controller                |
| a51094df-653e-4761-94c2-3f5de4f0a1f5 | OVN Metadata agent           | controller-2.localdomain   |                   | :-)   | UP    | networking-ovn-metadata-agent |
+--------------------------------------+------------------------------+----------------------------+-------------------+-------+-------+-------------------------------+
(overcloud) [stack@undercloud-0 ~]$ openstack network agent show 14141e98-6822-4797-aecb-8d94b7ad9618
+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field             | Value                                                                                                                                                                                                                                                                                                                                             |
+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| admin_state_up    | UP                                                                                                                                                                                                                                                                                                                                                |
| agent_type        | NIC Switch agent                                                                                                                                                                                                                                                                                                                                  |
| alive             | :-)                                                                                                                                                                                                                                                                                                                                               |
| availability_zone | None                                                                                                                                                                                                                                                                                                                                              |
| binary            | neutron-sriov-nic-agent                                                                                                                                                                                                                                                                                                                           |
| configuration     | {'device_mappings': {'datacentre': ['enp7s0f3', 'enp5s0f0']}, 'devices': 0, 'extensions': [], 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'enp7s0f3': 'computesriov-1', 'enp5s0f0': 'computesriov-1'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}} |
| created_at        | 2021-01-21 19:16:21                                                                                                                                                                                                                                                                                                                               |
| description       | None                                                                                                                                                                                                                                                                                                                                              |
| ha_state          | None                                                                                                                                                                                                                                                                                                                                              |
| host              | computesriov-1.localdomain                                                                                                                                                                                                                                                                                                                        |
| id                | 14141e98-6822-4797-aecb-8d94b7ad9618                                                                                                                                                                                                                                                                                                              |
| last_heartbeat_at | 2021-01-24 08:59:11                                                                                                                                                                                                                                                                                                                               |
| location          | cloud='', project.domain_id=, project.domain_name='Default', project.id='1c2caf8553554699b0a96b87cda6c6ba', project.name='admin', region_name='regionOne', zone=                                                                                                                                                                                  |
| name              | None                                                                                                                                                                                                                                                                                                                                              |
| resources_synced  | None                                                                                                                                                                                                                                                                                                                                              |
| started_at        | 2021-01-21 19:16:21                                                                                                                                                                                                                                                                                                                               |
| topic             | N/A                                                                                                                                                                                                                                                                                                                                               |
+-------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

Comment 28 errata-xmlrpc 2021-03-17 15:30:39 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat OpenStack Platform 16.1.4 director bug fix advisory), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:0817

Comment 29 errata-xmlrpc 2021-03-17 15:37:33 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat OpenStack Platform 16.1.4 director bug fix advisory), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:0817