Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1876045

Summary: Keystone users and endpoints for legacy telemetry services are not deleted during upgrades
Product: Red Hat OpenStack Reporter: Takashi Kajinami <tkajinam>
Component: tripleo-ansibleAssignee: Yadnesh Kulkarni <ykulkarn>
Status: CLOSED ERRATA QA Contact: Leonid Natapov <lnatapov>
Severity: low Docs Contact: mgeary <mgeary>
Priority: low    
Version: 16.1 (Train)CC: ifrangs, jamsmith, jstransk, lmadsen, mbultel, mburns, mrunge, rrasouli, ykulkarn
Target Milestone: z5Keywords: Triaged, ZStream
Target Release: 16.2 (Train on RHEL 8.4)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: tripleo-ansible-0.8.1-2.20230302004949.0db9f93.el8ost Doc Type: Bug Fix
Doc Text:
Before this update, endpoints of disabled telemetry services were not cleaned up after an upgrade. This omission did not impact the cloud. With this update, upgrades delete obsolete telemetry endpoints.
 Story Points: ---
Clone Of: Environment:
Last Closed: 2023-04-26 12:16:45 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Takashi Kajinami 2020-09-05 06:30:42 UTC 
Description of problem:

When we upgrade RHOSP13 to RHOSP16.1, legacy telemetry services(gnocchi, ceilometer, aodh) are removed during upgrade unless we explicitly enable these services, because in RHOSP16.1 these services are disabled by default.

However, even though these services are removed, there still remain keystone users and endpoints for these services.
~~~
(overcloud) [stack@undercloud-0 ~]$ openstack user list
+----------------------------------+-------------------------+
| ID                               | Name                    |
+----------------------------------+-------------------------+
| db28032d4d3048e6b44af9ebbe2849b6 | admin                   |
| 0a23478d66aa4fdc96cf2fefeef7f44c | heat_stack_domain_admin |
| 19a3ad7cefa149b2b4480baf95d45e1d | aodh                    |
| 6f10f80f4e3240d5b76440c0d80d545a | ceilometer              |
| 9f9dcdb0485c44579b433c4b5da24de8 | cinder                  |
| aa34ea7432b44b51a744f35c692f243b | glance                  |
| 190e94ef521c422fac56dc75c35766c7 | gnocchi                 |
| 0d90e4de67704248a983e1520552d13d | heat                    |
| ae6cd1d75df149479a68d2acb8c64cea | heat-cfn                |
| 1fd3600f0e6c4623b570992a24dab9d7 | neutron                 |
| 1c947c116ff341fc89f64bad5c1258bb | nova                    |
| aefd04ca26674f7cb5511f4774251163 | placement               |
| 87e3a3f02d4b4314a7bcd8f7a3d6e276 | panko                   |
| 146ab311240c44189b0b8e4047b7c547 | swift                   |
| 01ce39520ae74cc79de7a6bfc93277e9 | demo                    |
| 37a54fbcb1cf4f89ac0f15d4ab98b7a7 | alt_demo                |
+----------------------------------+-------------------------+
(overcloud) [stack@undercloud-0 ~]$ openstack endpoint list --service gnocchi
+----------------------------------+-----------+--------------+--------------+---------+-----------+-------------------------+
| ID                               | Region    | Service Name | Service Type | Enabled | Interface | URL                     |
+----------------------------------+-----------+--------------+--------------+---------+-----------+-------------------------+
| 00dd0d9ddae14efc8790e41dd35c2aca | regionOne | gnocchi      | metric       | True    | public    | http://10.0.0.145:8041  |
| 6fab462b31fd429fabc7d46969c1542e | regionOne | gnocchi      | metric       | True    | admin     | http://172.17.1.25:8041 |
| 8106b0c93cb14ab1ad6ca26a81e35f32 | regionOne | gnocchi      | metric       | True    | internal  | http://172.17.1.25:8041 |
+----------------------------------+-----------+--------------+--------------+---------+-----------+-------------------------+
(overcloud) [stack@undercloud-0 ~]$ openstack endpoint list --service aodh
+----------------------------------+-----------+--------------+--------------+---------+-----------+-------------------------+
| ID                               | Region    | Service Name | Service Type | Enabled | Interface | URL                     |
+----------------------------------+-----------+--------------+--------------+---------+-----------+-------------------------+
| 502aed5d2b9e44279703960da96ff9cf | regionOne | aodh         | alarming     | True    | admin     | http://172.17.1.25:8042 |
| 9102491fd47d4ccfb54d31b0ea08a2e4 | regionOne | aodh         | alarming     | True    | internal  | http://172.17.1.25:8042 |
| fdf5c941b8fc416c9c5e71acfb989c3a | regionOne | aodh         | alarming     | True    | public    | http://10.0.0.145:8042  |
+----------------------------------+-----------+--------------+--------------+---------+-----------+-------------------------+
(overcloud) [stack@undercloud-0 ~]$ openstack endpoint list --service panko
+----------------------------------+-----------+--------------+--------------+---------+-----------+-------------------------+
| ID                               | Region    | Service Name | Service Type | Enabled | Interface | URL                     |
+----------------------------------+-----------+--------------+--------------+---------+-----------+-------------------------+
| 258ba405245a4b298663a2ad0186da63 | regionOne | panko        | event        | True    | admin     | http://172.17.1.25:8977 |
| 6620c535de3f4c00bf4b1828d8e0b6ad | regionOne | panko        | event        | True    | internal  | http://172.17.1.25:8977 |
| eccb044e46824fa5a1a6e01835d97b72 | regionOne | panko        | event        | True    | public    | http://10.0.0.145:8977  |
+----------------------------------+-----------+--------------+--------------+---------+-----------+-------------------------+
~~~

~~~
[heat-admin@controller-0 ~]$ sudo podman ps | grep gnocchi 
[heat-admin@controller-0 ~]$ sudo podman ps | grep panko
[heat-admin@controller-0 ~]$ sudo podman ps | grep aodh
[heat-admin@controller-0 ~]$
~~~

Version-Release number of selected component (if applicable):
RHOSP16.1.1

How reproducible:
Always

Steps to Reproduce:
1. Upgrade RHOSP13 to RHOSP16.1 by following the upgrade documentation

Actual results:
Keystone users and endpoints for legacy telemetry services are left after upgrade

Expected results:
Keystone users and endpoints for legacy telemetry services are removed during upgrade

Additional info:
Comment 1 Takashi Kajinami 2020-09-05 06:32:54 UTC 
This might result in a doc bug, but I'm setting tht as the associated component first, because it would make sense to consider implementing a clean up steps in tripleo, which is executed during upgrade.
Comment 7 Leif Madsen 2022-05-30 14:56:35 UTC 
Looks like we can automate this via the deployment tooling in TripleO Heat Templates by creating inline Ansible in the `upgrade_tasks: []` sections of the various components.

However I don't have a test environment to test any changes I'm making, so I'll have to defer this to a future z-stream release of 16.2 since upgrades to 16.1 are no longer allowed.
Comment 11 Yadnesh Kulkarni 2023-01-31 06:19:09 UTC 
The work to automate cleaning up users of disabled telemetry services is abandoned. Only the endpoints will be deleteing using THT.

A manual cleanup will be required to delete users as a part of post upgrade action [1]

[1] https://bugzilla.redhat.com/show_bug.cgi?id=2165803
Comment 22 errata-xmlrpc 2023-04-26 12:16:45 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat OpenStack Platform 16.2.5 (Train) bug fix and enhancement advisory), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2023:1763