Description of problem: When we upgrade RHOSP13 to RHOSP16.1, legacy telemetry services(gnocchi, ceilometer, aodh) are removed during upgrade unless we explicitly enable these services, because in RHOSP16.1 these services are disabled by default. However, even though these services are removed, there still remain keystone users and endpoints for these services. ~~~ (overcloud) [stack@undercloud-0 ~]$ openstack user list +----------------------------------+-------------------------+ | ID | Name | +----------------------------------+-------------------------+ | db28032d4d3048e6b44af9ebbe2849b6 | admin | | 0a23478d66aa4fdc96cf2fefeef7f44c | heat_stack_domain_admin | | 19a3ad7cefa149b2b4480baf95d45e1d | aodh | | 6f10f80f4e3240d5b76440c0d80d545a | ceilometer | | 9f9dcdb0485c44579b433c4b5da24de8 | cinder | | aa34ea7432b44b51a744f35c692f243b | glance | | 190e94ef521c422fac56dc75c35766c7 | gnocchi | | 0d90e4de67704248a983e1520552d13d | heat | | ae6cd1d75df149479a68d2acb8c64cea | heat-cfn | | 1fd3600f0e6c4623b570992a24dab9d7 | neutron | | 1c947c116ff341fc89f64bad5c1258bb | nova | | aefd04ca26674f7cb5511f4774251163 | placement | | 87e3a3f02d4b4314a7bcd8f7a3d6e276 | panko | | 146ab311240c44189b0b8e4047b7c547 | swift | | 01ce39520ae74cc79de7a6bfc93277e9 | demo | | 37a54fbcb1cf4f89ac0f15d4ab98b7a7 | alt_demo | +----------------------------------+-------------------------+ (overcloud) [stack@undercloud-0 ~]$ openstack endpoint list --service gnocchi +----------------------------------+-----------+--------------+--------------+---------+-----------+-------------------------+ | ID | Region | Service Name | Service Type | Enabled | Interface | URL | +----------------------------------+-----------+--------------+--------------+---------+-----------+-------------------------+ | 00dd0d9ddae14efc8790e41dd35c2aca | regionOne | gnocchi | metric | True | public | http://10.0.0.145:8041 | | 6fab462b31fd429fabc7d46969c1542e | regionOne | gnocchi | metric | True | admin | http://172.17.1.25:8041 | | 8106b0c93cb14ab1ad6ca26a81e35f32 | regionOne | gnocchi | metric | True | internal | http://172.17.1.25:8041 | +----------------------------------+-----------+--------------+--------------+---------+-----------+-------------------------+ (overcloud) [stack@undercloud-0 ~]$ openstack endpoint list --service aodh +----------------------------------+-----------+--------------+--------------+---------+-----------+-------------------------+ | ID | Region | Service Name | Service Type | Enabled | Interface | URL | +----------------------------------+-----------+--------------+--------------+---------+-----------+-------------------------+ | 502aed5d2b9e44279703960da96ff9cf | regionOne | aodh | alarming | True | admin | http://172.17.1.25:8042 | | 9102491fd47d4ccfb54d31b0ea08a2e4 | regionOne | aodh | alarming | True | internal | http://172.17.1.25:8042 | | fdf5c941b8fc416c9c5e71acfb989c3a | regionOne | aodh | alarming | True | public | http://10.0.0.145:8042 | +----------------------------------+-----------+--------------+--------------+---------+-----------+-------------------------+ (overcloud) [stack@undercloud-0 ~]$ openstack endpoint list --service panko +----------------------------------+-----------+--------------+--------------+---------+-----------+-------------------------+ | ID | Region | Service Name | Service Type | Enabled | Interface | URL | +----------------------------------+-----------+--------------+--------------+---------+-----------+-------------------------+ | 258ba405245a4b298663a2ad0186da63 | regionOne | panko | event | True | admin | http://172.17.1.25:8977 | | 6620c535de3f4c00bf4b1828d8e0b6ad | regionOne | panko | event | True | internal | http://172.17.1.25:8977 | | eccb044e46824fa5a1a6e01835d97b72 | regionOne | panko | event | True | public | http://10.0.0.145:8977 | +----------------------------------+-----------+--------------+--------------+---------+-----------+-------------------------+ ~~~ ~~~ [heat-admin@controller-0 ~]$ sudo podman ps | grep gnocchi [heat-admin@controller-0 ~]$ sudo podman ps | grep panko [heat-admin@controller-0 ~]$ sudo podman ps | grep aodh [heat-admin@controller-0 ~]$ ~~~ Version-Release number of selected component (if applicable): RHOSP16.1.1 How reproducible: Always Steps to Reproduce: 1. Upgrade RHOSP13 to RHOSP16.1 by following the upgrade documentation Actual results: Keystone users and endpoints for legacy telemetry services are left after upgrade Expected results: Keystone users and endpoints for legacy telemetry services are removed during upgrade Additional info:
This might result in a doc bug, but I'm setting tht as the associated component first, because it would make sense to consider implementing a clean up steps in tripleo, which is executed during upgrade.
Looks like we can automate this via the deployment tooling in TripleO Heat Templates by creating inline Ansible in the `upgrade_tasks: []` sections of the various components. However I don't have a test environment to test any changes I'm making, so I'll have to defer this to a future z-stream release of 16.2 since upgrades to 16.1 are no longer allowed.
The work to automate cleaning up users of disabled telemetry services is abandoned. Only the endpoints will be deleteing using THT. A manual cleanup will be required to delete users as a part of post upgrade action [1] [1] https://bugzilla.redhat.com/show_bug.cgi?id=2165803
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Red Hat OpenStack Platform 16.2.5 (Train) bug fix and enhancement advisory), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2023:1763