Bug 2230264

Summary:	[OSP16.2] Missing tripleo volume type
Product:	Red Hat OpenStack	Reporter:	Matsvei Hauryliuk <mhauryli>
Component:	openstack-cinder	Assignee:	Cinder Bugs List <cinder-bugs>
Status:	NEW ---	QA Contact:	Evelina Shames <eshames>
Severity:	low	Docs Contact:	Andy Stillman <astillma>
Priority:	medium
Version:	16.2 (Train)	CC:	abishop, ldavidde, rdhasman
Target Milestone:	---	Keywords:	Triaged
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:		Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Matsvei Hauryliuk 2023-08-09 08:15:47 UTC

Description of problem:
Cu has had the tripleo  volume type missing in their deployment and as a result they couldn't create VMs neither volumes. Then in a remote with them we recreated the type, changed configuration on all controllers and restarted the cinder-volume container so it's working fine now. However Cu wishes to know why wasn't tripleo  there in the first place as they claim they didn't delete it manually.

Then I tested this in our lab and I only found logs for deleting volume types in /var/log/containers/cinder/cinder-api.log, mainly this:
cinder/cinder-api.log:2023-08-03 10:37:46.179 13 INFO cinder.api.openstack.wsgi [req-799593c0-069a-45da-8580-1b05e6cec382 e88be695c80b4a10af25b5cfc6b23216 f3f795a5d0754bfc8580bfb61f2a5794 - default default] DELETE <ip>/v3/f3f795a5d0754bfc8580bfb61f2a5794/types/8822e007-14a4-4fbb-8755-c8c649983a67
cinder/cinder-api.log:2023-08-03 10:37:46.443 13 INFO cinder.api.openstack.wsgi [req-799593c0-069a-45da-8580-1b05e6cec382 e88be695c80b4a10af25b5cfc6b23216 f3f795a5d0754bfc8580bfb61f2a5794 - default default] <ip>/v3/f3f795a5d0754bfc8580bfb61f2a5794/types/8822e007-14a4-4fbb-8755-c8c649983a67 returned with HTTP 202
httpd/cinder-api/cinder_wsgi_access.log:172.17.1.53 - - [03/Aug/2023:10:37:46 +0000] "DELETE /v3/f3f795a5d0754bfc8580bfb61f2a5794/types/8822e007-14a4-4fbb-8755-c8c649983a67 HTTP/1.1" 202 - "-" "python-cinderclient"

But it only shows UUID of deleted volume type and as we don't know what was the UUID of tripleo  we can't trace that.
Do you think there's anywhere in the logs that would reference the deleted volume type name instead of UUID?

Version-Release number of selected component (if applicable):
OSP 16.2.5
puppet-cinder-15.5.0-2.20211210013139.15557bb.el8ost.noarch

How reproducible:
Happened once.

Steps to Reproduce:
Details provided in the Description section.

Actual results:
Customer wasn't able to create new volumes/launch new VMs.

Expected results:
Successfully performing the minor upgrade.

Additional info:
Sosreports from all 3 controller nodes are available, 
as well as outputs of 
$ openstack volume type list
$ openstack volume service list
$ openstack volume list 
$ openstack volume type show <..> -> for all available volume types 
Also db dumps both from the day of the occurrence of the issue - Aug 2, and after the issue has been resolved - Aug 8.