Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1947414

Summary:	c-api returns 500 during volume attachment deletion while c-vol continues to unmap volume from compute
Product:	Red Hat OpenStack	Reporter:	Ganesh Kadam <gkadam>
Component:	openstack-cinder	Assignee:	Cinder Bugs List <cinder-bugs>
Status:	CLOSED INSUFFICIENT_DATA	QA Contact:	Evelina Shames <eshames>
Severity:	high	Docs Contact:	RHOS Documentation Team <rhos-docs>
Priority:	medium
Version:	13.0 (Queens)	CC:	abishop, dasmith, eglynn, eharney, jhakimra, kchamart, lyarwood, sbauza, sgordon, vromanso
Target Milestone:	---	Keywords:	Reopened, Triaged
Target Release:	---
Hardware:	All
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2025-01-07 15:42:51 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Comment 9 Alan Bishop 2021-04-27 17:55:24 UTC

I looked at the same logs discussed in comment #8, and it does seem the c-api request times out even though the detachment in cinder-volume eventually succeeds. And I agree with the advice to increase cinder's timeouts. You'll need to increase both the RPM and wsgi timeouts, and details for how to do that in OSP-13 (using puppet hiera data) is available in bug #1930806 comment #4.

What's not clear is why the cinder-volume service is lagging, and there are signs in the logs the periodic status reports are not completing in a timely manner every third time. The following sequence repeats:

2021-04-15 23:51:06.792 54 INFO cinder.volume.drivers.dell_emc.sc.storagecenter_api [req-fd97ea35-c239-4059-bef2-68f7aa768023 - - - - -] open_connection to 353606 at XXX
2021-04-15 23:52:06.792 54 INFO cinder.volume.drivers.dell_emc.sc.storagecenter_api [req-fd97ea35-c239-4059-bef2-68f7aa768023 - - - - -] open_connection to 353606 at XXX
2021-04-15 23:53:06.791 54 INFO cinder.volume.drivers.dell_emc.sc.storagecenter_api [req-fd97ea35-c239-4059-bef2-68f7aa768023 - - - - -] open_connection to 353606 at XXX
2021-04-15 23:54:22.850 54 WARNING oslo.service.loopingcall [req-fd97ea35-c239-4059-bef2-68f7aa768023 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 16.06 sec
2021-04-15 23:54:22.851 54 INFO cinder.volume.drivers.dell_emc.sc.storagecenter_api [req-fd97ea35-c239-4059-bef2-68f7aa768023 - - - - -] open_connection to 353606 at XXX
2021-04-15 23:55:22.854 54 INFO cinder.volume.drivers.dell_emc.sc.storagecenter_api [req-fd97ea35-c239-4059-bef2-68f7aa768023 - - - - -] open_connection to 353606 at XXX
2021-04-15 23:56:22.853 54 INFO cinder.volume.drivers.dell_emc.sc.storagecenter_api [req-fd97ea35-c239-4059-bef2-68f7aa768023 - - - - -] open_connection to 353606 at XXX
2021-04-15 23:57:34.590 54 WARNING oslo.service.loopingcall [req-fd97ea35-c239-4059-bef2-68f7aa768023 - - - - -] Function 'cinder.service.Service.periodic_tasks' run outlasted interval by 11.74 sec

But, to get any insight into what's happened, the service will need to run with DEBUG enabled.

Comment 14 Alan Bishop 2021-05-11 15:37:59 UTC

I just created bug #1959476 to cover backporting the puppet-cinder patch that adds support for tuning cinder's WSGI timeout.