Bug 2228733

Summary: PackStack Cant detach a volume from a VM (non root VM running or not)
Product: [Community] RDO Reporter: jon.hartley <jon.hartley>
Component: openstack-novaAssignee: nova-maint
Status: NEW --- QA Contact: nova-maint
Severity: urgent Docs Contact:
Priority: unspecified    
Version: unspecified   
Target Milestone: ---   
Target Release: trunk   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: ---
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
setup, cli output, nova, cinder logs none

Description jon.hartley@btinternet.com 2023-08-03 06:46:22 UTC
Created attachment 1981418 [details]
setup, cli output, nova, cinder logs

Install of PackStack on Centos Stream 9
See attachment


Version-Release number of selected component (if applicable):
openstack-rdo-antelope
openstack-rdo-zen
openstack-rdo-yoga

How reproducible:
Do a standard install on Packstack, cinder LVM on Centos 9 

Steps to Reproduce:
Do a standard install on Packstack, cinder LVM on Centos 9 
Add an Centos8 stream image
Boot image from volume
Create empty volume and attach (this bit goes fine) 
Try to detach either via UI or CLI and Cinder complains that the volume is in use by nova when nova has detached the volume (message in logs) 

Actual results:
Cinder volume remains "detaching", VM no longer has volume attached


Expected results:
Cinder volume detaches and ends up "available" and not attached


Additional info:
logs attached from Nova and Cinder


I mostly use OpenStack to run persistent workloads in Kubernetes, this makes it nearly unuseable as cinder-csi removes and attaches volumes on diffrent VMs as part of normal lifedycle

Comment 1 jon.hartley@btinternet.com 2023-08-03 07:14:19 UTC
This happens EVERY time I try to detach a volume, have tried all RDO releases for Centos9 Stream 
All other functionality seems fine, create VMs. networks, routers etc etc
Can run up a 5 node rancher k8s cluster but as soon as it comes to moving PV attachments around pods get stuck in Pending as the CSI cant mount the PV on the new node
Have ensured I am up to date with CSI hence testing via openstack cli tools