Bug 2215520

Summary: Recent update of cinder breaks auto-removal of volumes upon deletion of a VM
Product: [Community] RDO Reporter: Fritz Elfert <fritz>
Component: openstack-cinderAssignee: Eric Harney <eharney>
Status: CLOSED NOTABUG QA Contact: Evelina Shames <eshames>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: unspecifiedCC: amoralej, apevec, eharney, jcapitao, srevivo
Target Milestone: ---   
Target Release: trunk   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-06-21 15:13:50 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Log snippet of the compute node showing the error none

Description Fritz Elfert 2023-06-16 10:34:59 UTC 
Created attachment 1971143 [details]
Log snippet of the compute node showing the error

Created attachment 1971143 [details]
Log snippet of the compute node showing the error

Description of problem:
After the last update of openstack cinder components, automatic removal of volumes  after deletion of a VM does not work anymore.

Version-Release number of selected component (if applicable):
openstack-cinder-1:20.3.0-1.el8.noarch
python3-cinder-1:20.3.0-1.el8.noarch
python3-cinder-common-1:20.3.0-1.el8.noarch

How reproducible:
always


Steps to Reproduce:
1. create a server with a volume that is to be automaticall removed upon deletion of the VM. E.g.:

nova boot --flavor someflavor --block-device source=image,id=62eb0a94-a502-4bff-886d-f1021b752820,dest=volume,size=40,shutdown=remove,bootindex=0 --nic net-name=provider --security-groups default --poll testvm

2. Delete the above VM.
3.

Actual results:
Upon deletion of the VM, and error is shown on the compute node where the VM has been running on. (look for a line containing ConflictNovaUsingAttachment in the attached log).
The volume is not removed but remains attached to to the now non-existing VM 

Expected results:
Volume should be removed


Additional info:
- Downgrading to the previous version 1:20.2.0-1.el8 restores functionality.
- Probably introduced by an attempt to fix this: https://bugzilla.redhat.com/show_bug.cgi?id=2196861
- The OpenStack version we use was not selectable: Yoga from http://mirrorlist.centos.org/?release=$releasever-stream&arch=$basearch&repo=cloud-openstack-yoga
Comment 1 Alfredo Moralejo 2023-06-16 13:24:56 UTC 
I remember we had to do some config changes for that change in CI.

https://review.opendev.org/c/openstack/puppet-openstack-integration/+/884170

Please check following config:

in /etc/cinder/cinder.conf:

in section [keystone_authtoken]:

service_token_roles_required=True

and in /etc/nova/nova.conf you need to add a section [service_user] as shown in:

https://docs.openstack.org/nova/latest/admin/configuration/service-user-token.html

Could you try with that?

Alfredo
Comment 2 Fritz Elfert 2023-06-16 18:47:36 UTC 
(In reply to Alfredo Moralejo from comment #1)
[...]
... as shown
> in:
> 
> https://docs.openstack.org/nova/latest/admin/configuration/service-user-
> token.html
> 
> Could you try with that?
> 
> Alfredo

Thanks for your fast reply.

I can confirm, that the problem is fixed, if the changes described in

https://docs.openstack.org/cinder/latest/configuration/block-storage/service-token.html
(note the missing "-user" compared to your doc link)

are applied.

Thanks again for your help,

-Fritz
Comment 3 Alan Pevec 2023-06-21 15:13:50 UTC 
FTR configuration change required after the fix is documented in OpenStack Security Advisory https://security.openstack.org/ossa/OSSA-2023-003.html#configuration-change