RDO tickets are now tracked in Jira https://issues.redhat.com/projects/RDO/issues/
Bug 2215520 - Recent update of cinder breaks auto-removal of volumes upon deletion of a VM
Summary: Recent update of cinder breaks auto-removal of volumes upon deletion of a VM
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: RDO
Classification: Community
Component: openstack-cinder
Version: unspecified
Hardware: Unspecified
OS: Unspecified
unspecified
urgent
Target Milestone: ---
: trunk
Assignee: Eric Harney
QA Contact: Evelina Shames
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2023-06-16 10:34 UTC by Fritz Elfert
Modified: 2023-06-21 15:13 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2023-06-21 15:13:50 UTC
Embargoed:


Attachments (Terms of Use)
Log snippet of the compute node showing the error (4.74 KB, text/plain)
2023-06-16 10:34 UTC, Fritz Elfert
no flags Details

Description Fritz Elfert 2023-06-16 10:34:59 UTC
Created attachment 1971143 [details]
Log snippet of the compute node showing the error

Created attachment 1971143 [details]
Log snippet of the compute node showing the error

Description of problem:
After the last update of openstack cinder components, automatic removal of volumes  after deletion of a VM does not work anymore.

Version-Release number of selected component (if applicable):
openstack-cinder-1:20.3.0-1.el8.noarch
python3-cinder-1:20.3.0-1.el8.noarch
python3-cinder-common-1:20.3.0-1.el8.noarch

How reproducible:
always


Steps to Reproduce:
1. create a server with a volume that is to be automaticall removed upon deletion of the VM. E.g.:

nova boot --flavor someflavor --block-device source=image,id=62eb0a94-a502-4bff-886d-f1021b752820,dest=volume,size=40,shutdown=remove,bootindex=0 --nic net-name=provider --security-groups default --poll testvm

2. Delete the above VM.
3.

Actual results:
Upon deletion of the VM, and error is shown on the compute node where the VM has been running on. (look for a line containing ConflictNovaUsingAttachment in the attached log).
The volume is not removed but remains attached to to the now non-existing VM 

Expected results:
Volume should be removed


Additional info:
- Downgrading to the previous version 1:20.2.0-1.el8 restores functionality.
- Probably introduced by an attempt to fix this: https://bugzilla.redhat.com/show_bug.cgi?id=2196861
- The OpenStack version we use was not selectable: Yoga from http://mirrorlist.centos.org/?release=$releasever-stream&arch=$basearch&repo=cloud-openstack-yoga

Comment 1 Alfredo Moralejo 2023-06-16 13:24:56 UTC
I remember we had to do some config changes for that change in CI.

https://review.opendev.org/c/openstack/puppet-openstack-integration/+/884170

Please check following config:

in /etc/cinder/cinder.conf:

in section [keystone_authtoken]:

service_token_roles_required=True

and in /etc/nova/nova.conf you need to add a section [service_user] as shown in:

https://docs.openstack.org/nova/latest/admin/configuration/service-user-token.html

Could you try with that?

Alfredo

Comment 2 Fritz Elfert 2023-06-16 18:47:36 UTC
(In reply to Alfredo Moralejo from comment #1)
[...]
... as shown
> in:
> 
> https://docs.openstack.org/nova/latest/admin/configuration/service-user-
> token.html
> 
> Could you try with that?
> 
> Alfredo

Thanks for your fast reply.

I can confirm, that the problem is fixed, if the changes described in

https://docs.openstack.org/cinder/latest/configuration/block-storage/service-token.html
(note the missing "-user" compared to your doc link)

are applied.

Thanks again for your help,

-Fritz

Comment 3 Alan Pevec 2023-06-21 15:13:50 UTC
FTR configuration change required after the fix is documented in OpenStack Security Advisory https://security.openstack.org/ossa/OSSA-2023-003.html#configuration-change


Note You need to log in before you can comment on or make changes to this bug.