Description of problem: When using `cinder reset-state --state available --attach-status detached VOLUME` and attaching again the volume to the vm, it shows as duplicated When I used `openstack volume set --state available VOLUME`, on step 3. it never attaches: ~~~ (openstack) volume list +--------------------------------------+--------------+-----------+------+------------------------------+ | ID | Display Name | Status | Size | Attached to | +--------------------------------------+--------------+-----------+------+------------------------------+ | 5212a72c-bc78-4e14-a041-5d8e994baba7 | vol | attaching | 1 | Attached to vm1 on /dev/vdb | +--------------------------------------+--------------+-----------+------+------------------------------+ ~~~ Version-Release number of selected component (if applicable): puppet-cinder-9.5.0-1.el7ost.noarch openstack-cinder-9.1.4-3.el7ost.noarch python-cinderclient-1.9.0-6.el7ost.noarch python-cinder-9.1.4-3.el7ost.noarch How reproducible: Always Steps to Reproduce: 1. Attach a volume (vol) to a vm (vm1): openstack server add volume vm1 vol 2. Detach it using cinder client: cinder reset-state --state available --attach-status detached vol 3. Attach the volume to another vm (vm2) (Probably this step is optional): openstack server add volume vm2 vol 4. Detach it again using cinder client: cinder reset-state --state available --attach-status detached vol 5. Attach the volume again to the first volume: openstack server add volume vm1 vol 6. Check the volumes attached to the vm: openstack server show -c name -c os-extended-volumes:volumes_attached -c OS-EXT-STS:power_state -c OS-EXT-STS:vm_state vm1 Actual results: Expected results: Additional info: ~~~ # openstack (openstack) volume create vol --size 1 +---------------------+--------------------------------------+ | Field | Value | +---------------------+--------------------------------------+ | attachments | [] | | availability_zone | nova | | bootable | false | | consistencygroup_id | None | | created_at | 2020-10-12T09:02:26.425287 | | description | None | | encrypted | False | | id | 999d5bec-5573-4e43-824d-1569a06999d7 | | migration_status | None | | multiattach | False | | name | vol | | properties | | | replication_status | disabled | | size | 1 | | snapshot_id | None | | source_volid | None | | status | creating | | type | iscsi | | updated_at | None | | user_id | d7cccd7efdbb41769b52cf2ce0bc8209 | +---------------------+--------------------------------------+ (openstack) volume list +--------------------------------------+--------------+-----------+------+-------------+ | ID | Display Name | Status | Size | Attached to | +--------------------------------------+--------------+-----------+------+-------------+ | 999d5bec-5573-4e43-824d-1569a06999d7 | vol | available | 1 | | +--------------------------------------+--------------+-----------+------+-------------+ (openstack) server create --flavor 1 --image cirros --nic net-id=private vm1 --wait +--------------------------------------+----------------------------------------------------------+ | Field | Value | +--------------------------------------+----------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | osp10-1.gsslab.pnq2.redhat.com | | OS-EXT-SRV-ATTR:hypervisor_hostname | osp10-1.gsslab.pnq2.redhat.com | | OS-EXT-SRV-ATTR:instance_name | instance-00000003 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2020-10-12T09:03:00.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | private=10.0.0.5 | | adminPass | sCRggDhY5Xy5 | | config_drive | | | created | 2020-10-12T09:02:50Z | | flavor | m1.tiny (1) | | hostId | bfbcd2b4e530a357a627416e2e0ec479fb9dfbd585a34cb9dc723b18 | | id | 1496a79c-4d67-4449-ae79-0491be47df57 | | image | cirros (a5d93202-5930-499c-8025-665e7b03b9aa) | | key_name | None | | name | vm1 | | os-extended-volumes:volumes_attached | [] | | progress | 0 | | project_id | 69dba7b4214b4601af28a3df7fb3f9be | | properties | | | security_groups | [{u'name': u'default'}] | | status | ACTIVE | | updated | 2020-10-12T09:03:01Z | | user_id | d7cccd7efdbb41769b52cf2ce0bc8209 | +--------------------------------------+----------------------------------------------------------+ (openstack) server create --flavor 1 --image cirros --nic net-id=private vm2 --wait +--------------------------------------+----------------------------------------------------------+ | Field | Value | +--------------------------------------+----------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | osp10-1.gsslab.pnq2.redhat.com | | OS-EXT-SRV-ATTR:hypervisor_hostname | osp10-1.gsslab.pnq2.redhat.com | | OS-EXT-SRV-ATTR:instance_name | instance-00000004 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2020-10-12T09:03:45.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | private=10.0.0.11 | | adminPass | Tsh79Av7hRXF | | config_drive | | | created | 2020-10-12T09:03:33Z | | flavor | m1.tiny (1) | | hostId | bfbcd2b4e530a357a627416e2e0ec479fb9dfbd585a34cb9dc723b18 | | id | 22eb565d-6a91-45e0-80e2-0613b54b92ce | | image | cirros (a5d93202-5930-499c-8025-665e7b03b9aa) | | key_name | None | | name | vm2 | | os-extended-volumes:volumes_attached | [] | | progress | 0 | | project_id | 69dba7b4214b4601af28a3df7fb3f9be | | properties | | | security_groups | [{u'name': u'default'}] | | status | ACTIVE | | updated | 2020-10-12T09:03:46Z | | user_id | d7cccd7efdbb41769b52cf2ce0bc8209 | +--------------------------------------+----------------------------------------------------------+ (openstack) server list +--------------------------------------+------+--------+-------------------+------------+ | ID | Name | Status | Networks | Image Name | +--------------------------------------+------+--------+-------------------+------------+ | 22eb565d-6a91-45e0-80e2-0613b54b92ce | vm2 | ACTIVE | private=10.0.0.11 | cirros | | 1496a79c-4d67-4449-ae79-0491be47df57 | vm1 | ACTIVE | private=10.0.0.5 | cirros | +--------------------------------------+------+--------+-------------------+------------+ (openstack) server add volume vm1 vol (openstack) server show -c name -c os-extended-volumes:volumes_attached -c OS-EXT-STS:power_state -c OS-EXT-STS:vm_state vm1 +--------------------------------------+----------------------------------------------------+ | Field | Value | +--------------------------------------+----------------------------------------------------+ | OS-EXT-STS:power_state | Running | | OS-EXT-STS:vm_state | active | | name | vm1 | | os-extended-volumes:volumes_attached | [{u'id': u'999d5bec-5573-4e43-824d-1569a06999d7'}] | +--------------------------------------+----------------------------------------------------+ (openstack) volume list +--------------------------------------+--------------+--------+------+------------------------------+ | ID | Display Name | Status | Size | Attached to | +--------------------------------------+--------------+--------+------+------------------------------+ | 999d5bec-5573-4e43-824d-1569a06999d7 | vol | in-use | 1 | Attached to vm1 on /dev/vdb | +--------------------------------------+--------------+--------+------+------------------------------+ [root@osp10-1 ~(keystone_admin)]# cinder reset-state --state available --attach-status detached vol [root@osp10-1 ~(keystone_admin)]# openstack volume show -c name -c attachments -c status vol +-------------+-----------+ | Field | Value | +-------------+-----------+ | attachments | [] | | name | vol | | status | available | +-------------+-----------+ [root@osp10-1 ~(keystone_admin)]# sudo mysql -e "select volume_id, instance_uuid, id, attach_status, mountpoint, updated_at, deleted from cinder.volume_attachment where volume_id='999d5bec-5573-4e43-824d-1569a06999d7'" +--------------------------------------+--------------------------------------+--------------------------------------+---------------+------------+---------------------+---------+ | volume_id | instance_uuid | id | attach_status | mountpoint | updated_at | deleted | +--------------------------------------+--------------------------------------+--------------------------------------+---------------+------------+---------------------+---------+ | 999d5bec-5573-4e43-824d-1569a06999d7 | 1496a79c-4d67-4449-ae79-0491be47df57 | ea5d5fad-7153-4d3d-b23e-996b3a7fb934 | detached | /dev/vdb | 2020-10-12 09:12:21 | 1 | +--------------------------------------+--------------------------------------+--------------------------------------+---------------+------------+---------------------+---------+ [root@osp10-1 ~(keystone_admin)]# sudo mysql -e "select deleted_at, id, volume_id, instance_uuid, connection_info from nova.block_device_mapping where volume_id='999d5bec-5573-4e43-824d-1569a06999d7'" | NULL | 6 | 999d5bec-5573-4e43-824d-1569a06999d7 | 1496a79c-4d67-4449-ae79-0491be47df57 | {"driver_volume_type": "iscsi", "connector": {"platform": "x86_64", "host": "osp10-1.gsslab.pnq2.redhat.com", "do_local_attach": false, "ip": "10.74.129.7", "os_type": "linux2", "multipath": false, "initiator": "iqn.1994-05.com.redhat:cbea46b5675b"}, "serial": "999d5bec-5573-4e43-824d-1569a06999d7", "data": {"access_mode": "rw", "target_discovered": false, "encrypted": false, "qos_specs": null, "target_iqn": "iqn.2010-10.org.openstack:volume-999d5bec-5573-4e43-824d-1569a06999d7", "target_portal": "10.74.129.7:3260", "volume_id": "999d5bec-5573-4e43-824d-1569a06999d7", "target_lun": 0, "device_path": "/dev/disk/by-path/ip-10.74.129.7:3260-iscsi-iqn.2010-10.org.openstack:volume-999d5bec-5573-4e43-824d-1569a06999d7-lun-0", "auth_password": "wy5mrZ4hX6JKiYma", "auth_username": "MvewBFkLBv7MtFhMBkUe", "auth_method": "CHAP"}} | # openstack (openstack) server add volume vm2 vol (openstack) server show -c name -c os-extended-volumes:volumes_attached -c OS-EXT-STS:power_state -c OS-EXT-STS:vm_state vm2 +--------------------------------------+----------------------------------------------------+ | Field | Value | +--------------------------------------+----------------------------------------------------+ | OS-EXT-STS:power_state | Running | | OS-EXT-STS:vm_state | active | | name | vm2 | | os-extended-volumes:volumes_attached | [{u'id': u'999d5bec-5573-4e43-824d-1569a06999d7'}] | +--------------------------------------+----------------------------------------------------+ (openstack) server show -c name -c os-extended-volumes:volumes_attached -c OS-EXT-STS:power_state -c OS-EXT-STS:vm_state vm1 +--------------------------------------+----------------------------------------------------+ | Field | Value | +--------------------------------------+----------------------------------------------------+ | OS-EXT-STS:power_state | Running | | OS-EXT-STS:vm_state | active | | name | vm1 | | os-extended-volumes:volumes_attached | [{u'id': u'999d5bec-5573-4e43-824d-1569a06999d7'}] | +--------------------------------------+----------------------------------------------------+ (openstack) volume list +--------------------------------------+--------------+--------+------+------------------------------+ | ID | Display Name | Status | Size | Attached to | +--------------------------------------+--------------+--------+------+------------------------------+ | 999d5bec-5573-4e43-824d-1569a06999d7 | vol | in-use | 1 | Attached to vm2 on /dev/vdb | +--------------------------------------+--------------+--------+------+------------------------------+ # cinder reset-state --state available --attach-status detached vol [root@osp10-1 ~(keystone_admin)]# openstack volume show -c name -c attachments -c status vol +-------------+-----------+ | Field | Value | +-------------+-----------+ | attachments | [] | | name | vol | | status | available | +-------------+-----------+ [root@osp10-1 ~(keystone_admin)]# sudo mysql -e "select volume_id, instance_uuid, id, attach_status, mountpoint, updated_at, deleted from cinder.volume_attachment where volume_id='999d5bec-5573-4e43-824d-1569a06999d7'" +--------------------------------------+--------------------------------------+--------------------------------------+---------------+------------+---------------------+---------+ | volume_id | instance_uuid | id | attach_status | mountpoint | updated_at | deleted | +--------------------------------------+--------------------------------------+--------------------------------------+---------------+------------+---------------------+---------+ | 999d5bec-5573-4e43-824d-1569a06999d7 | 22eb565d-6a91-45e0-80e2-0613b54b92ce | 5726183c-a244-4676-ac4d-8d9fcda9685f | detached | /dev/vdb | 2020-10-12 09:21:04 | 1 | | 999d5bec-5573-4e43-824d-1569a06999d7 | 1496a79c-4d67-4449-ae79-0491be47df57 | ea5d5fad-7153-4d3d-b23e-996b3a7fb934 | detached | /dev/vdb | 2020-10-12 09:12:21 | 1 | +--------------------------------------+--------------------------------------+--------------------------------------+---------------+------------+---------------------+---------+ [root@osp10-1 ~(keystone_admin)]# sudo mysql -e "select deleted_at, id, volume_id, instance_uuid, connection_info from nova.block_device_mapping where volume_id='999d5bec-5573-4e43-824d-1569a06999d7'" | NULL | 6 | 999d5bec-5573-4e43-824d-1569a06999d7 | 1496a79c-4d67-4449-ae79-0491be47df57 | {"driver_volume_type": "iscsi", "connector": {"platform": "x86_64", "host": "osp10-1.gsslab.pnq2.redhat.com", "do_local_attach": false, "ip": "10.74.129.7", "os_type": "linux2", "multipath": false, "initiator": "iqn.1994-05.com.redhat:cbea46b5675b"}, "serial": "999d5bec-5573-4e43-824d-1569a06999d7", "data": {"access_mode": "rw", "target_discovered": false, "encrypted": false, "qos_specs": null, "target_iqn": "iqn.2010-10.org.openstack:volume-999d5bec-5573-4e43-824d-1569a06999d7", "target_portal": "10.74.129.7:3260", "volume_id": "999d5bec-5573-4e43-824d-1569a06999d7", "target_lun": 0, "device_path": "/dev/disk/by-path/ip-10.74.129.7:3260-iscsi-iqn.2010-10.org.openstack:volume-999d5bec-5573-4e43-824d-1569a06999d7-lun-0", "auth_password": "wy5mrZ4hX6JKiYma", "auth_username": "MvewBFkLBv7MtFhMBkUe", "auth_method": "CHAP"}} | | NULL | 7 | 999d5bec-5573-4e43-824d-1569a06999d7 | 22eb565d-6a91-45e0-80e2-0613b54b92ce | {"driver_volume_type": "iscsi", "connector": {"platform": "x86_64", "host": "osp10-1.gsslab.pnq2.redhat.com", "do_local_attach": false, "ip": "10.74.129.7", "os_type": "linux2", "multipath": false, "initiator": "iqn.1994-05.com.redhat:cbea46b5675b"}, "serial": "999d5bec-5573-4e43-824d-1569a06999d7", "data": {"access_mode": "rw", "target_discovered": false, "encrypted": false, "qos_specs": null, "target_iqn": "iqn.2010-10.org.openstack:volume-999d5bec-5573-4e43-824d-1569a06999d7", "target_portal": "10.74.129.7:3260", "volume_id": "999d5bec-5573-4e43-824d-1569a06999d7", "target_lun": 0, "device_path": "/dev/disk/by-path/ip-10.74.129.7:3260-iscsi-iqn.2010-10.org.openstack:volume-999d5bec-5573-4e43-824d-1569a06999d7-lun-0", "auth_password": "wy5mrZ4hX6JKiYma", "auth_username": "MvewBFkLBv7MtFhMBkUe", "auth_method": "CHAP"}} | # openstack (openstack) server add volume vm1 vol (openstack) server show -c name -c os-extended-volumes:volumes_attached -c OS-EXT-STS:power_state -c OS-EXT-STS:vm_state vm1 +--------------------------------------+------------------------------------------------------------------------------------------------------+ | Field | Value | +--------------------------------------+------------------------------------------------------------------------------------------------------+ | OS-EXT-STS:power_state | Running | | OS-EXT-STS:vm_state | active | | name | vm1 | | os-extended-volumes:volumes_attached | [{u'id': u'999d5bec-5573-4e43-824d-1569a06999d7'}, {u'id': u'999d5bec-5573-4e43-824d-1569a06999d7'}] | +--------------------------------------+------------------------------------------------------------------------------------------------------+ (openstack) server show -c name -c os-extended-volumes:volumes_attached -c OS-EXT-STS:power_state -c OS-EXT-STS:vm_state vm2 +--------------------------------------+----------------------------------------------------+ | Field | Value | +--------------------------------------+----------------------------------------------------+ | OS-EXT-STS:power_state | Running | | OS-EXT-STS:vm_state | active | | name | vm2 | | os-extended-volumes:volumes_attached | [{u'id': u'999d5bec-5573-4e43-824d-1569a06999d7'}] | +--------------------------------------+----------------------------------------------------+ (openstack) volume list +--------------------------------------+--------------+--------+------+------------------------------+ | ID | Display Name | Status | Size | Attached to | +--------------------------------------+--------------+--------+------+------------------------------+ | 999d5bec-5573-4e43-824d-1569a06999d7 | vol | in-use | 1 | Attached to vm1 on /dev/vdc | +--------------------------------------+--------------+--------+------+------------------------------+ (openstack) [root@osp10-1 ~(keystone_admin)]# sudo mysql -e "select volume_id, instance_uuid, id, attach_status, mountpoint, updated_at, deleted from cinder.volume_attachment where volume_id='999d5bec-5573-4e43-824d-1569a06999d7'" +--------------------------------------+--------------------------------------+--------------------------------------+---------------+------------+---------------------+---------+ | volume_id | instance_uuid | id | attach_status | mountpoint | updated_at | deleted | +--------------------------------------+--------------------------------------+--------------------------------------+---------------+------------+---------------------+---------+ | 999d5bec-5573-4e43-824d-1569a06999d7 | 22eb565d-6a91-45e0-80e2-0613b54b92ce | 5726183c-a244-4676-ac4d-8d9fcda9685f | detached | /dev/vdb | 2020-10-12 09:21:04 | 1 | | 999d5bec-5573-4e43-824d-1569a06999d7 | 1496a79c-4d67-4449-ae79-0491be47df57 | c7fc6c4c-7541-4e57-a491-0a3bfdf1a4c7 | attached | /dev/vdc | 2020-10-12 09:23:01 | 0 | | 999d5bec-5573-4e43-824d-1569a06999d7 | 1496a79c-4d67-4449-ae79-0491be47df57 | ea5d5fad-7153-4d3d-b23e-996b3a7fb934 | detached | /dev/vdb | 2020-10-12 09:12:21 | 1 | +--------------------------------------+--------------------------------------+--------------------------------------+---------------+------------+---------------------+---------+ [root@osp10-1 ~(keystone_admin)]# sudo mysql -e "select deleted_at, id, volume_id, instance_uuid, connection_info from nova.block_device_mapping where volume_id='999d5bec-5573-4e43-824d-1569a06999d7'" | NULL | 6 | 999d5bec-5573-4e43-824d-1569a06999d7 | 1496a79c-4d67-4449-ae79-0491be47df57 | {"driver_volume_type": "iscsi", "connector": {"platform": "x86_64", "host": "osp10-1.gsslab.pnq2.redhat.com", "do_local_attach": false, "ip": "10.74.129.7", "os_type": "linux2", "multipath": false, "initiator": "iqn.1994-05.com.redhat:cbea46b5675b"}, "serial": "999d5bec-5573-4e43-824d-1569a06999d7", "data": {"access_mode": "rw", "target_discovered": false, "encrypted": false, "qos_specs": null, "target_iqn": "iqn.2010-10.org.openstack:volume-999d5bec-5573-4e43-824d-1569a06999d7", "target_portal": "10.74.129.7:3260", "volume_id": "999d5bec-5573-4e43-824d-1569a06999d7", "target_lun": 0, "device_path": "/dev/disk/by-path/ip-10.74.129.7:3260-iscsi-iqn.2010-10.org.openstack:volume-999d5bec-5573-4e43-824d-1569a06999d7-lun-0", "auth_password": "wy5mrZ4hX6JKiYma", "auth_username": "MvewBFkLBv7MtFhMBkUe", "auth_method": "CHAP"}} | | NULL | 7 | 999d5bec-5573-4e43-824d-1569a06999d7 | 22eb565d-6a91-45e0-80e2-0613b54b92ce | {"driver_volume_type": "iscsi", "connector": {"platform": "x86_64", "host": "osp10-1.gsslab.pnq2.redhat.com", "do_local_attach": false, "ip": "10.74.129.7", "os_type": "linux2", "multipath": false, "initiator": "iqn.1994-05.com.redhat:cbea46b5675b"}, "serial": "999d5bec-5573-4e43-824d-1569a06999d7", "data": {"access_mode": "rw", "target_discovered": false, "encrypted": false, "qos_specs": null, "target_iqn": "iqn.2010-10.org.openstack:volume-999d5bec-5573-4e43-824d-1569a06999d7", "target_portal": "10.74.129.7:3260", "volume_id": "999d5bec-5573-4e43-824d-1569a06999d7", "target_lun": 0, "device_path": "/dev/disk/by-path/ip-10.74.129.7:3260-iscsi-iqn.2010-10.org.openstack:volume-999d5bec-5573-4e43-824d-1569a06999d7-lun-0", "auth_password": "wy5mrZ4hX6JKiYma", "auth_username": "MvewBFkLBv7MtFhMBkUe", "auth_method": "CHAP"}} | | NULL | 8 | 999d5bec-5573-4e43-824d-1569a06999d7 | 1496a79c-4d67-4449-ae79-0491be47df57 | {"driver_volume_type": "iscsi", "connector": {"platform": "x86_64", "host": "osp10-1.gsslab.pnq2.redhat.com", "do_local_attach": false, "ip": "10.74.129.7", "os_type": "linux2", "multipath": false, "initiator": "iqn.1994-05.com.redhat:cbea46b5675b"}, "serial": "999d5bec-5573-4e43-824d-1569a06999d7", "data": {"access_mode": "rw", "target_discovered": false, "encrypted": false, "qos_specs": null, "target_iqn": "iqn.2010-10.org.openstack:volume-999d5bec-5573-4e43-824d-1569a06999d7", "target_portal": "10.74.129.7:3260", "volume_id": "999d5bec-5573-4e43-824d-1569a06999d7", "target_lun": 0, "device_path": "/dev/disk/by-path/ip-10.74.129.7:3260-iscsi-iqn.2010-10.org.openstack:volume-999d5bec-5573-4e43-824d-1569a06999d7-lun-0", "auth_password": "wy5mrZ4hX6JKiYma", "auth_username": "MvewBFkLBv7MtFhMBkUe", "auth_method": "CHAP"}} | ~~~
Do we have any workaround for when this issue happens without modifying the database entries?
Why do you use cinder reset-state ? It changes the status in the database, but it doesn't detach the volume. Does it work with $ nova volume-detach <server> <volume> ?
Please describe the actual problem the customer is trying to solve. Resetting cinder's attachment status does not trigger anything in nova (nova will still consider the volume to be attached), and the behavior you describe in the BZ description is the expected behavior.
The ticket was opened after they tried the cinder reset-state. ~~~ It changes the status in the database, but it doesn't detach the volume. ~~~ ~~~ the behavior you describe in the BZ description is the expected behavior ~~~ Shouldn't we have some validation that wouldn't allow have the same volume attached?
(In reply to Andre from comment #5) > The ticket was opened after they tried the cinder reset-state. But why is the customer running this command? It's a "sharp knife" tool that can be judiciously used in situations where cinder's and nova's volume attachment status gets out of sync. But it does NOT control nova's attachment state. > ~~~ > It changes the status in the database, but it doesn't detach the volume. > ~~~ > > ~~~ > the behavior you describe in the BZ description is the expected behavior > ~~~ > > Shouldn't we have some validation that wouldn't allow have the same volume > attached? Sure, but that's nova's job. Remember that cinder is merely responsible for maintaining the state information supplied by nova. You cleared the needinfo, but we still have no info on the root problem the customer is trying to solve.
Closing as NOTABUG. As highlighted by my colleagues working on openstack-cinder, as written c#0 isn't a bug but an operator error. When using the os-volume_attachments [1] API via the `openstack server add volume` or `nova volume-attach` commands users should *only* use the `openstack server remove volume` or `nova volume-detach` commands to remove the volume. The use of `cinder reset-state` should be reserved for situations where direct manipulation of the Cinder database is required *after* the volume is detached within Nova. Whatever the reason was for the customer using this in the first place needs to be written up in a new bug, preferably with an example `openstack server event list $instance_uuid` provided showing the sequence of events that left them needing to detach the volume this way. [1] https://docs.openstack.org/api-ref/compute/#servers-with-volume-attachments-servers-os-volume-attachments