The function `delete_volume` of the DELL SC driver was unaware of migration and it did not check that the delete command was sent to the right backend. Consequently, the driver deleted a wrong volume. The problem has been fixed. The driver checks the volume before it deletes it from the configured backend. As a result, the migration completes successfully and the correct volume is deleted.
Description of problem:
We are trying to migrate one cinder volume from one backend to another.
We currently have two enabled backends:
cinder.conf
# A list of backend names to use. These backend names should be backed by a
# unique [CONFIG] group with its options (list value)
#enabled_backends = <None>
enabled_backends = dellfc,dellfcold
default_volume_type = dell_fc
scheduler_driver = cinder.scheduler.filter_scheduler.FilterScheduler
[default]
enforce_multipath_for_image_xfer = true
use_multipath_for_image_xfer = true
[dellfc]
dell_sc_server_folder=openstack
san_password=password123
volume_backend_name=dell_fc
san_login=EnterpriseManager
dell_sc_volume_folder=Volumes/openstack
dell_sc_api_port=3033
volume_driver=cinder.volume.drivers.dell.dell_storagecenter_fc.DellStorageCenterFCDriver
san_ip=10.1.6.34
dell_sc_ssn=83484
image_volume_cache_enabled = True
image_volume_cache_max_size_gb = 0
image_volume_cache_max_count = 0
[dellfcold]
dell_sc_server_folder=openstack
san_password=password123
volume_backend_name=dell_fc_old
san_login=EnterpriseManager
dell_sc_volume_folder=Volumes/openstack
dell_sc_api_port=3033
volume_driver=cinder.volume.drivers.dell.dell_storagecenter_fc.DellStorageCenterFCDriver
san_ip=10.1.6.34
dell_sc_ssn=32989
image_volume_cache_enabled = True
image_volume_cache_max_size_gb = 0
image_volume_cache_max_count = 0
And the corresponding volume types:
[stack@ospd-osp1-cs-01 ~]$ cinder extra-specs-list
+--------------------------------------+-------------+------------------------------------------+
| ID | Name | extra_specs |
+--------------------------------------+-------------+------------------------------------------+
| 277f034b-e745-43ab-9ca9-7d84198da534 | dell_fc | {u'volume_backend_name': u'dell_fc'} |
| 8f7ff652-77f9-45d2-a947-38d4cf632dd8 | dell_fc_old | {u'volume_backend_name': u'dell_fc_old'} |
+--------------------------------------+-------------+------------------------------------------+
Creating and deleting new volumes on both backends succeeds, as well as launching a new instance booting from the volumes (created from a glance image).
The default type is correctly picked up if not specified.
Trying to migrate a volume between the two backends (via retype) fails as cinder is effectively deleting the newly retyped volume at the end of the process.
Checking the disk array logs shows that the new LUN has been created on the target backend, then renamed, then deleted, while on the source backend the LUN is still present.
So it seems that after completing the migration, cinder is issuing the delete command against the wrong backend.
Version-Release number of selected component (if applicable):
openstack-cinder-9.1.4-3.el7ost.noarch Thu Jun 15 18:52:31 2017
puppet-cinder-9.5.0-1.el7ost.noarch Thu Jun 15 18:59:02 2017
python-cinder-9.1.4-3.el7ost.noarch Thu Jun 15 18:51:02 2017
python-cinderclient-1.9.0-6.el7ost.noarch Thu Jun 15 18:26:15 2017
How reproducible:
always
Steps to Reproduce:
1. define two cinder backends (using dellsc drivers)
2. define two cinder types corresponding to the two backends
3. try to retype
Actual results:
retype apparently succeeds but the volume to get deleted is the newly retype one, not the old one
Expected results:
newly retyped volume to survive, old volume to be deleted
Additional info:
The root cause is that the driver is not using the right field to identify the volume, it's using the id instead of the _name_id.
The driver is also not checking the provider_id to confirm that the delete is for the current backend, and that's why it is also deleting the volume from another backend.
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory, and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.
https://access.redhat.com/errata/RHBA-2017:3233
Description of problem: We are trying to migrate one cinder volume from one backend to another. We currently have two enabled backends: cinder.conf # A list of backend names to use. These backend names should be backed by a # unique [CONFIG] group with its options (list value) #enabled_backends = <None> enabled_backends = dellfc,dellfcold default_volume_type = dell_fc scheduler_driver = cinder.scheduler.filter_scheduler.FilterScheduler [default] enforce_multipath_for_image_xfer = true use_multipath_for_image_xfer = true [dellfc] dell_sc_server_folder=openstack san_password=password123 volume_backend_name=dell_fc san_login=EnterpriseManager dell_sc_volume_folder=Volumes/openstack dell_sc_api_port=3033 volume_driver=cinder.volume.drivers.dell.dell_storagecenter_fc.DellStorageCenterFCDriver san_ip=10.1.6.34 dell_sc_ssn=83484 image_volume_cache_enabled = True image_volume_cache_max_size_gb = 0 image_volume_cache_max_count = 0 [dellfcold] dell_sc_server_folder=openstack san_password=password123 volume_backend_name=dell_fc_old san_login=EnterpriseManager dell_sc_volume_folder=Volumes/openstack dell_sc_api_port=3033 volume_driver=cinder.volume.drivers.dell.dell_storagecenter_fc.DellStorageCenterFCDriver san_ip=10.1.6.34 dell_sc_ssn=32989 image_volume_cache_enabled = True image_volume_cache_max_size_gb = 0 image_volume_cache_max_count = 0 And the corresponding volume types: [stack@ospd-osp1-cs-01 ~]$ cinder extra-specs-list +--------------------------------------+-------------+------------------------------------------+ | ID | Name | extra_specs | +--------------------------------------+-------------+------------------------------------------+ | 277f034b-e745-43ab-9ca9-7d84198da534 | dell_fc | {u'volume_backend_name': u'dell_fc'} | | 8f7ff652-77f9-45d2-a947-38d4cf632dd8 | dell_fc_old | {u'volume_backend_name': u'dell_fc_old'} | +--------------------------------------+-------------+------------------------------------------+ Creating and deleting new volumes on both backends succeeds, as well as launching a new instance booting from the volumes (created from a glance image). The default type is correctly picked up if not specified. Trying to migrate a volume between the two backends (via retype) fails as cinder is effectively deleting the newly retyped volume at the end of the process. Checking the disk array logs shows that the new LUN has been created on the target backend, then renamed, then deleted, while on the source backend the LUN is still present. So it seems that after completing the migration, cinder is issuing the delete command against the wrong backend. Version-Release number of selected component (if applicable): openstack-cinder-9.1.4-3.el7ost.noarch Thu Jun 15 18:52:31 2017 puppet-cinder-9.5.0-1.el7ost.noarch Thu Jun 15 18:59:02 2017 python-cinder-9.1.4-3.el7ost.noarch Thu Jun 15 18:51:02 2017 python-cinderclient-1.9.0-6.el7ost.noarch Thu Jun 15 18:26:15 2017 How reproducible: always Steps to Reproduce: 1. define two cinder backends (using dellsc drivers) 2. define two cinder types corresponding to the two backends 3. try to retype Actual results: retype apparently succeeds but the volume to get deleted is the newly retype one, not the old one Expected results: newly retyped volume to survive, old volume to be deleted Additional info: