1794249 – cinder retype(with migration) from unencrypted to encrypted volume type in Ceph backend fails because of encryption header takes more space.

This bug has been migrated to another issue tracking site. It has been closed here and may no longer be being monitored.

If you would like to get updates for this issue, or to participate in it, you may do so at Red Hat Issue Tracker .

Bug 1794249 - cinder retype(with migration) from unencrypted to encrypted volume type in Ceph backend fails because of encryption header takes more space.

Summary: cinder retype(with migration) from unencrypted to encrypted volume type in Ce...

Keywords:
Status:	CLOSED MIGRATED
Alias:	None
Product:	Red Hat OpenStack
Classification:	Red Hat
Component:	python-os-brick
Sub Component:
Version:	17.0 (Wallaby)
Hardware:	x86_64
OS:	Linux
Priority:	high
Severity:	medium
Target Milestone:	---
Target Release:	---
Assignee:	Cinder Bugs List
QA Contact:	Evelina Shames
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:	1931621
TreeView+	depends on / blocked

Reported:	2020-01-23 03:50 UTC by Shatadru Bandyopadhyay
Modified:	2024-01-05 09:36 UTC (History)
CC List:	12 users (show)
Fixed In Version:
Doc Type:	If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:	2024-01-05 09:36:16 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)

Links
System	ID	Priority	Status	Summary	Last Updated
Launchpad	1687880	None	None	None	2020-07-30 05:30:47 UTC
OpenStack gerrit	749991	None	MERGED	Sizing encrypted volumes	2022-06-16 22:05:01 UTC
Red Hat Issue Tracker	OSP-2617	None	None	None	2024-01-05 09:36:15 UTC
Red Hat Issue Tracker	OSP-31050	None	None	None	2024-01-05 09:36:46 UTC

Internal Links: 1877908

Description Shatadru Bandyopadhyay 2020-01-23 03:50:06 UTC

Description of problem:
retype from non encrypted to encrypted volume in ceph backend fails

Version-Release number of selected component (if applicable):
RHOSP13 latest minor version

docker exec -it $(docker ps|grep -i volume |awk '{print $1}' )  bash
()[root@controller-0 /]# rpm -qa|grep -i cinder
python-cinder-12.0.8-3.el7ost.noarch
puppet-cinder-12.4.1-5.el7ost.noarch
openstack-cinder-12.0.8-3.el7ost.noarch
python2-cinderclient-3.5.0-1.el7ost.noarch
()[root@controller-0 /]# rpm -qa|grep -i os-brick
python2-os-brick-2.3.8-1.el7ost.noarch


How reproducible:
Always

Steps to Reproduce:
1. deploy RHOSP with cinder backend ceph, deploy barbican
2. create volume in ceph backend
3. create volume type for encryption

   openstack volume type create --encryption-provider nova.volume.encryptors.luks.LuksEncryptor --encryption-cipher aes-xts-plain64 --encryption-key-size 256 --encryption-control-location front-end LuksEncryptor-Template-256

4. try to retype 

   #cinder retype test-encrypt LuksEncryptor-Template-256 --migration-policy on-demand



cinder type-list
+--------------------------------------+----------------------------+-------------+-----------+
| ID                                   | Name                       | Description | Is_Public |
+--------------------------------------+----------------------------+-------------+-----------+
| 097fb345-e7af-4d41-bb4b-1c81ddba36f0 | tripleo_lvm                | -           | True      |
| 151cec8d-7592-48b9-809d-32f9cad9b85c | LuksEncryptor-Template-256 | -           | True      |
| 1bd2cf2a-0a98-454c-abd6-c929fccaf461 | tripleo_hpelefthand        | -           | True      |
| 1f5cf5ff-4ded-4283-9837-0fa9f2cc1999 | tripleo_nfs                | -           | True      |
| 535c37f4-a1c0-4fd6-a257-d4adf36eff4d | tripleo_ceph               | -           | True      |
+--------------------------------------+----------------------------+-------------+-----------+

openstack volume show test-encrypt
+--------------------------------+--------------------------------------+
| Field                          | Value                                |
+--------------------------------+--------------------------------------+
| attachments                    | []                                   |
| availability_zone              | nova                                 |
| bootable                       | false                                |
| consistencygroup_id            | None                                 |
| created_at                     | 2020-01-21T14:08:41.000000           |
| description                    | None                                 |
| encrypted                      | False                                |
| id                             | 119b36f2-1410-4a42-b04a-7a78769ed2be |
| migration_status               | error <<-- after retype failed       |
| multiattach                    | False                                |
| name                           | test-encrypt                         |
| os-vol-host-attr:host          | hostgroup@tripleo_ceph#tripleo_ceph  |
| os-vol-mig-status-attr:migstat | error                                |
| os-vol-mig-status-attr:name_id | None                                 |
| os-vol-tenant-attr:tenant_id   | a578afa9dad2434182d053c07ffba4b8     |
| properties                     |                                      |
| replication_status             | None                                 |
| size                           | 1                                    |
| snapshot_id                    | None                                 |
| source_volid                   | None                                 |
| status                         | available                            |
| type                           | tripleo_ceph                         |
| updated_at                     | 2020-01-21T14:44:03.000000           |
| user_id                        | f9ca5a979d024d58aba8804b29f9c804     |
+--------------------------------+--------------------------------------+


Actual results:
fails with traceback

2020-01-21 14:26:04.423 58 INFO cinder.volume.manager [req-31e8a87c-71a8-4e43-b0fb-c00632db5f02 f9ca5a979d024d58aba8804b29f9c804 a578afa9dad2434182d053c07ffba4b8 - default default] Remove volume export completed successfully.
2020-01-21 14:26:04.424 58 ERROR cinder.volume.manager [req-31e8a87c-71a8-4e43-b0fb-c00632db5f02 f9ca5a979d024d58aba8804b29f9c804 a578afa9dad2434182d053c07ffba4b8 - default default] Failed to copy volume 119b36f2-1410-4a42-b04a-7a78769ed2
be to 4bd9f446-23d0-4f9c-b138-ec9116203902: AttributeError: 'RBDVolumeIOWrapper' object has no attribute 'rfind'
2020-01-21 14:26:04.424 58 ERROR cinder.volume.manager Traceback (most recent call last):
2020-01-21 14:26:04.424 58 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2164, in _migrate_volume_generic
2020-01-21 14:26:04.424 58 ERROR cinder.volume.manager     self._copy_volume_data(ctxt, volume, new_volume, remote='dest')
2020-01-21 14:26:04.424 58 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2044, in _copy_volume_data
2020-01-21 14:26:04.424 58 ERROR cinder.volume.manager     attach_encryptor=attach_encryptor)
2020-01-21 14:26:04.424 58 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1991, in _attach_volume
2020-01-21 14:26:04.424 58 ERROR cinder.volume.manager     force=True)
2020-01-21 14:26:04.424 58 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2020-01-21 14:26:04.424 58 ERROR cinder.volume.manager     self.force_reraise()
2020-01-21 14:26:04.424 58 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2020-01-21 14:26:04.424 58 ERROR cinder.volume.manager     six.reraise(self.type_, self.value, self.tb)
2020-01-21 14:26:04.424 58 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1985, in _attach_volume
2020-01-21 14:26:04.424 58 ERROR cinder.volume.manager     encryption)
2020-01-21 14:26:04.424 58 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 535, in brick_attach_volume_encryptor
2020-01-21 14:26:04.424 58 ERROR cinder.volume.manager     **encryption)
2020-01-21 14:26:04.424 58 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 527, in brick_get_encryptor
2020-01-21 14:26:04.424 58 ERROR cinder.volume.manager     *args, **kwargs)
2020-01-21 14:26:04.424 58 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/os_brick/encryptors/__init__.py", line 91, in get_volume_encryptor
2020-01-21 14:26:04.424 58 ERROR cinder.volume.manager     **kwargs)
2020-01-21 14:26:04.424 58 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/oslo_utils/importutils.py", line 44, in import_object
2020-01-21 14:26:04.424 58 ERROR cinder.volume.manager     return import_class(import_str)(*args, **kwargs)
2020-01-21 14:26:04.424 58 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/os_brick/encryptors/luks.py", line 61, in __init__
2020-01-21 14:26:04.424 58 ERROR cinder.volume.manager     *args, **kwargs)
2020-01-21 14:26:04.424 58 ERROR cinder.volume.manager   File "/usr/lib/python2.7/site-packages/os_brick/encryptors/cryptsetup.py", line 60, in __init__
2020-01-21 14:26:04.424 58 ERROR cinder.volume.manager     self.dev_name = 'crypt-%s' % os.path.basename(self.symlink_path)
2020-01-21 14:26:04.424 58 ERROR cinder.volume.manager   File "/usr/lib64/python2.7/posixpath.py", line 121, in basename
2020-01-21 14:26:04.424 58 ERROR cinder.volume.manager     i = p.rfind('/') + 1
2020-01-21 14:26:04.424 58 ERROR cinder.volume.manager AttributeError: 'RBDVolumeIOWrapper' object has no attribute 'rfind'
2020-01-21 14:26:04.424 58 ERROR cinder.volume.manager 
2020-01-21 14:26:04.474 58 DEBUG cinder.coordination [req-31e8a87c-71a8-4e43-b0fb-c00632db5f02 f9ca5a979d024d58aba8804b29f9c804 a578afa9dad2434182d053c07ffba4b8 - default default] Lock "/var/lib/cinder/cinder-4bd9f446-23d0-4f9c-b138-ec9116203902-delete_volume" acquired by "delete_volume" :: waited 0.001s _synchronized /usr/lib/python2.7/site-packages/cinder/coordination.py:150
2020-01-21 14:26:04.584 58 DEBUG cinder.volume.drivers.rbd [req-31e8a87c-71a8-4e43-b0fb-c00632db5f02 f9ca5a979d024d58aba8804b29f9c804 a578afa9dad2434182d053c07ffba4b8 - default default] connecting to ceph (timeout=-1). _do_conn /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:337
2020-01-21 14:26:04.675 58 DEBUG cinder.volume.drivers.rbd [req-31e8a87c-71a8-4e43-b0fb-c00632db5f02 f9ca5a979d024d58aba8804b29f9c804 a578afa9dad2434182d053c07ffba4b8 - default default] volume has no backup snaps _delete_backup_snaps /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:950
2020-01-21 14:26:04.677 58 DEBUG cinder.volume.drivers.rbd [req-31e8a87c-71a8-4e43-b0fb-c00632db5f02 f9ca5a979d024d58aba8804b29f9c804 a578afa9dad2434182d053c07ffba4b8 - default default] Volume volume-4bd9f446-23d0-4f9c-b138-ec9116203902 is not a clone. _get_clone_info /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:973
2020-01-21 14:26:04.682 58 DEBUG cinder.volume.drivers.rbd [req-31e8a87c-71a8-4e43-b0fb-c00632db5f02 f9ca5a979d024d58aba8804b29f9c804 a578afa9dad2434182d053c07ffba4b8 - default default] deleting rbd volume volume-4bd9f446-23d0-4f9c-b138-ec9116203902 delete_volume /usr/lib/python2.7/site-packages/cinder/volume/drivers/rbd.py:1071
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server [req-31e8a87c-71a8-4e43-b0fb-c00632db5f02 f9ca5a979d024d58aba8804b29f9c804 a578afa9dad2434182d053c07ffba4b8 - default default] Exception during message handling: AttributeError: 'RBDVolumeIOWrapper' object has no attribute 'rfind'
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 166, in _process_incoming
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 220, in dispatch
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 190, in _do_dispatch
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2826, in retype
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server     new_reservations, status_update)
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server     self.force_reraise()
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2822, in retype
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server     new_type_id=new_type_id)
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2431, in migrate_volume
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server     volume.save()
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server     self.force_reraise()
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2424, in migrate_volume
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server     self._migrate_volume_generic(ctxt, volume, host, new_type_id)
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2186, in _migrate_volume_generic
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server     new_volume)
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server     self.force_reraise()
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2164, in _migrate_volume_generic
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server     self._copy_volume_data(ctxt, volume, new_volume, remote='dest')
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2044, in _copy_volume_data
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server     attach_encryptor=attach_encryptor)
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1991, in _attach_volume
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server     force=True)
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server     self.force_reraise()
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in 2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2164, in _migrate_volume_generic
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server     self._copy_volume_data(ctxt, volume, new_volume, remote='dest')
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 2044, in _copy_volume_data
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server     attach_encryptor=attach_encryptor)
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1991, in _attach_volume
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server     force=True)
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server     self.force_reraise()
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/volume/manager.py", line 1985, in _attach_volume
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server     encryption)
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 535, in brick_attach_volume_encryptor
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server     **encryption)
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/cinder/utils.py", line 527, in brick_get_encryptor
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server     *args, **kwargs)
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/os_brick/encryptors/__init__.py", line 91, in get_volume_encryptor
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server     **kwargs)
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/oslo_utils/importutils.py", line 44, in import_object
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server     return import_class(import_str)(*args, **kwargs)
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/os_brick/encryptors/luks.py", line 61, in __init__
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server     *args, **kwargs)
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server   File "/usr/lib/python2.7/site-packages/os_brick/encryptors/cryptsetup.py", line 60, in __init__
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server     self.dev_name = 'crypt-%s' % os.path.basename(self.symlink_path)
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server   File "/usr/lib64/python2.7/posixpath.py", line 121, in basename
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server     i = p.rfind('/') + 1
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server AttributeError: 'RBDVolumeIOWrapper' object has no attribute 'rfind'
2020-01-21 14:26:04.738 58 ERROR oslo_messaging.rpc.server 
2020-01-21 14:26:04.799 58 INFO cinder.volume.manager [req-31e8a87c-71a8-4e43-b0fb-c00632db5f02 f9ca5a979d024d58aba8804b29f9c804 a578afa9dad2434182d053c07ffba4b8 - default default] Deleted volume successfully.
2020-01-21 14:26:04.800 58 DEBUG cinder.coordination [req-31e8a87c-71a8-4e43-b0fb-c00632db5f02 f9ca5a979d024d58aba8804b29f9c804 a578afa9dad2434182d053c07ffba4b8 - default default] Lock "/var/lib/cinder/cinder-4bd9f446-23d0-4f9c-b138-ec9116203902-delete_volume" released by "delete_volume" :: held 0.326s _synchronized /usr/lib/python2.7/site-packages/cinder/coordination.py:162
force_reraise


Expected results:

re - type should complete 

Additional info:

Comment 1 Shatadru Bandyopadhyay 2020-01-23 04:05:11 UTC

encrypted volume creation in ceph backend works fine. 

(overcloud) [stack@undercloud-0 ~]$ openstack volume create --size 1 --type LuksEncryptor-Template-256 'Encrypted-Test-Volume'
+---------------------+--------------------------------------+
| Field               | Value                                |
+---------------------+--------------------------------------+
| attachments         | []                                   |
| availability_zone   | nova                                 |
| bootable            | false                                |
| consistencygroup_id | None                                 |
| created_at          | 2020-01-23T04:00:00.000000           |
| description         | None                                 |
| encrypted           | True                                 |
| id                  | 300e44af-bd71-4bf5-b0d4-aa97928892c5 |
| migration_status    | None                                 |
| multiattach         | False                                |
| name                | Encrypted-Test-Volume                |
| properties          |                                      |
| replication_status  | None                                 |
| size                | 1                                    |
| snapshot_id         | None                                 |
| source_volid        | None                                 |
| status              | creating                             |
| type                | LuksEncryptor-Template-256           |
| updated_at          | None                                 |
| user_id             | f9ca5a979d024d58aba8804b29f9c804     |
+---------------------+--------------------------------------+



(overcloud) [stack@undercloud-0 ~]$ openstack volume show 'Encrypted-Test-Volume'
+--------------------------------+--------------------------------------+
| Field                          | Value                                |
+--------------------------------+--------------------------------------+
| attachments                    | []                                   |
| availability_zone              | nova                                 |
| bootable                       | false                                |
| consistencygroup_id            | None                                 |
| created_at                     | 2020-01-23T04:00:00.000000           |
| description                    | None                                 |
| encrypted                      | True                                 |
| id                             | 300e44af-bd71-4bf5-b0d4-aa97928892c5 |
| migration_status               | None                                 |
| multiattach                    | False                                |
| name                           | Encrypted-Test-Volume                |
| os-vol-host-attr:host          | hostgroup@tripleo_ceph#tripleo_ceph  |
| os-vol-mig-status-attr:migstat | None                                 |
| os-vol-mig-status-attr:name_id | None                                 |
| os-vol-tenant-attr:tenant_id   | a578afa9dad2434182d053c07ffba4b8     |
| properties                     |                                      |
| replication_status             | None                                 |
| size                           | 1                                    |
| snapshot_id                    | None                                 |
| source_volid                   | None                                 |
| status                         | available                            |
| type                           | LuksEncryptor-Template-256           |
| updated_at                     | 2020-01-23T04:00:19.000000           |
| user_id                        | f9ca5a979d024d58aba8804b29f9c804     |
+--------------------------------+--------------------------------------+


I have set volume_backend_name='tripleo_ceph' for `LuksEncryptor-Template-256` type. 

(overcloud) [stack@undercloud-0 ~]$ openstack volume type show LuksEncryptor-Template-256
+--------------------+--------------------------------------+
| Field              | Value                                |
+--------------------+--------------------------------------+
| access_project_ids | None                                 |
| description        | None                                 |
| id                 | 151cec8d-7592-48b9-809d-32f9cad9b85c |
| is_public          | True                                 |
| name               | LuksEncryptor-Template-256           |
| properties         | volume_backend_name='tripleo_ceph'   |
| qos_specs_id       | None                                 |
+--------------------+--------------------------------------+


openstack secret list
+------------------------------------------------------------------------+------+---------------------------+--------+-------------------------------------------+-----------+------------+-------------+------+------------+
| Secret href                                                            | Name | Created                   | Status | Content types                             | Algorithm | Bit length | Secret type | Mode | Expiration |
+------------------------------------------------------------------------+------+---------------------------+--------+-------------------------------------------+-----------+------------+-------------+------+------------+
| http://10.0.0.103:9311/v1/secrets/11b71833-01dc-4177-8f2a-844a0666f77c | None | 2020-01-23T03:59:58+00:00 | ACTIVE | {u'default': u'application/octet-stream'} | aes       |        256 | symmetric   | None | None       |
| http://10.0.0.103:9311/v1/secrets/1b5b6150-2db0-416f-a44c-fdc33924892f | None | 2020-01-21T14:17:16+00:00 | ACTIVE | {u'default': u'application/octet-stream'} | aes       |        256 | symmetric   | None | None       |
| http://10.0.0.103:9311/v1/secrets/5aa0fe07-1a23-4e92-87f2-8268b6252ea8 | None | 2020-01-21T14:22:20+00:00 | ACTIVE | {u'default': u'application/octet-stream'} | aes       |        256 | symmetric   | None | None       |
| http://10.0.0.103:9311/v1/secrets/7128a241-dc3d-4e00-91ca-4cb82b333f9b | None | 2020-01-21T14:35:19+00:00 | ACTIVE | {u'default': u'application/octet-stream'} | aes       |        256 | symmetric   | None | None       |
| http://10.0.0.103:9311/v1/secrets/97ff5917-81c5-4eaf-a715-7f5c16827fd8 | None | 2020-01-21T14:25:27+00:00 | ACTIVE | {u'default': u'application/octet-stream'} | aes       |        256 | symmetric   | None | None       |
| http://10.0.0.103:9311/v1/secrets/bc6c77ac-48fc-417f-9979-34214f733dd0 | None | 2020-01-21T14:43:46+00:00 | ACTIVE | {u'default': u'application/octet-stream'} | aes       |        256 | symmetric   | None | None       |
+------------------------------------------------------------------------+------+---------------------------+--------+-------------------------------------------+-----------+------------+-------------+------+------------+

Comment 2 Shatadru Bandyopadhyay 2020-01-23 13:39:25 UTC

yet to narrow down but I see the 'connection_info' is wrong which us being passed to os.path.basename

from log :
~~~
2020-01-23 13:31:21.488 714897 DEBUG os_brick.initiator.connectors.rbd [req-8700d37b-b7ee-4d2d-b330-10edfe915e7c f9ca5a979d024d58aba8804b29f9c804 a578afa9dad2434182d053c07ffba4b8 - default default] ==> disconnect_volume: call u"{'connection_properties': {u'secret_type': u'ceph', 'device_path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7fd3e710da90>, u'name': u'volumes/volume-872ad53b-3ca7-4b99-a913-4a1313a3715b', u'encrypted': True, u'keyring': None, u'cluster_name': u'ceph', u'secret_uuid': None, u'qos_specs': None, u'auth_enabled': True, u'hosts': [u'172.17.3.15', u'172.17.3.27', u'172.17.3.32'], u'volume_id': u'872ad53b-3ca7-4b99-a913-4a1313a3715b', u'discard': True, u'access_mode': u'rw', u'auth_username': u'openstack', u'ports': [u'6789', u'6789', u'6789']}, 'self': <os_brick.initiator.connectors.rbd.RBDConnector object at 0x7fd3e6fa6950>, 'force': True, 'device_info': {'path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7fd3e710da90>}, 'ignore_errors': False}" trace_logging_wrapper /usr/lib/python2.7/site-packages/os_brick/utils.py:146
~~~

Not sure why rbd has a device_path set and that to a object rather than a string.

~~~
'device_path': <os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7fd3e710da90>
~~~

~~~
(Pdb) connection_info['data']['device_path']
<os_brick.initiator.linuxrbd.RBDVolumeIOWrapper object at 0x7fba641df450>
~~~

and this object being passed to os.path.basename which is causing the error.


     57         self.symlink_path = connection_info['data']['device_path']
     58 
     59         # a unique name for the volume -- e.g., the iSCSI participant name
     60         self.dev_name = 'crypt-%s' % os.path.basename(self.symlink_path) < ----

Note You need to log in before you can comment on or make changes to this bug.