Bug 2106389
| Summary: | [RHOS-17] AttributeError: 'RBDVolumeIOWrapper' object has no attribute 'startswith' | ||
|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | bkopilov <bkopilov> |
| Component: | python-os-brick | Assignee: | Gorka Eguileor <geguileo> |
| Status: | CLOSED ERRATA | QA Contact: | Tzach Shefi <tshefi> |
| Severity: | high | Docs Contact: | |
| Priority: | high | ||
| Version: | 17.0 (Wallaby) | CC: | apevec, eharney, geguileo, jschluet, lhh, ltoscano, spower |
| Target Milestone: | ga | Keywords: | Regression, Triaged |
| Target Release: | 17.0 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | python-os-brick-4.3.3-0.20220714180755.d09dc9e.el9ost | Doc Type: | No Doc Update |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2022-09-21 12:23:49 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Verified on: python3-os-brick-4.3.3-0.20220715140802.d09dc9e.el9ost.noarch.rpm Comparing the lists of failing test results, the latest build run #39 which includes the above fixed-in vs. Benny's original failing tests results run #34, Confirm there are way less failing tests in general 20 vs. 69, Plus almost all of the encrypted volume tests which initially failed on run#34 are now now passing on #39. Good to verify. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Release of components for Red Hat OpenStack Platform 17.0 (Wallaby)), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2022:6543 |
Description of problem: Latest rhos17, ceph backend. Multiple failures from tempest run related to encrypted volumes. Still under investigation , from cinder volume logs : 2T11:14:04Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestEncryptedCinderVolumes-1159329857',ec2_id=None,encryption_key_id=bd0fda73-a2c2-4415-ae87-9f9819b583e4,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=6796aff7-2711-41a5-bb39-618988f68d60,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='c94c3887b70d4721a11be22c540925a0',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-07-12T11:14:04Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=<?>,source_volid=e9db96d7-835d-4418-b6e3-ae6530ef811e,status='creating',terminated_at=None,updated_at=2022-07-12T11:14:04Z,user_id='32fbace38bb44ef2b10d637cbcc96985',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(5d17ed68-f3b5-4105-871c-75ec97d281a8),volume_type_id=5d17ed68-f3b5-4105-871c-75ec97d281a8)}, 'provides': Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=True,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2022-07-12T11:14:04Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestEncryptedCinderVolumes-1159329857',ec2_id=None,encryption_key_id=bd0fda73-a2c2-4415-ae87-9f9819b583e4,glance_metadata=<?>,group=<?>,group_id=None,host='hostgroup@tripleo_ceph#tripleo_ceph',id=6796aff7-2711-41a5-bb39-618988f68d60,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='c94c3887b70d4721a11be22c540925a0',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-07-12T11:14:04Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=<?>,source_volid=e9db96d7-835d-4418-b6e3-ae6530ef811e,status='creating',terminated_at=None,updated_at=2022-07-12T11:14:04Z,user_id='32fbace38bb44ef2b10d637cbcc96985',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(5d17ed68-f3b5-4105-871c-75ec97d281a8),volume_type_id=5d17ed68-f3b5-4105-871c-75ec97d281a8)} |__Flow 'volume_create_manager': os_brick.exception.ExceptionChainer: Chained Exception #1 Traceback (most recent call last): File "/usr/lib/python3.9/site-packages/cinder/volume/driver.py", line 510, in _detach_volume connector.disconnect_volume(attach_info['conn']['data'], File "/usr/lib/python3.9/site-packages/os_brick/utils.py", line 159, in trace_logging_wrapper result = f(*args, **kwargs) File "/usr/lib/python3.9/site-packages/os_brick/utils.py", line 331, in change_encrypted devpath = _device_path_from_symlink(symlink) File "/usr/lib/python3.9/site-packages/os_brick/utils.py", line 226, in _device_path_from_symlink if symlink and symlink.startswith(CUSTOM_LINK_PREFIX): AttributeError: 'RBDVolumeIOWrapper' object has no attribute 'startswith' 2022-07-12 11:14:08.684 16978 ERROR cinder.volume.manager Traceback (most recent call last): 2022-07-12 11:14:08.684 16978 ERROR cinder.volume.manager File "/usr/lib/python3.9/site-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task 2022-07-12 11:14:08.684 16978 ERROR cinder.volume.manager result = task.execute(**arguments) 2022-07-12 11:14:08.684 16978 ERROR cinder.volume.manager File "/usr/lib/python3.9/site-packages/cinder/volume/flows/manager/create_volume.py", line 1152, in execute 2022-07-12 11:14:08.684 16978 ERROR cinder.volume.manager model_update = self._create_from_source_volume( 2022-07-12 11:14:08.684 16978 ERROR cinder.volume.manager File "/usr/lib/python3.9/site-packages/cinder/volume/flows/manager/create_volume.py", line 637, in _create_from_source_volume 2022-07-12 11:14:08.684 16978 ERROR cinder.volume.manager rekey_model_update = self._rekey_volume(context, volume) 2022-07-12 11:14:08.684 16978 ERROR cinder.volume.manager File "/usr/lib/python3.9/site-packages/cinder/volume/flows/manager/create_volume.py", line 614, in _rekey_volume 2022-07-12 11:14:08.684 16978 ERROR cinder.volume.manager self.driver._detach_volume(context, 2022-07-12 11:14:08.684 16978 ERROR cinder.volume.manager File "/usr/lib/python3.9/site-packages/cinder/volume/driver.py", line 547, in _detach_volume 2022-07-12 11:14:08.684 16978 ERROR cinder.volume.manager raise exc 2022-07-12 11:14:08.684 16978 ERROR cinder.volume.manager os_brick.exception.ExceptionChainer: 2022-07-12 11:14:08.684 16978 ERROR cinder.volume.manager Chained Exception #1 2022-07-12 11:14:08.684 16978 ERROR cinder.volume.manager Traceback (most recent call last): 2022-07-12 11:14:08.684 16978 ERROR cinder.volume.manager File "/usr/lib/python3.9/site-packages/cinder/volume/driver.py", line 510, in _detach_volume 2022-07-12 11:14:08.684 16978 ERROR cinder.volume.manager connector.disconnect_volume(attach_info['conn']['data'], 2022-07-12 11:14:08.684 16978 ERROR cinder.volume.manager File "/usr/lib/python3.9/site-packages/os_brick/utils.py", line 159, in trace_logging_wrapper 2022-07-12 11:14:08.684 16978 ERROR cinder.volume.manager result = f(*args, **kwargs) 2022-07-12 11:14:08.684 16978 ERROR cinder.volume.manager File "/usr/lib/python3.9/site-packages/os_brick/utils.py", line 331, in change_encrypted 2022-07-12 11:14:08.684 16978 ERROR cinder.volume.manager devpath = _device_path_from_symlink(symlink) 2022-07-12 11:14:08.684 16978 ERROR cinder.volume.manager File "/usr/lib/python3.9/site-packages/os_brick/utils.py", line 226, in _device_path_from_symlink 2022-07-12 11:14:08.684 16978 ERROR cinder.volume.manager if symlink and symlink.startswith(CUSTOM_LINK_PREFIX): 2022-07-12 11:14:08.684 16978 ERROR cinder.volume.manager AttributeError: 'RBDVolumeIOWrapper' object has no attribute 'startswith' 2022-07-12 11:14:08.684 16978 ERROR cinder.volume.manager 2022-07-12 11:14:08.684 16978 ERROR cinder.volume.manager 2022-07-12 11:14:08.690 16978 DEBUG cinder.volume.manager [req-9b5dc3a8-40eb-40a4-a3b9-cb8835230e05 32fbace38bb44ef2b10d637cbcc96985 c94c3887b70d4721a11be22c540925a0 - - -] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (cc461218-d6fe-418e-ba7f-46b343fecca4) transitioned into state 'REVERTING' from state 'FAILURE' _task_receiver /usr/lib/python3.9/site-packages/taskflow/listeners/logging.py:190 2022-07-12 11:14:08.692 16978 WARNING cinder.volume.manager [req-9b5dc3a8-40eb-40a4-a3b9-cb8835230e05 32fbace38bb44ef2b10d637cbcc96985 c94c3887b70d4721a11be22c540925a0 - - -] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (cc461218-d6fe-418e-ba7f-46b343fecca4) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None' 2022-07-12 11:14:08.693 16978 DEBUG cinder.volume.manager [req-9b5dc3a8-40eb-40a4-a3b9-cb8835230e05 32fbace38bb44ef2b10d637cbcc96985 c94c3887b70d4721a11be22c540925a0 - - -] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (05a38551-8186-4537-9fb5-46baa6c97cdc) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python3.9/site-packages/taskflow/listeners/logging.py:190 2022-07-12 11:14:08.695 16978 WARNING cinder.volume.manager [req-9b5dc3a8-40eb-40a4-a3b9-cb8835230e05 32fbace38bb44ef2b10d637cbcc96985 c94c3887b70d4721a11be22c540925a0 - - -] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (05a38551-8186-4537-9fb5-46baa6c97cdc) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None' 2022-07-12 11:14:08.696 16978 DEBUG cinder.volume.manager [req-9b5dc3a8-40eb-40a4-a3b9-cb8835230e05 32fbace38bb44ef2b10d637cbcc96985 c94c3887b70d4721a11be22c540925a0 - - -] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (78e91962-e41b-421b-878d-02b16aaf2032) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python3.9/site-packages/taskflow/listeners/logging.py:190 2022-07-12 11:14:08.697 16978 WARNING cinder.volume.manager [req-9b5dc3a8-40eb-40a4-a3b9-cb8835230e05 32fbace38bb44ef2b10d637cbcc96985 c94c3887b70d4721a11be22c540925a0 - - -] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (78e91962-e41b-421b-878d-02b16aaf2032) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None' 2022-07-12 11:14:08.699 16978 DEBUG cinder.volume.manager [req-9b5dc3a8-40eb-40a4-a3b9-cb8835230e05 32fbace38bb44ef2b10d637cbcc96985 c94c3887b70d4721a11be22c540925a0 - - -] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (a47a2a92-b943-40b4-ae41-9255cb5df6f6) transitioned into state 'REVERTING' from state 'SUCCESS' _task_receiver /usr/lib/python3.9/site-packages/taskflow/listeners/logging.py:190 2022-07-12 11:14:08.700 16978 DEBUG cinder.volume.flows.manager.create_volume [req-9b5dc3a8-40eb-40a4-a3b9-cb8835230e05 32fbace38bb44ef2b10d637cbcc96985 c94c3887b70d4721a11be22c540925a0 - - -] Updating volume 6796aff7-2711-41a5-bb39-618988f68d60 with {'scheduled_at': datetime.datetime(2022, 7, 12, 11, 14, 8, 700003), 'host': None}. _pre_reschedule /usr/lib/python3.9/site-packages/cinder/volume/flows/manager/create_volume.py:133 2022-07-12 11:14:08.711 16978 DEBUG cinder.volume.flows.manager.create_volume [req-9b5dc3a8-40eb-40a4-a3b9-cb8835230e05 32fbace38bb44ef2b10d637cbcc96985 c94c3887b70d4721a11be22c540925a0 - - -] Volume 6796aff7-2711-41a5-bb39-618988f68d60: re-scheduling SchedulerAPI.create_volume attempt 1 due to Chained Exception #1 Traceback (most recent call last): File "/usr/lib/python3.9/site-packages/cinder/volume/driver.py", line 510, in _detach_volume connector.disconnect_volume(attach_info['conn']['data'], File "/usr/lib/python3.9/site-packages/os_brick/utils.py", line 159, in trace_logging_wrapper result = f(*args, **kwargs) File "/usr/lib/python3.9/site-packages/os_brick/utils.py", line 331, in change_encrypted devpath = _device_path_from_symlink(symlink) File "/usr/lib/python3.9/site-packages/os_brick/utils.py", line 226, in _device_path_from_symlink if symlink and symlink.startswith(CUSTOM_LINK_PREFIX): AttributeError: 'RBDVolumeIOWrapper' object has no attribute 'startswith' _reschedule /usr/lib/python3.9/site-packages/cinder/volume/flows/manager/create_volume.py:156 Version-Release number of selected component (if applicable):